Jul 06 15:31:35.947526 ip-10-0-129-154 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Jul 06 15:31:35.947541 ip-10-0-129-154 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Jul 06 15:31:35.947552 ip-10-0-129-154 systemd[1]: kubelet.service: Failed with result 'resources'. Jul 06 15:31:35.947765 ip-10-0-129-154 systemd[1]: Failed to start Kubernetes Kubelet. Jul 06 15:31:46.195883 ip-10-0-129-154 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found. Jul 06 15:31:46.195899 ip-10-0-129-154 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot ed7cbf23a8f743418fa129e776b70ca1 -- Jul 06 15:33:42.002533 ip-10-0-129-154 systemd[1]: Starting Kubernetes Kubelet... Jul 06 15:33:42.335468 ip-10-0-129-154 kubenswrapper[2563]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 06 15:33:42.335468 ip-10-0-129-154 kubenswrapper[2563]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 06 15:33:42.335468 ip-10-0-129-154 kubenswrapper[2563]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 06 15:33:42.335468 ip-10-0-129-154 kubenswrapper[2563]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 06 15:33:42.335468 ip-10-0-129-154 kubenswrapper[2563]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 06 15:33:42.336214 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:42.336123 2563 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 06 15:33:42.340485 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.340465 2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 06 15:33:42.340485 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.340481 2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 06 15:33:42.340485 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.340487 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 06 15:33:42.340485 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.340490 2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 06 15:33:42.340485 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.340494 2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 06 15:33:42.340769 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.340499 2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 06 15:33:42.340769 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.340503 2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 06 15:33:42.340769 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.340507 2563 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 06 15:33:42.340769 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.340511 2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 06 15:33:42.340769 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.340515 2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 06 15:33:42.340769 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.340518 2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 06 15:33:42.340769 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.340521 2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 06 15:33:42.340769 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.340532 2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 06 15:33:42.340769 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.340536 2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 06 15:33:42.340769 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.340541 2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 06 15:33:42.340769 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.340544 2563 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 06 15:33:42.340769 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.340548 2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 06 15:33:42.340769 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.340552 2563 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 06 15:33:42.340769 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.340556 2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 06 15:33:42.340769 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.340559 2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 06 15:33:42.340769 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.340563 2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 06 15:33:42.340769 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.340567 2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 06 15:33:42.340769 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.340570 2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 06 15:33:42.340769 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.340575 2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 06 15:33:42.340769 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.340578 2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 06 15:33:42.341607 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.340582 2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 06 15:33:42.341607 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.340586 2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 06 15:33:42.341607 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.340589 2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 06 15:33:42.341607 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.340594 2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 06 15:33:42.341607 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.340597 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 06 15:33:42.341607 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.340602 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 06 15:33:42.341607 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.340606 2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 06 15:33:42.341607 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.340610 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 06 15:33:42.341607 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.340614 2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 06 15:33:42.341607 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.340618 2563 feature_gate.go:328] unrecognized feature gate: Example2 Jul 06 15:33:42.341607 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.340622 2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 06 15:33:42.341607 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.340627 2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 06 15:33:42.341607 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.340631 2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 06 15:33:42.341607 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.340635 2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 06 15:33:42.341607 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.340639 2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 06 15:33:42.341607 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.340643 2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 06 15:33:42.341607 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.340647 2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 06 15:33:42.341607 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.340651 2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 06 15:33:42.341607 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.340656 2563 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 06 15:33:42.341607 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.340660 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 06 15:33:42.342456 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.340665 2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 06 15:33:42.342456 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.340669 2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 06 15:33:42.342456 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.340673 2563 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 06 15:33:42.342456 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.340677 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 06 15:33:42.342456 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.340681 2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 06 15:33:42.342456 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.340685 2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 06 15:33:42.342456 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.340689 2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 06 15:33:42.342456 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.340693 2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 06 15:33:42.342456 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.340697 2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 06 15:33:42.342456 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.340702 2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 06 15:33:42.342456 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.340707 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 06 15:33:42.342456 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.340711 2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 06 15:33:42.342456 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.340715 2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 06 15:33:42.342456 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.340719 2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 06 15:33:42.342456 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.340724 2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 06 15:33:42.342456 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.340728 2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 06 15:33:42.342456 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.340732 2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 06 15:33:42.342456 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.340739 2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 06 15:33:42.342456 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.340745 2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 06 15:33:42.342456 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.340750 2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 06 15:33:42.343300 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.340756 2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 06 15:33:42.343300 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.340760 2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 06 15:33:42.343300 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.340767 2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 06 15:33:42.343300 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.340772 2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 06 15:33:42.343300 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.340777 2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 06 15:33:42.343300 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.340781 2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 06 15:33:42.343300 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.340787 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 06 15:33:42.343300 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.340792 2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 06 15:33:42.343300 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.340798 2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 06 15:33:42.343300 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.340804 2563 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 06 15:33:42.343300 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.340808 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 06 15:33:42.343300 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.340813 2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 06 15:33:42.343300 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.340817 2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 06 15:33:42.343300 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.340822 2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 06 15:33:42.343300 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.340826 2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 06 15:33:42.343300 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.340831 2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 06 15:33:42.343300 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.340835 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 06 15:33:42.343300 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.340839 2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 06 15:33:42.343300 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.340844 2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 06 15:33:42.344108 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.340848 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 06 15:33:42.344108 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.340854 2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 06 15:33:42.344108 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.340861 2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 06 15:33:42.344108 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.340866 2563 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 06 15:33:42.344108 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.340870 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 06 15:33:42.344108 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.340874 2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 06 15:33:42.344108 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.340879 2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 06 15:33:42.344108 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.340883 2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 06 15:33:42.344108 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.340887 2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 06 15:33:42.344108 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.340892 2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 06 15:33:42.344108 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.340896 2563 feature_gate.go:328] unrecognized feature gate: Example Jul 06 15:33:42.344108 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.340901 2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 06 15:33:42.344108 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.340905 2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 06 15:33:42.344108 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.340909 2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 06 15:33:42.344108 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.340914 2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 06 15:33:42.344108 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.342098 2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 06 15:33:42.344108 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.342107 2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 06 15:33:42.344108 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.342112 2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 06 15:33:42.344108 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.342117 2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 06 15:33:42.344108 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.342121 2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 06 15:33:42.344971 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.342127 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 06 15:33:42.344971 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.342131 2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 06 15:33:42.344971 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.342135 2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 06 15:33:42.344971 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.342139 2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 06 15:33:42.344971 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.342143 2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 06 15:33:42.344971 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.342147 2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 06 15:33:42.344971 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.342151 2563 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 06 15:33:42.344971 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.342155 2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 06 15:33:42.344971 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.342159 2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 06 15:33:42.344971 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.342164 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 06 15:33:42.344971 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.342168 2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 06 15:33:42.344971 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.342172 2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 06 15:33:42.344971 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.342176 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 06 15:33:42.344971 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.342181 2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 06 15:33:42.344971 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.342185 2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 06 15:33:42.344971 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.342189 2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 06 15:33:42.344971 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.342193 2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 06 15:33:42.344971 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.342197 2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 06 15:33:42.344971 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.342202 2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 06 15:33:42.344971 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.342206 2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 06 15:33:42.345583 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.342210 2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 06 15:33:42.345583 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.342214 2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 06 15:33:42.345583 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.342219 2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 06 15:33:42.345583 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.342223 2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 06 15:33:42.345583 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.342227 2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 06 15:33:42.345583 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.342232 2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 06 15:33:42.345583 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.342236 2563 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 06 15:33:42.345583 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.342240 2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 06 15:33:42.345583 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.342244 2563 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 06 15:33:42.345583 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.342248 2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 06 15:33:42.345583 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.342274 2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 06 15:33:42.345583 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.342278 2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 06 15:33:42.345583 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.342282 2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 06 15:33:42.345583 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.342291 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 06 15:33:42.345583 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.342296 2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 06 15:33:42.345583 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.342300 2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 06 15:33:42.345583 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.342304 2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 06 15:33:42.345583 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.342311 2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 06 15:33:42.345583 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.342316 2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 06 15:33:42.346045 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.342320 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 06 15:33:42.346045 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.342324 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 06 15:33:42.346045 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.342328 2563 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 06 15:33:42.346045 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.342332 2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 06 15:33:42.346045 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.342338 2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 06 15:33:42.346045 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.342345 2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 06 15:33:42.346045 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.342351 2563 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 06 15:33:42.346045 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.342358 2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 06 15:33:42.346045 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.342364 2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 06 15:33:42.346045 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.342368 2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 06 15:33:42.346045 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.342373 2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 06 15:33:42.346045 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.342378 2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 06 15:33:42.346045 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.342382 2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 06 15:33:42.346045 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.342386 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 06 15:33:42.346045 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.342390 2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 06 15:33:42.346045 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.342395 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 06 15:33:42.346045 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.342400 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 06 15:33:42.346045 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.342404 2563 feature_gate.go:328] unrecognized feature gate: Example Jul 06 15:33:42.346045 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.342408 2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 06 15:33:42.346521 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.342412 2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 06 15:33:42.346521 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.342417 2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 06 15:33:42.346521 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.342421 2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 06 15:33:42.346521 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.342425 2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 06 15:33:42.346521 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.342429 2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 06 15:33:42.346521 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.342433 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 06 15:33:42.346521 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.342438 2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 06 15:33:42.346521 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.342442 2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 06 15:33:42.346521 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.342449 2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 06 15:33:42.346521 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.342454 2563 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 06 15:33:42.346521 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.342458 2563 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 06 15:33:42.346521 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.342461 2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 06 15:33:42.346521 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.342465 2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 06 15:33:42.346521 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.342469 2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 06 15:33:42.346521 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.342473 2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 06 15:33:42.346521 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.342477 2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 06 15:33:42.346521 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.342481 2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 06 15:33:42.346521 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.342485 2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 06 15:33:42.346521 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.342489 2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 06 15:33:42.346521 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.342494 2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 06 15:33:42.346998 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.342498 2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 06 15:33:42.346998 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.342503 2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 06 15:33:42.346998 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.342507 2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 06 15:33:42.346998 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.342512 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 06 15:33:42.346998 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.342516 2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 06 15:33:42.346998 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.342520 2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 06 15:33:42.346998 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.342524 2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 06 15:33:42.346998 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.342528 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 06 15:33:42.346998 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.342532 2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 06 15:33:42.346998 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.342536 2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 06 15:33:42.346998 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.342541 2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 06 15:33:42.346998 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.342545 2563 feature_gate.go:328] unrecognized feature gate: Example2 Jul 06 15:33:42.346998 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.342549 2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 06 15:33:42.346998 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.342554 2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 06 15:33:42.346998 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.342558 2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 06 15:33:42.346998 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.342562 2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 06 15:33:42.346998 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.343154 2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 06 15:33:42.346998 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.343163 2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 06 15:33:42.346998 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.343168 2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 06 15:33:42.347498 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.343173 2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 06 15:33:42.347498 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.343178 2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 06 15:33:42.347498 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.343182 2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 06 15:33:42.347498 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.343186 2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 06 15:33:42.347498 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.343191 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 06 15:33:42.347498 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.343195 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 06 15:33:42.347498 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.343199 2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 06 15:33:42.347498 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.343203 2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 06 15:33:42.347498 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.343208 2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 06 15:33:42.347498 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.343211 2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 06 15:33:42.347498 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.343216 2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 06 15:33:42.347498 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.343220 2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 06 15:33:42.347498 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.343226 2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 06 15:33:42.347498 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.343230 2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 06 15:33:42.347498 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.343234 2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 06 15:33:42.347498 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.343238 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 06 15:33:42.347498 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.343242 2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 06 15:33:42.347498 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.343246 2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 06 15:33:42.347498 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.343273 2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 06 15:33:42.347956 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.343279 2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 06 15:33:42.347956 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.343284 2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 06 15:33:42.347956 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.343288 2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 06 15:33:42.347956 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.343293 2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 06 15:33:42.347956 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.343297 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 06 15:33:42.347956 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.343300 2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 06 15:33:42.347956 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.343304 2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 06 15:33:42.347956 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.343308 2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 06 15:33:42.347956 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.343311 2563 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 06 15:33:42.347956 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.343315 2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 06 15:33:42.347956 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.343319 2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 06 15:33:42.347956 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.343327 2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 06 15:33:42.347956 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.343331 2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 06 15:33:42.347956 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.343335 2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 06 15:33:42.347956 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.343339 2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 06 15:33:42.347956 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.343344 2563 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 06 15:33:42.347956 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.343348 2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 06 15:33:42.347956 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.343352 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 06 15:33:42.347956 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.343356 2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 06 15:33:42.347956 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.343360 2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 06 15:33:42.348493 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.343365 2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 06 15:33:42.348493 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.343370 2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 06 15:33:42.348493 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.343376 2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 06 15:33:42.348493 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.343380 2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 06 15:33:42.348493 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.343384 2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 06 15:33:42.348493 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.343389 2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 06 15:33:42.348493 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.343393 2563 feature_gate.go:328] unrecognized feature gate: Example Jul 06 15:33:42.348493 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.343398 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 06 15:33:42.348493 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.343402 2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 06 15:33:42.348493 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.343407 2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 06 15:33:42.348493 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.343411 2563 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 06 15:33:42.348493 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.343415 2563 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 06 15:33:42.348493 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.343419 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 06 15:33:42.348493 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.343423 2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 06 15:33:42.348493 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.343427 2563 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 06 15:33:42.348493 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.343431 2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 06 15:33:42.348493 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.343435 2563 feature_gate.go:328] unrecognized feature gate: Example2 Jul 06 15:33:42.348493 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.343441 2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 06 15:33:42.348493 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.343445 2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 06 15:33:42.348493 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.343449 2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 06 15:33:42.348970 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.343453 2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 06 15:33:42.348970 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.343457 2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 06 15:33:42.348970 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.343461 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 06 15:33:42.348970 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.343467 2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 06 15:33:42.348970 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.343472 2563 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 06 15:33:42.348970 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.343476 2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 06 15:33:42.348970 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.343480 2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 06 15:33:42.348970 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.343484 2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 06 15:33:42.348970 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.343489 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 06 15:33:42.348970 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.343493 2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 06 15:33:42.348970 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.343497 2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 06 15:33:42.348970 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.343501 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 06 15:33:42.348970 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.343505 2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 06 15:33:42.348970 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.343509 2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 06 15:33:42.348970 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.343514 2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 06 15:33:42.348970 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.343518 2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 06 15:33:42.348970 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.343523 2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 06 15:33:42.348970 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.343527 2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 06 15:33:42.348970 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.343531 2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 06 15:33:42.348970 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.343535 2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 06 15:33:42.349467 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.343539 2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 06 15:33:42.349467 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.343543 2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 06 15:33:42.349467 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.343547 2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 06 15:33:42.349467 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.343551 2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 06 15:33:42.349467 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.343555 2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 06 15:33:42.349467 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.343559 2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 06 15:33:42.349467 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.343563 2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 06 15:33:42.349467 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.343569 2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 06 15:33:42.349467 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.343575 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 06 15:33:42.349467 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.343579 2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 06 15:33:42.349467 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.343584 2563 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 06 15:33:42.349467 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.343587 2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 06 15:33:42.349467 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.343591 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 06 15:33:42.349467 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.343596 2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 06 15:33:42.349467 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.343630 2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 06 15:33:42.349467 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.343639 2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 06 15:33:42.349467 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.343643 2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 06 15:33:42.349467 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:42.343743 2563 flags.go:64] FLAG: --address="0.0.0.0" Jul 06 15:33:42.349467 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:42.343768 2563 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 06 15:33:42.349467 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:42.343780 2563 flags.go:64] FLAG: --anonymous-auth="true" Jul 06 15:33:42.349976 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:42.343787 2563 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 06 15:33:42.349976 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:42.343795 2563 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 06 15:33:42.349976 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:42.343800 2563 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 06 15:33:42.349976 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:42.343807 2563 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 06 15:33:42.349976 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:42.343813 2563 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 06 15:33:42.349976 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:42.343818 2563 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 06 15:33:42.349976 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:42.343823 2563 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 06 15:33:42.349976 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:42.343828 2563 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 06 15:33:42.349976 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:42.343835 2563 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 06 15:33:42.349976 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:42.343840 2563 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 06 15:33:42.349976 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:42.343845 2563 flags.go:64] FLAG: --cgroup-root="" Jul 06 15:33:42.349976 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:42.343849 2563 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 06 15:33:42.349976 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:42.343854 2563 flags.go:64] FLAG: --client-ca-file="" Jul 06 15:33:42.349976 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:42.343858 2563 flags.go:64] FLAG: --cloud-provider="external" Jul 06 15:33:42.349976 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:42.343862 2563 flags.go:64] FLAG: --cluster-dns="[]" Jul 06 15:33:42.349976 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:42.343869 2563 flags.go:64] FLAG: --cluster-domain="" Jul 06 15:33:42.349976 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:42.343874 2563 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 06 15:33:42.349976 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:42.343879 2563 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 06 15:33:42.349976 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:42.343884 2563 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 06 15:33:42.349976 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:42.343889 2563 flags.go:64] FLAG: --container-log-max-files="5" Jul 06 15:33:42.349976 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:42.343894 2563 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 06 15:33:42.349976 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:42.343899 2563 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 06 15:33:42.349976 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:42.343904 2563 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 06 15:33:42.350536 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:42.343909 2563 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 06 15:33:42.350536 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:42.343914 2563 flags.go:64] FLAG: --contention-profiling="false" Jul 06 15:33:42.350536 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:42.343918 2563 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 06 15:33:42.350536 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:42.343923 2563 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 06 15:33:42.350536 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:42.343929 2563 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 06 15:33:42.350536 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:42.343933 2563 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 06 15:33:42.350536 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:42.343940 2563 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 06 15:33:42.350536 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:42.343945 2563 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 06 15:33:42.350536 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:42.343949 2563 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 06 15:33:42.350536 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:42.343954 2563 flags.go:64] FLAG: --enable-load-reader="false" Jul 06 15:33:42.350536 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:42.343958 2563 flags.go:64] FLAG: --enable-server="true" Jul 06 15:33:42.350536 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:42.343964 2563 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 06 15:33:42.350536 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:42.343970 2563 flags.go:64] FLAG: --event-burst="100" Jul 06 15:33:42.350536 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:42.343975 2563 flags.go:64] FLAG: --event-qps="50" Jul 06 15:33:42.350536 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:42.343980 2563 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 06 15:33:42.350536 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:42.343985 2563 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 06 15:33:42.350536 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:42.343989 2563 flags.go:64] FLAG: --eviction-hard="" Jul 06 15:33:42.350536 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:42.343997 2563 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 06 15:33:42.350536 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:42.344002 2563 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 06 15:33:42.350536 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:42.344007 2563 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 06 15:33:42.350536 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:42.344012 2563 flags.go:64] FLAG: --eviction-soft="" Jul 06 15:33:42.350536 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:42.344016 2563 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 06 15:33:42.350536 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:42.344021 2563 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 06 15:33:42.350536 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:42.344025 2563 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 06 15:33:42.350536 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:42.344030 2563 flags.go:64] FLAG: --experimental-mounter-path="" Jul 06 15:33:42.351139 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:42.344035 2563 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 06 15:33:42.351139 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:42.344039 2563 flags.go:64] FLAG: --fail-swap-on="true" Jul 06 15:33:42.351139 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:42.344044 2563 flags.go:64] FLAG: --feature-gates="" Jul 06 15:33:42.351139 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:42.344050 2563 flags.go:64] FLAG: --file-check-frequency="20s" Jul 06 15:33:42.351139 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:42.344055 2563 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 06 15:33:42.351139 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:42.344068 2563 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 06 15:33:42.351139 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:42.344073 2563 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 06 15:33:42.351139 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:42.344077 2563 flags.go:64] FLAG: --healthz-port="10248" Jul 06 15:33:42.351139 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:42.344082 2563 flags.go:64] FLAG: --help="false" Jul 06 15:33:42.351139 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:42.344087 2563 flags.go:64] FLAG: --hostname-override="ip-10-0-129-154.ec2.internal" Jul 06 15:33:42.351139 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:42.344092 2563 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 06 15:33:42.351139 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:42.344099 2563 flags.go:64] FLAG: --http-check-frequency="20s" Jul 06 15:33:42.351139 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:42.344103 2563 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 06 15:33:42.351139 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:42.344108 2563 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 06 15:33:42.351139 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:42.344113 2563 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 06 15:33:42.351139 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:42.344117 2563 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 06 15:33:42.351139 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:42.344122 2563 flags.go:64] FLAG: --image-service-endpoint="" Jul 06 15:33:42.351139 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:42.344126 2563 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 06 15:33:42.351139 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:42.344130 2563 flags.go:64] FLAG: --kube-api-burst="100" Jul 06 15:33:42.351139 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:42.344145 2563 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 06 15:33:42.351139 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:42.344151 2563 flags.go:64] FLAG: --kube-api-qps="50" Jul 06 15:33:42.351139 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:42.344156 2563 flags.go:64] FLAG: --kube-reserved="" Jul 06 15:33:42.351139 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:42.344161 2563 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 06 15:33:42.351139 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:42.344165 2563 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 06 15:33:42.352168 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:42.344172 2563 flags.go:64] FLAG: --kubelet-cgroups="" Jul 06 15:33:42.352168 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:42.344176 2563 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 06 15:33:42.352168 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:42.344181 2563 flags.go:64] FLAG: --lock-file="" Jul 06 15:33:42.352168 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:42.344185 2563 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 06 15:33:42.352168 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:42.344190 2563 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 06 15:33:42.352168 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:42.344194 2563 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 06 15:33:42.352168 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:42.344203 2563 flags.go:64] FLAG: --log-json-split-stream="false" Jul 06 15:33:42.352168 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:42.344208 2563 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 06 15:33:42.352168 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:42.344213 2563 flags.go:64] FLAG: --log-text-split-stream="false" Jul 06 15:33:42.352168 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:42.344217 2563 flags.go:64] FLAG: --logging-format="text" Jul 06 15:33:42.352168 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:42.344221 2563 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 06 15:33:42.352168 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:42.344227 2563 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 06 15:33:42.352168 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:42.344231 2563 flags.go:64] FLAG: --manifest-url="" Jul 06 15:33:42.352168 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:42.344236 2563 flags.go:64] FLAG: --manifest-url-header="" Jul 06 15:33:42.352168 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:42.344243 2563 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 06 15:33:42.352168 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:42.344247 2563 flags.go:64] FLAG: --max-open-files="1000000" Jul 06 15:33:42.352168 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:42.344271 2563 flags.go:64] FLAG: --max-pods="110" Jul 06 15:33:42.352168 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:42.344276 2563 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 06 15:33:42.352168 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:42.344281 2563 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 06 15:33:42.352168 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:42.344287 2563 flags.go:64] FLAG: --memory-manager-policy="None" Jul 06 15:33:42.352168 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:42.344292 2563 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 06 15:33:42.352168 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:42.344297 2563 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 06 15:33:42.352168 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:42.344301 2563 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 06 15:33:42.352168 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:42.344306 2563 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 06 15:33:42.352168 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:42.344321 2563 flags.go:64] FLAG: --node-status-max-images="50" Jul 06 15:33:42.353235 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:42.344325 2563 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 06 15:33:42.353235 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:42.344329 2563 flags.go:64] FLAG: --oom-score-adj="-999" Jul 06 15:33:42.353235 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:42.344334 2563 flags.go:64] FLAG: --pod-cidr="" Jul 06 15:33:42.353235 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:42.344347 2563 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ce93f6b4197b9c62a26a3131f8057ffe9c9379c5079a4f62e5919b75151394e5" Jul 06 15:33:42.353235 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:42.344356 2563 flags.go:64] FLAG: --pod-manifest-path="" Jul 06 15:33:42.353235 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:42.344360 2563 flags.go:64] FLAG: --pod-max-pids="-1" Jul 06 15:33:42.353235 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:42.344365 2563 flags.go:64] FLAG: --pods-per-core="0" Jul 06 15:33:42.353235 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:42.344369 2563 flags.go:64] FLAG: --port="10250" Jul 06 15:33:42.353235 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:42.344376 2563 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 06 15:33:42.353235 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:42.344381 2563 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-027aa190a827d73c7" Jul 06 15:33:42.353235 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:42.344385 2563 flags.go:64] FLAG: --qos-reserved="" Jul 06 15:33:42.353235 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:42.344390 2563 flags.go:64] FLAG: --read-only-port="10255" Jul 06 15:33:42.353235 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:42.344395 2563 flags.go:64] FLAG: --register-node="true" Jul 06 15:33:42.353235 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:42.344399 2563 flags.go:64] FLAG: --register-with-taints="" Jul 06 15:33:42.353235 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:42.344405 2563 flags.go:64] FLAG: --registry-burst="10" Jul 06 15:33:42.353235 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:42.344409 2563 flags.go:64] FLAG: --registry-qps="5" Jul 06 15:33:42.353235 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:42.344414 2563 flags.go:64] FLAG: --reserved-cpus="" Jul 06 15:33:42.353235 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:42.344418 2563 flags.go:64] FLAG: --reserved-memory="" Jul 06 15:33:42.353235 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:42.344424 2563 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 06 15:33:42.353235 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:42.344429 2563 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 06 15:33:42.353235 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:42.344434 2563 flags.go:64] FLAG: --rotate-certificates="false" Jul 06 15:33:42.353235 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:42.344438 2563 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 06 15:33:42.353235 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:42.344443 2563 flags.go:64] FLAG: --runonce="false" Jul 06 15:33:42.353235 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:42.344448 2563 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 06 15:33:42.353235 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:42.344452 2563 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 06 15:33:42.353934 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:42.344457 2563 flags.go:64] FLAG: --seccomp-default="false" Jul 06 15:33:42.353934 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:42.344463 2563 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 06 15:33:42.353934 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:42.344468 2563 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 06 15:33:42.353934 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:42.344473 2563 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 06 15:33:42.353934 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:42.344477 2563 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 06 15:33:42.353934 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:42.344482 2563 flags.go:64] FLAG: --storage-driver-password="root" Jul 06 15:33:42.353934 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:42.344487 2563 flags.go:64] FLAG: --storage-driver-secure="false" Jul 06 15:33:42.353934 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:42.344491 2563 flags.go:64] FLAG: --storage-driver-table="stats" Jul 06 15:33:42.353934 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:42.344496 2563 flags.go:64] FLAG: --storage-driver-user="root" Jul 06 15:33:42.353934 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:42.344500 2563 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 06 15:33:42.353934 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:42.344505 2563 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 06 15:33:42.353934 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:42.344517 2563 flags.go:64] FLAG: --system-cgroups="" Jul 06 15:33:42.353934 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:42.344522 2563 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 06 15:33:42.353934 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:42.344530 2563 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 06 15:33:42.353934 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:42.344534 2563 flags.go:64] FLAG: --tls-cert-file="" Jul 06 15:33:42.353934 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:42.344541 2563 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 06 15:33:42.353934 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:42.344552 2563 flags.go:64] FLAG: --tls-min-version="" Jul 06 15:33:42.353934 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:42.344556 2563 flags.go:64] FLAG: --tls-private-key-file="" Jul 06 15:33:42.353934 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:42.344561 2563 flags.go:64] FLAG: --topology-manager-policy="none" Jul 06 15:33:42.353934 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:42.344565 2563 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 06 15:33:42.353934 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:42.344570 2563 flags.go:64] FLAG: --topology-manager-scope="container" Jul 06 15:33:42.353934 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:42.344574 2563 flags.go:64] FLAG: --v="2" Jul 06 15:33:42.353934 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:42.344581 2563 flags.go:64] FLAG: --version="false" Jul 06 15:33:42.353934 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:42.344587 2563 flags.go:64] FLAG: --vmodule="" Jul 06 15:33:42.353934 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:42.344594 2563 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 06 15:33:42.354942 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:42.344599 2563 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 06 15:33:42.354942 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.344804 2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 06 15:33:42.354942 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.344812 2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 06 15:33:42.354942 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.344817 2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 06 15:33:42.354942 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.344822 2563 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 06 15:33:42.354942 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.344826 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 06 15:33:42.354942 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.344830 2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 06 15:33:42.354942 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.344834 2563 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 06 15:33:42.354942 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.344840 2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 06 15:33:42.354942 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.344844 2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 06 15:33:42.354942 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.344848 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 06 15:33:42.354942 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.344853 2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 06 15:33:42.354942 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.344857 2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 06 15:33:42.354942 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.344861 2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 06 15:33:42.354942 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.344866 2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 06 15:33:42.354942 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.344870 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 06 15:33:42.354942 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.344874 2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 06 15:33:42.354942 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.344878 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 06 15:33:42.354942 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.344882 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 06 15:33:42.354942 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.344894 2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 06 15:33:42.355714 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.344899 2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 06 15:33:42.355714 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.344904 2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 06 15:33:42.355714 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.344909 2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 06 15:33:42.355714 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.344913 2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 06 15:33:42.355714 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.344917 2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 06 15:33:42.355714 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.344922 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 06 15:33:42.355714 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.344926 2563 feature_gate.go:328] unrecognized feature gate: Example Jul 06 15:33:42.355714 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.344930 2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 06 15:33:42.355714 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.344934 2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 06 15:33:42.355714 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.344938 2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 06 15:33:42.355714 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.344943 2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 06 15:33:42.355714 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.344947 2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 06 15:33:42.355714 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.344952 2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 06 15:33:42.355714 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.344956 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 06 15:33:42.355714 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.344959 2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 06 15:33:42.355714 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.344963 2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 06 15:33:42.355714 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.344967 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 06 15:33:42.355714 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.344971 2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 06 15:33:42.355714 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.344975 2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 06 15:33:42.355714 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.344979 2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 06 15:33:42.356396 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.344984 2563 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 06 15:33:42.356396 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.344988 2563 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 06 15:33:42.356396 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.344992 2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 06 15:33:42.356396 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.344997 2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 06 15:33:42.356396 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.345000 2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 06 15:33:42.356396 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.345005 2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 06 15:33:42.356396 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.345009 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 06 15:33:42.356396 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.345013 2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 06 15:33:42.356396 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.345017 2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 06 15:33:42.356396 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.345021 2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 06 15:33:42.356396 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.345025 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 06 15:33:42.356396 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.345029 2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 06 15:33:42.356396 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.345034 2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 06 15:33:42.356396 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.345038 2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 06 15:33:42.356396 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.345043 2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 06 15:33:42.356396 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.345047 2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 06 15:33:42.356396 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.345051 2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 06 15:33:42.356396 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.345055 2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 06 15:33:42.356396 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.345059 2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 06 15:33:42.357330 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.345065 2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 06 15:33:42.357330 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.345070 2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 06 15:33:42.357330 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.345074 2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 06 15:33:42.357330 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.345079 2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 06 15:33:42.357330 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.345083 2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 06 15:33:42.357330 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.345087 2563 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 06 15:33:42.357330 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.345091 2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 06 15:33:42.357330 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.345095 2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 06 15:33:42.357330 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.345099 2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 06 15:33:42.357330 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.345103 2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 06 15:33:42.357330 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.345107 2563 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 06 15:33:42.357330 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.345111 2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 06 15:33:42.357330 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.345115 2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 06 15:33:42.357330 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.345122 2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 06 15:33:42.357330 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.345126 2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 06 15:33:42.357330 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.345130 2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 06 15:33:42.357330 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.345134 2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 06 15:33:42.357330 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.345138 2563 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 06 15:33:42.357330 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.345142 2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 06 15:33:42.357330 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.345147 2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 06 15:33:42.357896 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.345153 2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 06 15:33:42.357896 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.345181 2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 06 15:33:42.357896 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.345205 2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 06 15:33:42.357896 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.345210 2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 06 15:33:42.357896 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.345213 2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 06 15:33:42.357896 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.345219 2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 06 15:33:42.357896 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.345222 2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 06 15:33:42.357896 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.345226 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 06 15:33:42.357896 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.345229 2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 06 15:33:42.357896 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.345232 2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 06 15:33:42.357896 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.345235 2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 06 15:33:42.357896 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.345238 2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 06 15:33:42.357896 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.345241 2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 06 15:33:42.357896 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.345246 2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 06 15:33:42.357896 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.345251 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 06 15:33:42.357896 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.345271 2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 06 15:33:42.357896 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.345277 2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 06 15:33:42.357896 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.345280 2563 feature_gate.go:328] unrecognized feature gate: Example2 Jul 06 15:33:42.357896 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.345283 2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 06 15:33:42.358485 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.345285 2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 06 15:33:42.358485 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.345288 2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 06 15:33:42.358485 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:42.346031 2563 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 06 15:33:42.358485 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:42.353392 2563 server.go:529] "Kubelet version" kubeletVersion="v1.34.8" Jul 06 15:33:42.358485 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:42.353408 2563 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 06 15:33:42.358485 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:42.354119 2563 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 06 15:33:42.358485 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:42.354130 2563 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 06 15:33:42.358485 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.354211 2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 06 15:33:42.358485 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.354221 2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 06 15:33:42.358485 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.354225 2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 06 15:33:42.358485 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.354230 2563 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 06 15:33:42.358485 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.354234 2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 06 15:33:42.358485 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.354239 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 06 15:33:42.358485 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.354243 2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 06 15:33:42.358865 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.354247 2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 06 15:33:42.358865 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.354251 2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 06 15:33:42.358865 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.354274 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 06 15:33:42.358865 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.354279 2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 06 15:33:42.358865 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.354283 2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 06 15:33:42.358865 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.354287 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 06 15:33:42.358865 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.354291 2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 06 15:33:42.358865 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.354296 2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 06 15:33:42.358865 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.354300 2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 06 15:33:42.358865 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.354304 2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 06 15:33:42.358865 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.354309 2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 06 15:33:42.358865 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.354313 2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 06 15:33:42.358865 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.354316 2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 06 15:33:42.358865 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.354324 2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 06 15:33:42.358865 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.354330 2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 06 15:33:42.358865 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.354335 2563 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 06 15:33:42.358865 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.354339 2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 06 15:33:42.358865 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.354342 2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 06 15:33:42.358865 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.354346 2563 feature_gate.go:328] unrecognized feature gate: Example Jul 06 15:33:42.358865 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.354350 2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 06 15:33:42.359363 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.354354 2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 06 15:33:42.359363 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.354358 2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 06 15:33:42.359363 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.354363 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 06 15:33:42.359363 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.354367 2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 06 15:33:42.359363 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.354373 2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 06 15:33:42.359363 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.354378 2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 06 15:33:42.359363 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.354382 2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 06 15:33:42.359363 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.354387 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 06 15:33:42.359363 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.354391 2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 06 15:33:42.359363 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.354395 2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 06 15:33:42.359363 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.354400 2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 06 15:33:42.359363 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.354404 2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 06 15:33:42.359363 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.354408 2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 06 15:33:42.359363 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.354412 2563 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 06 15:33:42.359363 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.354416 2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 06 15:33:42.359363 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.354421 2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 06 15:33:42.359363 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.354425 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 06 15:33:42.359363 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.354429 2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 06 15:33:42.359363 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.354434 2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 06 15:33:42.359363 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.354438 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 06 15:33:42.359892 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.354442 2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 06 15:33:42.359892 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.354448 2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 06 15:33:42.359892 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.354454 2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 06 15:33:42.359892 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.354458 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 06 15:33:42.359892 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.354463 2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 06 15:33:42.359892 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.354467 2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 06 15:33:42.359892 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.354472 2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 06 15:33:42.359892 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.354476 2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 06 15:33:42.359892 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.354480 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 06 15:33:42.359892 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.354484 2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 06 15:33:42.359892 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.354488 2563 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 06 15:33:42.359892 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.354492 2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 06 15:33:42.359892 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.354496 2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 06 15:33:42.359892 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.354500 2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 06 15:33:42.359892 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.354504 2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 06 15:33:42.359892 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.354508 2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 06 15:33:42.359892 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.354512 2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 06 15:33:42.359892 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.354516 2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 06 15:33:42.359892 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.354520 2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 06 15:33:42.359892 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.354524 2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 06 15:33:42.360455 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.354530 2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 06 15:33:42.360455 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.354536 2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 06 15:33:42.360455 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.354541 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 06 15:33:42.360455 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.354545 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 06 15:33:42.360455 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.354550 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 06 15:33:42.360455 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.354555 2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 06 15:33:42.360455 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.354559 2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 06 15:33:42.360455 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.354564 2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 06 15:33:42.360455 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.354568 2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 06 15:33:42.360455 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.354572 2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 06 15:33:42.360455 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.354576 2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 06 15:33:42.360455 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.354581 2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 06 15:33:42.360455 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.354585 2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 06 15:33:42.360455 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.354589 2563 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 06 15:33:42.360455 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.354594 2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 06 15:33:42.360455 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.354598 2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 06 15:33:42.360455 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.354602 2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 06 15:33:42.360455 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.354606 2563 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 06 15:33:42.360455 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.354610 2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 06 15:33:42.360944 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.354614 2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 06 15:33:42.360944 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.354618 2563 feature_gate.go:328] unrecognized feature gate: Example2 Jul 06 15:33:42.360944 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.354622 2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 06 15:33:42.360944 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.354626 2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 06 15:33:42.360944 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.354630 2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 06 15:33:42.360944 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.354635 2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 06 15:33:42.360944 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.354639 2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 06 15:33:42.360944 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.354644 2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 06 15:33:42.360944 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.354647 2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 06 15:33:42.360944 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.354651 2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 06 15:33:42.360944 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.354655 2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 06 15:33:42.360944 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.354659 2563 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 06 15:33:42.360944 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.354664 2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 06 15:33:42.360944 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:42.354671 2563 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 06 15:33:42.360944 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.354846 2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 06 15:33:42.361350 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.354853 2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 06 15:33:42.361350 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.354858 2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 06 15:33:42.361350 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.354863 2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 06 15:33:42.361350 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.354868 2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 06 15:33:42.361350 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.354872 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 06 15:33:42.361350 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.354876 2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 06 15:33:42.361350 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.354880 2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 06 15:33:42.361350 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.354884 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 06 15:33:42.361350 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.354888 2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 06 15:33:42.361350 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.354892 2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 06 15:33:42.361350 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.354896 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 06 15:33:42.361350 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.354900 2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 06 15:33:42.361350 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.354904 2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 06 15:33:42.361350 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.354908 2563 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 06 15:33:42.361350 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.354912 2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 06 15:33:42.361350 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.354917 2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 06 15:33:42.361350 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.354921 2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 06 15:33:42.361350 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.354925 2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 06 15:33:42.361350 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.354929 2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 06 15:33:42.362124 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.354933 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 06 15:33:42.362124 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.354937 2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 06 15:33:42.362124 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.354941 2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 06 15:33:42.362124 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.354944 2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 06 15:33:42.362124 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.354948 2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 06 15:33:42.362124 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.354952 2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 06 15:33:42.362124 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.354955 2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 06 15:33:42.362124 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.354958 2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 06 15:33:42.362124 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.354964 2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 06 15:33:42.362124 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.354971 2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 06 15:33:42.362124 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.354975 2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 06 15:33:42.362124 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.354979 2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 06 15:33:42.362124 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.354982 2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 06 15:33:42.362124 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.354988 2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 06 15:33:42.362124 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.354992 2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 06 15:33:42.362124 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.354996 2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 06 15:33:42.362124 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.355000 2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 06 15:33:42.362124 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.355004 2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 06 15:33:42.362124 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.355008 2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 06 15:33:42.362915 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.355012 2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 06 15:33:42.362915 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.355017 2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 06 15:33:42.362915 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.355022 2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 06 15:33:42.362915 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.355026 2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 06 15:33:42.362915 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.355030 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 06 15:33:42.362915 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.355034 2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 06 15:33:42.362915 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.355038 2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 06 15:33:42.362915 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.355042 2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 06 15:33:42.362915 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.355046 2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 06 15:33:42.362915 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.355050 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 06 15:33:42.362915 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.355054 2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 06 15:33:42.362915 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.355058 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 06 15:33:42.362915 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.355062 2563 feature_gate.go:328] unrecognized feature gate: Example2 Jul 06 15:33:42.362915 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.355066 2563 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 06 15:33:42.362915 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.355070 2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 06 15:33:42.362915 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.355074 2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 06 15:33:42.362915 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.355078 2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 06 15:33:42.362915 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.355082 2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 06 15:33:42.362915 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.355086 2563 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 06 15:33:42.362915 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.355090 2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 06 15:33:42.363744 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.355093 2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 06 15:33:42.363744 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.355098 2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 06 15:33:42.363744 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.355102 2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 06 15:33:42.363744 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.355105 2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 06 15:33:42.363744 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.355109 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 06 15:33:42.363744 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.355113 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 06 15:33:42.363744 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.355118 2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 06 15:33:42.363744 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.355122 2563 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 06 15:33:42.363744 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.355126 2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 06 15:33:42.363744 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.355131 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 06 15:33:42.363744 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.355134 2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 06 15:33:42.363744 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.355138 2563 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 06 15:33:42.363744 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.355142 2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 06 15:33:42.363744 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.355149 2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 06 15:33:42.363744 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.355154 2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 06 15:33:42.363744 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.355158 2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 06 15:33:42.363744 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.355162 2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 06 15:33:42.363744 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.355167 2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 06 15:33:42.363744 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.355171 2563 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 06 15:33:42.363744 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.355175 2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 06 15:33:42.364241 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.355179 2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 06 15:33:42.364241 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.355183 2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 06 15:33:42.364241 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.355187 2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 06 15:33:42.364241 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.355192 2563 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 06 15:33:42.364241 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.355195 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 06 15:33:42.364241 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.355199 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 06 15:33:42.364241 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.355203 2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 06 15:33:42.364241 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.355207 2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 06 15:33:42.364241 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.355211 2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 06 15:33:42.364241 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.355215 2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 06 15:33:42.364241 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.355219 2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 06 15:33:42.364241 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.355223 2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 06 15:33:42.364241 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.355227 2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 06 15:33:42.364241 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.355231 2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 06 15:33:42.364241 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.355235 2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 06 15:33:42.364241 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.355239 2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 06 15:33:42.364241 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.355243 2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 06 15:33:42.364241 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.355250 2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 06 15:33:42.364241 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.355273 2563 feature_gate.go:328] unrecognized feature gate: Example Jul 06 15:33:42.364241 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:42.355279 2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 06 15:33:42.364863 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:42.355293 2563 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 06 15:33:42.364863 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:42.355432 2563 server.go:962] "Client rotation is on, will bootstrap in background" Jul 06 15:33:42.364863 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:42.357969 2563 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 06 15:33:42.364863 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:42.359413 2563 server.go:1019] "Starting client certificate rotation" Jul 06 15:33:42.364863 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:42.359525 2563 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 06 15:33:42.364863 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:42.359562 2563 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 06 15:33:42.364863 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:42.361933 2563 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 06 15:33:42.364863 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:42.363704 2563 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 06 15:33:42.375827 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:42.375805 2563 log.go:25] "Validated CRI v1 runtime API" Jul 06 15:33:42.377863 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:42.377846 2563 log.go:25] "Validated CRI v1 image API" Jul 06 15:33:42.379203 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:42.379186 2563 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 06 15:33:42.383966 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:42.383947 2563 fs.go:135] Filesystem UUIDs: map[5ccd7388-3f56-4a59-93ee-ef91a2fb5bfb:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 9abbbfde-a1a5-4c37-bdeb-eba5cd83c51a:/dev/nvme0n1p3] Jul 06 15:33:42.384034 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:42.383965 2563 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 06 15:33:42.389709 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:42.389603 2563 manager.go:216] Machine: {Timestamp:2026-07-06 15:33:42.388500631 +0000 UTC m=+0.276896355 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3098930 MemoryCapacity:32812171264 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2ff4aad21052475356c46fe13347d4 SystemUUID:ec2ff4aa-d210-5247-5356-c46fe13347d4 BootID:ed7cbf23-a8f7-4341-8fa1-29e776b70ca1 Filesystems:[{Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406085632 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406085632 Type:vfs Inodes:4005392 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562435072 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:77:4b:06:4d:01 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:77:4b:06:4d:01 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:5a:e0:b3:eb:3a:d2 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812171264 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 06 15:33:42.389709 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:42.389708 2563 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 06 15:33:42.389828 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:42.389807 2563 manager.go:232] Version: {KernelVersion:5.14.0-570.125.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260630-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 06 15:33:42.392495 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:42.392469 2563 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 06 15:33:42.392626 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:42.392498 2563 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-129-154.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 06 15:33:42.393158 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:42.393148 2563 topology_manager.go:138] "Creating topology manager with none policy" Jul 06 15:33:42.393187 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:42.393160 2563 container_manager_linux.go:309] "Creating device plugin manager" Jul 06 15:33:42.393187 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:42.393167 2563 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 06 15:33:42.393765 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:42.393754 2563 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 06 15:33:42.395024 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:42.395011 2563 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 06 15:33:42.397395 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:42.397378 2563 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 06 15:33:42.398951 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:42.398940 2563 state_mem.go:36] "Initialized new in-memory state store" Jul 06 15:33:42.399056 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:42.399047 2563 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 06 15:33:42.399092 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:42.399075 2563 kubelet.go:486] "Attempting to sync node with API server" Jul 06 15:33:42.399092 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:42.399085 2563 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 06 15:33:42.399150 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:42.399103 2563 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 06 15:33:42.399150 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:42.399115 2563 kubelet.go:398] "Adding apiserver pod source" Jul 06 15:33:42.399150 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:42.399127 2563 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 06 15:33:42.399538 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:42.399522 2563 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 06 15:33:42.399603 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:42.399550 2563 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 06 15:33:42.401483 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:42.401468 2563 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.8-3.rhaos4.21.git91b4b0b.el9" apiVersion="v1" Jul 06 15:33:42.403356 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:42.403340 2563 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 06 15:33:42.403438 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:42.403378 2563 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 06 15:33:42.404720 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:42.404706 2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 06 15:33:42.404792 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:42.404726 2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 06 15:33:42.404792 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:42.404761 2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 06 15:33:42.404792 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:42.404771 2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 06 15:33:42.404792 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:42.404785 2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 06 15:33:42.404952 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:42.404795 2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 06 15:33:42.404952 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:42.404813 2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 06 15:33:42.404952 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:42.404823 2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 06 15:33:42.404952 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:42.404836 2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 06 15:33:42.404952 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:42.404846 2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 06 15:33:42.404952 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:42.404867 2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 06 15:33:42.404952 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:42.404876 2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 06 15:33:42.405657 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:42.405646 2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 06 15:33:42.405705 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:42.405660 2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 06 15:33:42.408846 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:42.408830 2563 server.go:1268] "Started kubelet" Jul 06 15:33:42.408955 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:42.408930 2563 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 06 15:33:42.409045 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:42.408983 2563 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 06 15:33:42.409140 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:42.409070 2563 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 06 15:33:42.409771 ip-10-0-129-154 systemd[1]: Started Kubernetes Kubelet. Jul 06 15:33:42.410166 ip-10-0-129-154 kubenswrapper[2563]: E0706 15:33:42.410139 2563 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-129-154.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 06 15:33:42.410166 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:42.410151 2563 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 06 15:33:42.410337 ip-10-0-129-154 kubenswrapper[2563]: E0706 15:33:42.410323 2563 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 06 15:33:42.410840 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:42.410628 2563 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-129-154.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 06 15:33:42.414072 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:42.414053 2563 server.go:310] "Adding debug handlers to kubelet server" Jul 06 15:33:42.416755 ip-10-0-129-154 kubenswrapper[2563]: E0706 15:33:42.416733 2563 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 06 15:33:42.417307 ip-10-0-129-154 kubenswrapper[2563]: E0706 15:33:42.416543 2563 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-129-154.ec2.internal.18bfbd373d26b474 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-129-154.ec2.internal,UID:ip-10-0-129-154.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-129-154.ec2.internal,},FirstTimestamp:2026-07-06 15:33:42.408803444 +0000 UTC m=+0.297199167,LastTimestamp:2026-07-06 15:33:42.408803444 +0000 UTC m=+0.297199167,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-129-154.ec2.internal,}" Jul 06 15:33:42.418972 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:42.418956 2563 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 06 15:33:42.419043 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:42.418951 2563 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 06 15:33:42.419153 ip-10-0-129-154 kubenswrapper[2563]: E0706 15:33:42.419135 2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-154.ec2.internal\" not found" Jul 06 15:33:42.419495 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:42.419477 2563 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 06 15:33:42.419495 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:42.419488 2563 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 06 15:33:42.419625 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:42.419504 2563 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 06 15:33:42.419625 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:42.419583 2563 reconstruct.go:97] "Volume reconstruction finished" Jul 06 15:33:42.419625 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:42.419605 2563 reconciler.go:29] "Reconciler: start to sync state" Jul 06 15:33:42.420327 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:42.420312 2563 factory.go:153] Registering CRI-O factory Jul 06 15:33:42.420327 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:42.420329 2563 factory.go:223] Registration of the crio container factory successfully Jul 06 15:33:42.420426 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:42.420374 2563 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 06 15:33:42.420426 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:42.420381 2563 factory.go:55] Registering systemd factory Jul 06 15:33:42.420426 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:42.420386 2563 factory.go:223] Registration of the systemd container factory successfully Jul 06 15:33:42.420426 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:42.420403 2563 factory.go:103] Registering Raw factory Jul 06 15:33:42.420426 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:42.420412 2563 manager.go:1196] Started watching for new ooms in manager Jul 06 15:33:42.420863 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:42.420851 2563 manager.go:349] Starting recovery of all containers Jul 06 15:33:42.430591 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:42.430575 2563 manager.go:354] Recovery completed Jul 06 15:33:42.433323 ip-10-0-129-154 kubenswrapper[2563]: E0706 15:33:42.433298 2563 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 06 15:33:42.433323 ip-10-0-129-154 kubenswrapper[2563]: E0706 15:33:42.433315 2563 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-129-154.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 06 15:33:42.434836 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:42.434819 2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 06 15:33:42.437000 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:42.436967 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-154.ec2.internal" event="NodeHasSufficientMemory" Jul 06 15:33:42.437095 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:42.437018 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-154.ec2.internal" event="NodeHasNoDiskPressure" Jul 06 15:33:42.437095 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:42.437035 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-154.ec2.internal" event="NodeHasSufficientPID" Jul 06 15:33:42.438325 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:42.438309 2563 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 06 15:33:42.438325 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:42.438326 2563 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 06 15:33:42.438470 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:42.438345 2563 state_mem.go:36] "Initialized new in-memory state store" Jul 06 15:33:42.438470 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:42.438335 2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 06 15:33:42.439132 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:42.439119 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-154.ec2.internal" event="NodeHasSufficientMemory" Jul 06 15:33:42.439194 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:42.439146 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-154.ec2.internal" event="NodeHasNoDiskPressure" Jul 06 15:33:42.439194 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:42.439157 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-154.ec2.internal" event="NodeHasSufficientPID" Jul 06 15:33:42.440603 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:42.440590 2563 policy_none.go:49] "None policy: Start" Jul 06 15:33:42.440673 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:42.440609 2563 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 06 15:33:42.440673 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:42.440623 2563 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 06 15:33:42.441091 ip-10-0-129-154 kubenswrapper[2563]: E0706 15:33:42.441022 2563 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-129-154.ec2.internal.18bfbd373ed4f04b default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-129-154.ec2.internal,UID:ip-10-0-129-154.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-129-154.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-129-154.ec2.internal,},FirstTimestamp:2026-07-06 15:33:42.436999243 +0000 UTC m=+0.325394978,LastTimestamp:2026-07-06 15:33:42.436999243 +0000 UTC m=+0.325394978,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-129-154.ec2.internal,}" Jul 06 15:33:42.442601 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:42.442588 2563 policy_none.go:47] "Start" Jul 06 15:33:42.447991 ip-10-0-129-154 kubenswrapper[2563]: E0706 15:33:42.447929 2563 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-129-154.ec2.internal.18bfbd373ed558ea default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-129-154.ec2.internal,UID:ip-10-0-129-154.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-129-154.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-129-154.ec2.internal,},FirstTimestamp:2026-07-06 15:33:42.437026026 +0000 UTC m=+0.325421750,LastTimestamp:2026-07-06 15:33:42.437026026 +0000 UTC m=+0.325421750,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-129-154.ec2.internal,}" Jul 06 15:33:42.462592 ip-10-0-129-154 kubenswrapper[2563]: E0706 15:33:42.462509 2563 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-129-154.ec2.internal.18bfbd373ed59125 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-129-154.ec2.internal,UID:ip-10-0-129-154.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-129-154.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-129-154.ec2.internal,},FirstTimestamp:2026-07-06 15:33:42.437040421 +0000 UTC m=+0.325436148,LastTimestamp:2026-07-06 15:33:42.437040421 +0000 UTC m=+0.325436148,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-129-154.ec2.internal,}" Jul 06 15:33:42.463595 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:42.463573 2563 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-57n4g" Jul 06 15:33:42.473029 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:42.473012 2563 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-57n4g" Jul 06 15:33:42.474246 ip-10-0-129-154 kubenswrapper[2563]: E0706 15:33:42.474192 2563 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-129-154.ec2.internal.18bfbd373ed4f04b\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-129-154.ec2.internal.18bfbd373ed4f04b default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-129-154.ec2.internal,UID:ip-10-0-129-154.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-129-154.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-129-154.ec2.internal,},FirstTimestamp:2026-07-06 15:33:42.436999243 +0000 UTC m=+0.325394978,LastTimestamp:2026-07-06 15:33:42.439130536 +0000 UTC m=+0.327526261,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-129-154.ec2.internal,}" Jul 06 15:33:42.475805 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:42.475790 2563 manager.go:337] "Starting Device Plugin manager" Jul 06 15:33:42.475877 ip-10-0-129-154 kubenswrapper[2563]: E0706 15:33:42.475821 2563 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 06 15:33:42.475877 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:42.475832 2563 server.go:87] "Starting device plugin registration server" Jul 06 15:33:42.476032 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:42.476017 2563 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 06 15:33:42.476126 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:42.476031 2563 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 06 15:33:42.476215 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:42.476182 2563 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 06 15:33:42.476313 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:42.476301 2563 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 06 15:33:42.476313 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:42.476312 2563 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 06 15:33:42.476726 ip-10-0-129-154 kubenswrapper[2563]: E0706 15:33:42.476699 2563 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 06 15:33:42.476796 ip-10-0-129-154 kubenswrapper[2563]: E0706 15:33:42.476749 2563 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-129-154.ec2.internal\" not found" Jul 06 15:33:42.558408 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:42.558383 2563 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 06 15:33:42.560504 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:42.559633 2563 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 06 15:33:42.560504 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:42.559668 2563 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 06 15:33:42.560504 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:42.559689 2563 kubelet.go:2448] "Starting kubelet main sync loop" Jul 06 15:33:42.560504 ip-10-0-129-154 kubenswrapper[2563]: E0706 15:33:42.559764 2563 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 06 15:33:42.562300 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:42.562279 2563 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 06 15:33:42.576293 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:42.576277 2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 06 15:33:42.577012 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:42.576997 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-154.ec2.internal" event="NodeHasSufficientMemory" Jul 06 15:33:42.577071 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:42.577028 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-154.ec2.internal" event="NodeHasNoDiskPressure" Jul 06 15:33:42.577071 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:42.577038 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-154.ec2.internal" event="NodeHasSufficientPID" Jul 06 15:33:42.577071 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:42.577058 2563 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-129-154.ec2.internal" Jul 06 15:33:42.586472 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:42.586432 2563 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-129-154.ec2.internal" Jul 06 15:33:42.586472 ip-10-0-129-154 kubenswrapper[2563]: E0706 15:33:42.586453 2563 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-129-154.ec2.internal\": node \"ip-10-0-129-154.ec2.internal\" not found" Jul 06 15:33:42.615792 ip-10-0-129-154 kubenswrapper[2563]: E0706 15:33:42.615768 2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-154.ec2.internal\" not found" Jul 06 15:33:42.660268 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:42.660233 2563 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["kube-system/kube-apiserver-proxy-ip-10-0-129-154.ec2.internal","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-154.ec2.internal"] Jul 06 15:33:42.660339 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:42.660307 2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 06 15:33:42.661498 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:42.661483 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-154.ec2.internal" event="NodeHasSufficientMemory" Jul 06 15:33:42.661561 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:42.661510 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-154.ec2.internal" event="NodeHasNoDiskPressure" Jul 06 15:33:42.661561 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:42.661520 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-154.ec2.internal" event="NodeHasSufficientPID" Jul 06 15:33:42.663889 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:42.663876 2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 06 15:33:42.664014 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:42.664000 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-154.ec2.internal" Jul 06 15:33:42.664078 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:42.664033 2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 06 15:33:42.664585 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:42.664565 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-154.ec2.internal" event="NodeHasSufficientMemory" Jul 06 15:33:42.664663 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:42.664565 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-154.ec2.internal" event="NodeHasSufficientMemory" Jul 06 15:33:42.664663 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:42.664600 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-154.ec2.internal" event="NodeHasNoDiskPressure" Jul 06 15:33:42.664663 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:42.664611 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-154.ec2.internal" event="NodeHasSufficientPID" Jul 06 15:33:42.664663 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:42.664623 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-154.ec2.internal" event="NodeHasNoDiskPressure" Jul 06 15:33:42.664663 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:42.664635 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-154.ec2.internal" event="NodeHasSufficientPID" Jul 06 15:33:42.666820 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:42.666806 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-154.ec2.internal" Jul 06 15:33:42.666889 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:42.666832 2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 06 15:33:42.667472 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:42.667456 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-154.ec2.internal" event="NodeHasSufficientMemory" Jul 06 15:33:42.667549 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:42.667486 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-154.ec2.internal" event="NodeHasNoDiskPressure" Jul 06 15:33:42.667549 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:42.667503 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-154.ec2.internal" event="NodeHasSufficientPID" Jul 06 15:33:42.695982 ip-10-0-129-154 kubenswrapper[2563]: E0706 15:33:42.695958 2563 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-129-154.ec2.internal\" not found" node="ip-10-0-129-154.ec2.internal" Jul 06 15:33:42.700456 ip-10-0-129-154 kubenswrapper[2563]: E0706 15:33:42.700439 2563 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-129-154.ec2.internal\" not found" node="ip-10-0-129-154.ec2.internal" Jul 06 15:33:42.716829 ip-10-0-129-154 kubenswrapper[2563]: E0706 15:33:42.716809 2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-154.ec2.internal\" not found" Jul 06 15:33:42.721432 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:42.721418 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/89a933a4336ad4eeb3842cbd7d2c9e02-config\") pod \"kube-apiserver-proxy-ip-10-0-129-154.ec2.internal\" (UID: \"89a933a4336ad4eeb3842cbd7d2c9e02\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-129-154.ec2.internal" Jul 06 15:33:42.721482 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:42.721439 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/aebce7470264e8df37783ba3c605e8cd-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-129-154.ec2.internal\" (UID: \"aebce7470264e8df37783ba3c605e8cd\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-154.ec2.internal" Jul 06 15:33:42.721482 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:42.721456 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/aebce7470264e8df37783ba3c605e8cd-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-129-154.ec2.internal\" (UID: \"aebce7470264e8df37783ba3c605e8cd\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-154.ec2.internal" Jul 06 15:33:42.816890 ip-10-0-129-154 kubenswrapper[2563]: E0706 15:33:42.816857 2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-154.ec2.internal\" not found" Jul 06 15:33:42.822225 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:42.822206 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/aebce7470264e8df37783ba3c605e8cd-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-129-154.ec2.internal\" (UID: \"aebce7470264e8df37783ba3c605e8cd\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-154.ec2.internal" Jul 06 15:33:42.822303 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:42.822286 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/aebce7470264e8df37783ba3c605e8cd-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-129-154.ec2.internal\" (UID: \"aebce7470264e8df37783ba3c605e8cd\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-154.ec2.internal" Jul 06 15:33:42.822303 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:42.822294 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/aebce7470264e8df37783ba3c605e8cd-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-129-154.ec2.internal\" (UID: \"aebce7470264e8df37783ba3c605e8cd\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-154.ec2.internal" Jul 06 15:33:42.822371 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:42.822333 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/aebce7470264e8df37783ba3c605e8cd-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-129-154.ec2.internal\" (UID: \"aebce7470264e8df37783ba3c605e8cd\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-154.ec2.internal" Jul 06 15:33:42.822371 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:42.822323 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/89a933a4336ad4eeb3842cbd7d2c9e02-config\") pod \"kube-apiserver-proxy-ip-10-0-129-154.ec2.internal\" (UID: \"89a933a4336ad4eeb3842cbd7d2c9e02\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-129-154.ec2.internal" Jul 06 15:33:42.822437 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:42.822373 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/89a933a4336ad4eeb3842cbd7d2c9e02-config\") pod \"kube-apiserver-proxy-ip-10-0-129-154.ec2.internal\" (UID: \"89a933a4336ad4eeb3842cbd7d2c9e02\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-129-154.ec2.internal" Jul 06 15:33:42.917715 ip-10-0-129-154 kubenswrapper[2563]: E0706 15:33:42.917645 2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-154.ec2.internal\" not found" Jul 06 15:33:42.999153 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:42.999115 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-154.ec2.internal" Jul 06 15:33:43.002988 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:43.002971 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-154.ec2.internal" Jul 06 15:33:43.018230 ip-10-0-129-154 kubenswrapper[2563]: E0706 15:33:43.018207 2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-154.ec2.internal\" not found" Jul 06 15:33:43.118769 ip-10-0-129-154 kubenswrapper[2563]: E0706 15:33:43.118720 2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-154.ec2.internal\" not found" Jul 06 15:33:43.219328 ip-10-0-129-154 kubenswrapper[2563]: E0706 15:33:43.219221 2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-154.ec2.internal\" not found" Jul 06 15:33:43.252873 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:43.252843 2563 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 06 15:33:43.319876 ip-10-0-129-154 kubenswrapper[2563]: E0706 15:33:43.319840 2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-154.ec2.internal\" not found" Jul 06 15:33:43.359207 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:43.359181 2563 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 06 15:33:43.359867 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:43.359353 2563 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 06 15:33:43.359867 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:43.359355 2563 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 06 15:33:43.420417 ip-10-0-129-154 kubenswrapper[2563]: E0706 15:33:43.420391 2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-154.ec2.internal\" not found" Jul 06 15:33:43.469370 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:43.469325 2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 06 15:33:43.470216 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:43.470192 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-154.ec2.internal" event="NodeHasSufficientMemory" Jul 06 15:33:43.470333 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:43.470224 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-154.ec2.internal" event="NodeHasNoDiskPressure" Jul 06 15:33:43.470333 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:43.470235 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-154.ec2.internal" event="NodeHasSufficientPID" Jul 06 15:33:43.478474 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:43.478446 2563 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-05 15:28:43 +0000 UTC" deadline="2027-12-25 10:46:27.199403674 +0000 UTC" Jul 06 15:33:43.478541 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:43.478475 2563 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12883h12m43.72093278s" Jul 06 15:33:43.521110 ip-10-0-129-154 kubenswrapper[2563]: E0706 15:33:43.521072 2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-154.ec2.internal\" not found" Jul 06 15:33:43.621527 ip-10-0-129-154 kubenswrapper[2563]: E0706 15:33:43.621375 2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-154.ec2.internal\" not found" Jul 06 15:33:45.101154 ip-10-0-129-154 kubenswrapper[2563]: E0706 15:33:45.101040 2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-154.ec2.internal\" not found" Jul 06 15:33:45.119353 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:45.119315 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaebce7470264e8df37783ba3c605e8cd.slice/crio-0ae41df9185e56db90ba08e15b0e9631e4ed3a07819d63d9731b288db8a8960c WatchSource:0}: Error finding container 0ae41df9185e56db90ba08e15b0e9631e4ed3a07819d63d9731b288db8a8960c: Status 404 returned error can't find the container with id 0ae41df9185e56db90ba08e15b0e9631e4ed3a07819d63d9731b288db8a8960c Jul 06 15:33:45.119636 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:45.119610 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod89a933a4336ad4eeb3842cbd7d2c9e02.slice/crio-801ebe6e5a94f1038bef8891e837349211552bf2bfd62f9753a8032b96e21fba WatchSource:0}: Error finding container 801ebe6e5a94f1038bef8891e837349211552bf2bfd62f9753a8032b96e21fba: Status 404 returned error can't find the container with id 801ebe6e5a94f1038bef8891e837349211552bf2bfd62f9753a8032b96e21fba Jul 06 15:33:45.125650 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:45.125636 2563 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 06 15:33:45.201273 ip-10-0-129-154 kubenswrapper[2563]: E0706 15:33:45.201240 2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-154.ec2.internal\" not found" Jul 06 15:33:45.269470 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:45.269449 2563 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 06 15:33:45.299083 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:45.299058 2563 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-154.ec2.internal" Jul 06 15:33:45.310525 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:45.310504 2563 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 06 15:33:45.312191 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:45.312179 2563 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-154.ec2.internal" Jul 06 15:33:45.329861 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:45.329841 2563 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 06 15:33:45.364259 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:45.364192 2563 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 06 15:33:45.778927 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:45.778849 2563 apiserver.go:52] "Watching apiserver" Jul 06 15:33:45.787336 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:45.787311 2563 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 06 15:33:45.790180 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:45.790148 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-target-pdw4n","openshift-ovn-kubernetes/ovnkube-node-tv82v","kube-system/global-pull-secret-syncer-w9qbt","kube-system/konnectivity-agent-hq8kh","kube-system/kube-apiserver-proxy-ip-10-0-129-154.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7s97z","openshift-dns/node-resolver-pgkmz","openshift-image-registry/node-ca-rmsk7","openshift-multus/network-metrics-daemon-v8fxz","openshift-network-operator/iptables-alerter-bfx9b","openshift-cluster-node-tuning-operator/tuned-d2c7p","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-154.ec2.internal","openshift-multus/multus-8zxfb","openshift-multus/multus-additional-cni-plugins-nmfg6"] Jul 06 15:33:45.794022 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:45.794003 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-d2c7p" Jul 06 15:33:45.796133 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:45.796114 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-tv82v" Jul 06 15:33:45.796982 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:45.796959 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 06 15:33:45.797059 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:45.796958 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 06 15:33:45.797059 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:45.797022 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-lf7jx\"" Jul 06 15:33:45.798531 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:45.798515 2563 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 06 15:33:45.798673 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:45.798659 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 06 15:33:45.798801 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:45.798783 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-pgkmz" Jul 06 15:33:45.798864 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:45.798797 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-ckwgv\"" Jul 06 15:33:45.799051 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:45.799035 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 06 15:33:45.799737 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:45.799716 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 06 15:33:45.799842 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:45.799740 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 06 15:33:45.799908 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:45.799881 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 06 15:33:45.799990 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:45.799972 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 06 15:33:45.800977 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:45.800958 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-hq8kh" Jul 06 15:33:45.801796 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:45.801781 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 06 15:33:45.801979 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:45.801966 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-d2xmv\"" Jul 06 15:33:45.802184 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:45.802171 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 06 15:33:45.803140 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:45.803126 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7s97z" Jul 06 15:33:45.804189 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:45.804168 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 06 15:33:45.804301 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:45.804282 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 06 15:33:45.804393 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:45.804354 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-qp8dh\"" Jul 06 15:33:45.805501 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:45.805482 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pdw4n" Jul 06 15:33:45.805585 ip-10-0-129-154 kubenswrapper[2563]: E0706 15:33:45.805564 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pdw4n" podUID="e6761760-82ef-43c3-99f0-9e027d7c859c" Jul 06 15:33:45.807057 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:45.807039 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-rvrd2\"" Jul 06 15:33:45.807735 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:45.807716 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-rmsk7" Jul 06 15:33:45.808292 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:45.808276 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 06 15:33:45.808470 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:45.808455 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/b8afa6a4-5899-4409-87d5-de85f2a850ac-etc-modprobe-d\") pod \"tuned-d2c7p\" (UID: \"b8afa6a4-5899-4409-87d5-de85f2a850ac\") " pod="openshift-cluster-node-tuning-operator/tuned-d2c7p" Jul 06 15:33:45.808548 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:45.808477 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/cba33cb2-37bb-421e-9c92-e8d5894c6b1d-var-lib-openvswitch\") pod \"ovnkube-node-tv82v\" (UID: \"cba33cb2-37bb-421e-9c92-e8d5894c6b1d\") " pod="openshift-ovn-kubernetes/ovnkube-node-tv82v" Jul 06 15:33:45.808548 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:45.808493 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/cba33cb2-37bb-421e-9c92-e8d5894c6b1d-ovn-node-metrics-cert\") pod \"ovnkube-node-tv82v\" (UID: \"cba33cb2-37bb-421e-9c92-e8d5894c6b1d\") " pod="openshift-ovn-kubernetes/ovnkube-node-tv82v" Jul 06 15:33:45.808548 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:45.808508 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/e67f3ab6-afb7-47ff-8219-adc98fe79e74-konnectivity-ca\") pod \"konnectivity-agent-hq8kh\" (UID: \"e67f3ab6-afb7-47ff-8219-adc98fe79e74\") " pod="kube-system/konnectivity-agent-hq8kh" Jul 06 15:33:45.808548 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:45.808543 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/cba33cb2-37bb-421e-9c92-e8d5894c6b1d-run-ovn\") pod \"ovnkube-node-tv82v\" (UID: \"cba33cb2-37bb-421e-9c92-e8d5894c6b1d\") " pod="openshift-ovn-kubernetes/ovnkube-node-tv82v" Jul 06 15:33:45.808743 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:45.808567 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/b8afa6a4-5899-4409-87d5-de85f2a850ac-etc-sysconfig\") pod \"tuned-d2c7p\" (UID: \"b8afa6a4-5899-4409-87d5-de85f2a850ac\") " pod="openshift-cluster-node-tuning-operator/tuned-d2c7p" Jul 06 15:33:45.808743 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:45.808584 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/b8afa6a4-5899-4409-87d5-de85f2a850ac-etc-sysctl-conf\") pod \"tuned-d2c7p\" (UID: \"b8afa6a4-5899-4409-87d5-de85f2a850ac\") " pod="openshift-cluster-node-tuning-operator/tuned-d2c7p" Jul 06 15:33:45.808743 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:45.808605 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b8afa6a4-5899-4409-87d5-de85f2a850ac-lib-modules\") pod \"tuned-d2c7p\" (UID: \"b8afa6a4-5899-4409-87d5-de85f2a850ac\") " pod="openshift-cluster-node-tuning-operator/tuned-d2c7p" Jul 06 15:33:45.808743 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:45.808641 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b8afa6a4-5899-4409-87d5-de85f2a850ac-host\") pod \"tuned-d2c7p\" (UID: \"b8afa6a4-5899-4409-87d5-de85f2a850ac\") " pod="openshift-cluster-node-tuning-operator/tuned-d2c7p" Jul 06 15:33:45.808743 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:45.808654 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 06 15:33:45.808743 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:45.808680 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/b8afa6a4-5899-4409-87d5-de85f2a850ac-etc-tuned\") pod \"tuned-d2c7p\" (UID: \"b8afa6a4-5899-4409-87d5-de85f2a850ac\") " pod="openshift-cluster-node-tuning-operator/tuned-d2c7p" Jul 06 15:33:45.808743 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:45.808708 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pgqlp\" (UniqueName: \"kubernetes.io/projected/a3b0e3f1-74a1-4f15-9770-1a8ac2a02c4f-kube-api-access-pgqlp\") pod \"node-resolver-pgkmz\" (UID: \"a3b0e3f1-74a1-4f15-9770-1a8ac2a02c4f\") " pod="openshift-dns/node-resolver-pgkmz" Jul 06 15:33:45.808743 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:45.808734 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/cba33cb2-37bb-421e-9c92-e8d5894c6b1d-host-kubelet\") pod \"ovnkube-node-tv82v\" (UID: \"cba33cb2-37bb-421e-9c92-e8d5894c6b1d\") " pod="openshift-ovn-kubernetes/ovnkube-node-tv82v" Jul 06 15:33:45.809091 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:45.808756 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/cba33cb2-37bb-421e-9c92-e8d5894c6b1d-etc-openvswitch\") pod \"ovnkube-node-tv82v\" (UID: \"cba33cb2-37bb-421e-9c92-e8d5894c6b1d\") " pod="openshift-ovn-kubernetes/ovnkube-node-tv82v" Jul 06 15:33:45.809091 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:45.808800 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 06 15:33:45.809091 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:45.808799 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/cba33cb2-37bb-421e-9c92-e8d5894c6b1d-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-tv82v\" (UID: \"cba33cb2-37bb-421e-9c92-e8d5894c6b1d\") " pod="openshift-ovn-kubernetes/ovnkube-node-tv82v" Jul 06 15:33:45.809091 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:45.808887 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gk7r6\" (UniqueName: \"kubernetes.io/projected/cba33cb2-37bb-421e-9c92-e8d5894c6b1d-kube-api-access-gk7r6\") pod \"ovnkube-node-tv82v\" (UID: \"cba33cb2-37bb-421e-9c92-e8d5894c6b1d\") " pod="openshift-ovn-kubernetes/ovnkube-node-tv82v" Jul 06 15:33:45.809091 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:45.808912 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/cba33cb2-37bb-421e-9c92-e8d5894c6b1d-host-cni-bin\") pod \"ovnkube-node-tv82v\" (UID: \"cba33cb2-37bb-421e-9c92-e8d5894c6b1d\") " pod="openshift-ovn-kubernetes/ovnkube-node-tv82v" Jul 06 15:33:45.809091 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:45.808954 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/cba33cb2-37bb-421e-9c92-e8d5894c6b1d-ovnkube-script-lib\") pod \"ovnkube-node-tv82v\" (UID: \"cba33cb2-37bb-421e-9c92-e8d5894c6b1d\") " pod="openshift-ovn-kubernetes/ovnkube-node-tv82v" Jul 06 15:33:45.809091 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:45.808980 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/e67f3ab6-afb7-47ff-8219-adc98fe79e74-agent-certs\") pod \"konnectivity-agent-hq8kh\" (UID: \"e67f3ab6-afb7-47ff-8219-adc98fe79e74\") " pod="kube-system/konnectivity-agent-hq8kh" Jul 06 15:33:45.809091 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:45.809017 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-22rxw\" (UniqueName: \"kubernetes.io/projected/b8afa6a4-5899-4409-87d5-de85f2a850ac-kube-api-access-22rxw\") pod \"tuned-d2c7p\" (UID: \"b8afa6a4-5899-4409-87d5-de85f2a850ac\") " pod="openshift-cluster-node-tuning-operator/tuned-d2c7p" Jul 06 15:33:45.809091 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:45.809053 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/a3b0e3f1-74a1-4f15-9770-1a8ac2a02c4f-hosts-file\") pod \"node-resolver-pgkmz\" (UID: \"a3b0e3f1-74a1-4f15-9770-1a8ac2a02c4f\") " pod="openshift-dns/node-resolver-pgkmz" Jul 06 15:33:45.809091 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:45.809076 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b8afa6a4-5899-4409-87d5-de85f2a850ac-etc-kubernetes\") pod \"tuned-d2c7p\" (UID: \"b8afa6a4-5899-4409-87d5-de85f2a850ac\") " pod="openshift-cluster-node-tuning-operator/tuned-d2c7p" Jul 06 15:33:45.809583 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:45.809096 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/b8afa6a4-5899-4409-87d5-de85f2a850ac-run\") pod \"tuned-d2c7p\" (UID: \"b8afa6a4-5899-4409-87d5-de85f2a850ac\") " pod="openshift-cluster-node-tuning-operator/tuned-d2c7p" Jul 06 15:33:45.809583 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:45.809117 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/cba33cb2-37bb-421e-9c92-e8d5894c6b1d-systemd-units\") pod \"ovnkube-node-tv82v\" (UID: \"cba33cb2-37bb-421e-9c92-e8d5894c6b1d\") " pod="openshift-ovn-kubernetes/ovnkube-node-tv82v" Jul 06 15:33:45.809583 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:45.809139 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/cba33cb2-37bb-421e-9c92-e8d5894c6b1d-host-slash\") pod \"ovnkube-node-tv82v\" (UID: \"cba33cb2-37bb-421e-9c92-e8d5894c6b1d\") " pod="openshift-ovn-kubernetes/ovnkube-node-tv82v" Jul 06 15:33:45.809583 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:45.809166 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/cba33cb2-37bb-421e-9c92-e8d5894c6b1d-host-run-netns\") pod \"ovnkube-node-tv82v\" (UID: \"cba33cb2-37bb-421e-9c92-e8d5894c6b1d\") " pod="openshift-ovn-kubernetes/ovnkube-node-tv82v" Jul 06 15:33:45.809583 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:45.809193 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/cba33cb2-37bb-421e-9c92-e8d5894c6b1d-ovnkube-config\") pod \"ovnkube-node-tv82v\" (UID: \"cba33cb2-37bb-421e-9c92-e8d5894c6b1d\") " pod="openshift-ovn-kubernetes/ovnkube-node-tv82v" Jul 06 15:33:45.809583 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:45.809222 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/cba33cb2-37bb-421e-9c92-e8d5894c6b1d-node-log\") pod \"ovnkube-node-tv82v\" (UID: \"cba33cb2-37bb-421e-9c92-e8d5894c6b1d\") " pod="openshift-ovn-kubernetes/ovnkube-node-tv82v" Jul 06 15:33:45.809583 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:45.809246 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/cba33cb2-37bb-421e-9c92-e8d5894c6b1d-log-socket\") pod \"ovnkube-node-tv82v\" (UID: \"cba33cb2-37bb-421e-9c92-e8d5894c6b1d\") " pod="openshift-ovn-kubernetes/ovnkube-node-tv82v" Jul 06 15:33:45.809583 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:45.809262 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/b8afa6a4-5899-4409-87d5-de85f2a850ac-etc-sysctl-d\") pod \"tuned-d2c7p\" (UID: \"b8afa6a4-5899-4409-87d5-de85f2a850ac\") " pod="openshift-cluster-node-tuning-operator/tuned-d2c7p" Jul 06 15:33:45.809583 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:45.809281 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b8afa6a4-5899-4409-87d5-de85f2a850ac-sys\") pod \"tuned-d2c7p\" (UID: \"b8afa6a4-5899-4409-87d5-de85f2a850ac\") " pod="openshift-cluster-node-tuning-operator/tuned-d2c7p" Jul 06 15:33:45.809583 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:45.809302 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b8afa6a4-5899-4409-87d5-de85f2a850ac-tmp\") pod \"tuned-d2c7p\" (UID: \"b8afa6a4-5899-4409-87d5-de85f2a850ac\") " pod="openshift-cluster-node-tuning-operator/tuned-d2c7p" Jul 06 15:33:45.809583 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:45.809344 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/cba33cb2-37bb-421e-9c92-e8d5894c6b1d-host-run-ovn-kubernetes\") pod \"ovnkube-node-tv82v\" (UID: \"cba33cb2-37bb-421e-9c92-e8d5894c6b1d\") " pod="openshift-ovn-kubernetes/ovnkube-node-tv82v" Jul 06 15:33:45.809583 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:45.809372 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/cba33cb2-37bb-421e-9c92-e8d5894c6b1d-host-cni-netd\") pod \"ovnkube-node-tv82v\" (UID: \"cba33cb2-37bb-421e-9c92-e8d5894c6b1d\") " pod="openshift-ovn-kubernetes/ovnkube-node-tv82v" Jul 06 15:33:45.809583 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:45.809401 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/b8afa6a4-5899-4409-87d5-de85f2a850ac-etc-systemd\") pod \"tuned-d2c7p\" (UID: \"b8afa6a4-5899-4409-87d5-de85f2a850ac\") " pod="openshift-cluster-node-tuning-operator/tuned-d2c7p" Jul 06 15:33:45.809583 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:45.809443 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b8afa6a4-5899-4409-87d5-de85f2a850ac-var-lib-kubelet\") pod \"tuned-d2c7p\" (UID: \"b8afa6a4-5899-4409-87d5-de85f2a850ac\") " pod="openshift-cluster-node-tuning-operator/tuned-d2c7p" Jul 06 15:33:45.809583 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:45.809466 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/cba33cb2-37bb-421e-9c92-e8d5894c6b1d-run-systemd\") pod \"ovnkube-node-tv82v\" (UID: \"cba33cb2-37bb-421e-9c92-e8d5894c6b1d\") " pod="openshift-ovn-kubernetes/ovnkube-node-tv82v" Jul 06 15:33:45.809583 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:45.809487 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/cba33cb2-37bb-421e-9c92-e8d5894c6b1d-run-openvswitch\") pod \"ovnkube-node-tv82v\" (UID: \"cba33cb2-37bb-421e-9c92-e8d5894c6b1d\") " pod="openshift-ovn-kubernetes/ovnkube-node-tv82v" Jul 06 15:33:45.809583 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:45.809528 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/cba33cb2-37bb-421e-9c92-e8d5894c6b1d-env-overrides\") pod \"ovnkube-node-tv82v\" (UID: \"cba33cb2-37bb-421e-9c92-e8d5894c6b1d\") " pod="openshift-ovn-kubernetes/ovnkube-node-tv82v" Jul 06 15:33:45.810086 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:45.809574 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a3b0e3f1-74a1-4f15-9770-1a8ac2a02c4f-tmp-dir\") pod \"node-resolver-pgkmz\" (UID: \"a3b0e3f1-74a1-4f15-9770-1a8ac2a02c4f\") " pod="openshift-dns/node-resolver-pgkmz" Jul 06 15:33:45.810277 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:45.810260 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-v8fxz" Jul 06 15:33:45.810358 ip-10-0-129-154 kubenswrapper[2563]: E0706 15:33:45.810342 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-v8fxz" podUID="5771d1de-202e-4920-b0de-5d12a4e4e2fc" Jul 06 15:33:45.810718 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:45.810691 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 06 15:33:45.810794 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:45.810689 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 06 15:33:45.810794 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:45.810750 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 06 15:33:45.810794 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:45.810777 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-wshd7\"" Jul 06 15:33:45.812617 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:45.812604 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-bfx9b" Jul 06 15:33:45.814800 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:45.814785 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w9qbt" Jul 06 15:33:45.814893 ip-10-0-129-154 kubenswrapper[2563]: E0706 15:33:45.814839 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-w9qbt" podUID="bce194c9-b363-4021-bb8a-c7cc65facc39" Jul 06 15:33:45.815076 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:45.815061 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-82k6p\"" Jul 06 15:33:45.815169 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:45.815155 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 06 15:33:45.815237 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:45.815217 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 06 15:33:45.815350 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:45.815336 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 06 15:33:45.817075 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:45.817058 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-8zxfb" Jul 06 15:33:45.819805 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:45.819789 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-nmfg6" Jul 06 15:33:45.820958 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:45.820942 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 06 15:33:45.821103 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:45.821091 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-wrp4m\"" Jul 06 15:33:45.821233 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:45.821217 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 06 15:33:45.821278 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:45.821246 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 06 15:33:45.821319 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:45.821292 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 06 15:33:45.824072 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:45.822350 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 06 15:33:45.824072 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:45.822534 2563 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 06 15:33:45.824072 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:45.822617 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 06 15:33:45.824072 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:45.823155 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-zgr8v\"" Jul 06 15:33:45.876250 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:45.876211 2563 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-cw52t" Jul 06 15:33:45.886101 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:45.886072 2563 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-cw52t" Jul 06 15:33:45.899711 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:45.899691 2563 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 06 15:33:45.910754 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:45.910726 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/bce194c9-b363-4021-bb8a-c7cc65facc39-kubelet-config\") pod \"global-pull-secret-syncer-w9qbt\" (UID: \"bce194c9-b363-4021-bb8a-c7cc65facc39\") " pod="kube-system/global-pull-secret-syncer-w9qbt" Jul 06 15:33:45.910906 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:45.910769 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/cba33cb2-37bb-421e-9c92-e8d5894c6b1d-node-log\") pod \"ovnkube-node-tv82v\" (UID: \"cba33cb2-37bb-421e-9c92-e8d5894c6b1d\") " pod="openshift-ovn-kubernetes/ovnkube-node-tv82v" Jul 06 15:33:45.910906 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:45.910795 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b8afa6a4-5899-4409-87d5-de85f2a850ac-sys\") pod \"tuned-d2c7p\" (UID: \"b8afa6a4-5899-4409-87d5-de85f2a850ac\") " pod="openshift-cluster-node-tuning-operator/tuned-d2c7p" Jul 06 15:33:45.910906 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:45.910827 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/cba33cb2-37bb-421e-9c92-e8d5894c6b1d-host-run-ovn-kubernetes\") pod \"ovnkube-node-tv82v\" (UID: \"cba33cb2-37bb-421e-9c92-e8d5894c6b1d\") " pod="openshift-ovn-kubernetes/ovnkube-node-tv82v" Jul 06 15:33:45.910906 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:45.910864 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/cba33cb2-37bb-421e-9c92-e8d5894c6b1d-node-log\") pod \"ovnkube-node-tv82v\" (UID: \"cba33cb2-37bb-421e-9c92-e8d5894c6b1d\") " pod="openshift-ovn-kubernetes/ovnkube-node-tv82v" Jul 06 15:33:45.910906 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:45.910876 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b8afa6a4-5899-4409-87d5-de85f2a850ac-sys\") pod \"tuned-d2c7p\" (UID: \"b8afa6a4-5899-4409-87d5-de85f2a850ac\") " pod="openshift-cluster-node-tuning-operator/tuned-d2c7p" Jul 06 15:33:45.910906 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:45.910856 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e32dedb7-fb62-41a5-9126-63a337f62a53-cni-binary-copy\") pod \"multus-additional-cni-plugins-nmfg6\" (UID: \"e32dedb7-fb62-41a5-9126-63a337f62a53\") " pod="openshift-multus/multus-additional-cni-plugins-nmfg6" Jul 06 15:33:45.911188 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:45.910930 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/cba33cb2-37bb-421e-9c92-e8d5894c6b1d-run-openvswitch\") pod \"ovnkube-node-tv82v\" (UID: \"cba33cb2-37bb-421e-9c92-e8d5894c6b1d\") " pod="openshift-ovn-kubernetes/ovnkube-node-tv82v" Jul 06 15:33:45.911188 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:45.910931 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/cba33cb2-37bb-421e-9c92-e8d5894c6b1d-host-run-ovn-kubernetes\") pod \"ovnkube-node-tv82v\" (UID: \"cba33cb2-37bb-421e-9c92-e8d5894c6b1d\") " pod="openshift-ovn-kubernetes/ovnkube-node-tv82v" Jul 06 15:33:45.911188 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:45.910958 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a3b0e3f1-74a1-4f15-9770-1a8ac2a02c4f-tmp-dir\") pod \"node-resolver-pgkmz\" (UID: \"a3b0e3f1-74a1-4f15-9770-1a8ac2a02c4f\") " pod="openshift-dns/node-resolver-pgkmz" Jul 06 15:33:45.911188 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:45.910990 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a83f40ad-b56d-4d3f-9275-1a09f0c89e0c-host-var-lib-cni-bin\") pod \"multus-8zxfb\" (UID: \"a83f40ad-b56d-4d3f-9275-1a09f0c89e0c\") " pod="openshift-multus/multus-8zxfb" Jul 06 15:33:45.911188 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:45.910998 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/cba33cb2-37bb-421e-9c92-e8d5894c6b1d-run-openvswitch\") pod \"ovnkube-node-tv82v\" (UID: \"cba33cb2-37bb-421e-9c92-e8d5894c6b1d\") " pod="openshift-ovn-kubernetes/ovnkube-node-tv82v" Jul 06 15:33:45.911188 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:45.911014 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/a83f40ad-b56d-4d3f-9275-1a09f0c89e0c-host-run-multus-certs\") pod \"multus-8zxfb\" (UID: \"a83f40ad-b56d-4d3f-9275-1a09f0c89e0c\") " pod="openshift-multus/multus-8zxfb" Jul 06 15:33:45.911188 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:45.911039 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a83f40ad-b56d-4d3f-9275-1a09f0c89e0c-etc-kubernetes\") pod \"multus-8zxfb\" (UID: \"a83f40ad-b56d-4d3f-9275-1a09f0c89e0c\") " pod="openshift-multus/multus-8zxfb" Jul 06 15:33:45.911188 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:45.911077 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fb6wt\" (UniqueName: \"kubernetes.io/projected/e32dedb7-fb62-41a5-9126-63a337f62a53-kube-api-access-fb6wt\") pod \"multus-additional-cni-plugins-nmfg6\" (UID: \"e32dedb7-fb62-41a5-9126-63a337f62a53\") " pod="openshift-multus/multus-additional-cni-plugins-nmfg6" Jul 06 15:33:45.911188 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:45.911119 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a83f40ad-b56d-4d3f-9275-1a09f0c89e0c-system-cni-dir\") pod \"multus-8zxfb\" (UID: \"a83f40ad-b56d-4d3f-9275-1a09f0c89e0c\") " pod="openshift-multus/multus-8zxfb" Jul 06 15:33:45.911188 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:45.911152 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/b8afa6a4-5899-4409-87d5-de85f2a850ac-etc-modprobe-d\") pod \"tuned-d2c7p\" (UID: \"b8afa6a4-5899-4409-87d5-de85f2a850ac\") " pod="openshift-cluster-node-tuning-operator/tuned-d2c7p" Jul 06 15:33:45.911188 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:45.911181 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/cba33cb2-37bb-421e-9c92-e8d5894c6b1d-var-lib-openvswitch\") pod \"ovnkube-node-tv82v\" (UID: \"cba33cb2-37bb-421e-9c92-e8d5894c6b1d\") " pod="openshift-ovn-kubernetes/ovnkube-node-tv82v" Jul 06 15:33:45.911764 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:45.911206 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tj49r\" (UniqueName: \"kubernetes.io/projected/0ce018e7-3a1f-43dc-8988-5d8054f04c69-kube-api-access-tj49r\") pod \"node-ca-rmsk7\" (UID: \"0ce018e7-3a1f-43dc-8988-5d8054f04c69\") " pod="openshift-image-registry/node-ca-rmsk7" Jul 06 15:33:45.911764 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:45.911234 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/a83f40ad-b56d-4d3f-9275-1a09f0c89e0c-host-run-k8s-cni-cncf-io\") pod \"multus-8zxfb\" (UID: \"a83f40ad-b56d-4d3f-9275-1a09f0c89e0c\") " pod="openshift-multus/multus-8zxfb" Jul 06 15:33:45.911764 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:45.911255 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a83f40ad-b56d-4d3f-9275-1a09f0c89e0c-host-var-lib-kubelet\") pod \"multus-8zxfb\" (UID: \"a83f40ad-b56d-4d3f-9275-1a09f0c89e0c\") " pod="openshift-multus/multus-8zxfb" Jul 06 15:33:45.911764 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:45.911278 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-twj8p\" (UniqueName: \"kubernetes.io/projected/e6761760-82ef-43c3-99f0-9e027d7c859c-kube-api-access-twj8p\") pod \"network-check-target-pdw4n\" (UID: \"e6761760-82ef-43c3-99f0-9e027d7c859c\") " pod="openshift-network-diagnostics/network-check-target-pdw4n" Jul 06 15:33:45.911764 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:45.911305 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/11b1836e-d028-4749-baf8-3fc1e799d0be-host-slash\") pod \"iptables-alerter-bfx9b\" (UID: \"11b1836e-d028-4749-baf8-3fc1e799d0be\") " pod="openshift-network-operator/iptables-alerter-bfx9b" Jul 06 15:33:45.911764 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:45.911310 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/cba33cb2-37bb-421e-9c92-e8d5894c6b1d-var-lib-openvswitch\") pod \"ovnkube-node-tv82v\" (UID: \"cba33cb2-37bb-421e-9c92-e8d5894c6b1d\") " pod="openshift-ovn-kubernetes/ovnkube-node-tv82v" Jul 06 15:33:45.911764 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:45.911313 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/b8afa6a4-5899-4409-87d5-de85f2a850ac-etc-modprobe-d\") pod \"tuned-d2c7p\" (UID: \"b8afa6a4-5899-4409-87d5-de85f2a850ac\") " pod="openshift-cluster-node-tuning-operator/tuned-d2c7p" Jul 06 15:33:45.911764 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:45.911329 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/cba33cb2-37bb-421e-9c92-e8d5894c6b1d-run-ovn\") pod \"ovnkube-node-tv82v\" (UID: \"cba33cb2-37bb-421e-9c92-e8d5894c6b1d\") " pod="openshift-ovn-kubernetes/ovnkube-node-tv82v" Jul 06 15:33:45.911764 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:45.911339 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a3b0e3f1-74a1-4f15-9770-1a8ac2a02c4f-tmp-dir\") pod \"node-resolver-pgkmz\" (UID: \"a3b0e3f1-74a1-4f15-9770-1a8ac2a02c4f\") " pod="openshift-dns/node-resolver-pgkmz" Jul 06 15:33:45.911764 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:45.911353 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/b8afa6a4-5899-4409-87d5-de85f2a850ac-etc-sysctl-conf\") pod \"tuned-d2c7p\" (UID: \"b8afa6a4-5899-4409-87d5-de85f2a850ac\") " pod="openshift-cluster-node-tuning-operator/tuned-d2c7p" Jul 06 15:33:45.911764 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:45.911391 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/cba33cb2-37bb-421e-9c92-e8d5894c6b1d-run-ovn\") pod \"ovnkube-node-tv82v\" (UID: \"cba33cb2-37bb-421e-9c92-e8d5894c6b1d\") " pod="openshift-ovn-kubernetes/ovnkube-node-tv82v" Jul 06 15:33:45.911764 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:45.911409 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b8afa6a4-5899-4409-87d5-de85f2a850ac-lib-modules\") pod \"tuned-d2c7p\" (UID: \"b8afa6a4-5899-4409-87d5-de85f2a850ac\") " pod="openshift-cluster-node-tuning-operator/tuned-d2c7p" Jul 06 15:33:45.911764 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:45.911483 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/b8afa6a4-5899-4409-87d5-de85f2a850ac-etc-tuned\") pod \"tuned-d2c7p\" (UID: \"b8afa6a4-5899-4409-87d5-de85f2a850ac\") " pod="openshift-cluster-node-tuning-operator/tuned-d2c7p" Jul 06 15:33:45.911764 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:45.911529 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/b8afa6a4-5899-4409-87d5-de85f2a850ac-etc-sysctl-conf\") pod \"tuned-d2c7p\" (UID: \"b8afa6a4-5899-4409-87d5-de85f2a850ac\") " pod="openshift-cluster-node-tuning-operator/tuned-d2c7p" Jul 06 15:33:45.911764 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:45.911536 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b8afa6a4-5899-4409-87d5-de85f2a850ac-lib-modules\") pod \"tuned-d2c7p\" (UID: \"b8afa6a4-5899-4409-87d5-de85f2a850ac\") " pod="openshift-cluster-node-tuning-operator/tuned-d2c7p" Jul 06 15:33:45.911764 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:45.911546 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a83f40ad-b56d-4d3f-9275-1a09f0c89e0c-multus-cni-dir\") pod \"multus-8zxfb\" (UID: \"a83f40ad-b56d-4d3f-9275-1a09f0c89e0c\") " pod="openshift-multus/multus-8zxfb" Jul 06 15:33:45.911764 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:45.911643 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dqbgv\" (UniqueName: \"kubernetes.io/projected/11b1836e-d028-4749-baf8-3fc1e799d0be-kube-api-access-dqbgv\") pod \"iptables-alerter-bfx9b\" (UID: \"11b1836e-d028-4749-baf8-3fc1e799d0be\") " pod="openshift-network-operator/iptables-alerter-bfx9b" Jul 06 15:33:45.912556 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:45.911677 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e32dedb7-fb62-41a5-9126-63a337f62a53-system-cni-dir\") pod \"multus-additional-cni-plugins-nmfg6\" (UID: \"e32dedb7-fb62-41a5-9126-63a337f62a53\") " pod="openshift-multus/multus-additional-cni-plugins-nmfg6" Jul 06 15:33:45.912556 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:45.911732 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e32dedb7-fb62-41a5-9126-63a337f62a53-cnibin\") pod \"multus-additional-cni-plugins-nmfg6\" (UID: \"e32dedb7-fb62-41a5-9126-63a337f62a53\") " pod="openshift-multus/multus-additional-cni-plugins-nmfg6" Jul 06 15:33:45.912556 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:45.911758 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gk7r6\" (UniqueName: \"kubernetes.io/projected/cba33cb2-37bb-421e-9c92-e8d5894c6b1d-kube-api-access-gk7r6\") pod \"ovnkube-node-tv82v\" (UID: \"cba33cb2-37bb-421e-9c92-e8d5894c6b1d\") " pod="openshift-ovn-kubernetes/ovnkube-node-tv82v" Jul 06 15:33:45.912556 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:45.911782 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/1aead9b0-c187-46cb-b965-57f2cb23d421-etc-selinux\") pod \"aws-ebs-csi-driver-node-7s97z\" (UID: \"1aead9b0-c187-46cb-b965-57f2cb23d421\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7s97z" Jul 06 15:33:45.912556 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:45.911839 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/cba33cb2-37bb-421e-9c92-e8d5894c6b1d-host-cni-bin\") pod \"ovnkube-node-tv82v\" (UID: \"cba33cb2-37bb-421e-9c92-e8d5894c6b1d\") " pod="openshift-ovn-kubernetes/ovnkube-node-tv82v" Jul 06 15:33:45.912556 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:45.911896 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/cba33cb2-37bb-421e-9c92-e8d5894c6b1d-ovnkube-script-lib\") pod \"ovnkube-node-tv82v\" (UID: \"cba33cb2-37bb-421e-9c92-e8d5894c6b1d\") " pod="openshift-ovn-kubernetes/ovnkube-node-tv82v" Jul 06 15:33:45.912556 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:45.911882 2563 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 06 15:33:45.912556 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:45.911929 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/1aead9b0-c187-46cb-b965-57f2cb23d421-kubelet-dir\") pod \"aws-ebs-csi-driver-node-7s97z\" (UID: \"1aead9b0-c187-46cb-b965-57f2cb23d421\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7s97z" Jul 06 15:33:45.912556 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:45.911954 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a83f40ad-b56d-4d3f-9275-1a09f0c89e0c-host-run-netns\") pod \"multus-8zxfb\" (UID: \"a83f40ad-b56d-4d3f-9275-1a09f0c89e0c\") " pod="openshift-multus/multus-8zxfb" Jul 06 15:33:45.912556 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:45.911989 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/a83f40ad-b56d-4d3f-9275-1a09f0c89e0c-multus-daemon-config\") pod \"multus-8zxfb\" (UID: \"a83f40ad-b56d-4d3f-9275-1a09f0c89e0c\") " pod="openshift-multus/multus-8zxfb" Jul 06 15:33:45.912556 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:45.912019 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kj6p9\" (UniqueName: \"kubernetes.io/projected/5771d1de-202e-4920-b0de-5d12a4e4e2fc-kube-api-access-kj6p9\") pod \"network-metrics-daemon-v8fxz\" (UID: \"5771d1de-202e-4920-b0de-5d12a4e4e2fc\") " pod="openshift-multus/network-metrics-daemon-v8fxz" Jul 06 15:33:45.912556 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:45.912039 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/11b1836e-d028-4749-baf8-3fc1e799d0be-iptables-alerter-script\") pod \"iptables-alerter-bfx9b\" (UID: \"11b1836e-d028-4749-baf8-3fc1e799d0be\") " pod="openshift-network-operator/iptables-alerter-bfx9b" Jul 06 15:33:45.912556 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:45.912054 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/e32dedb7-fb62-41a5-9126-63a337f62a53-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-nmfg6\" (UID: \"e32dedb7-fb62-41a5-9126-63a337f62a53\") " pod="openshift-multus/multus-additional-cni-plugins-nmfg6" Jul 06 15:33:45.912556 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:45.912073 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-22rxw\" (UniqueName: \"kubernetes.io/projected/b8afa6a4-5899-4409-87d5-de85f2a850ac-kube-api-access-22rxw\") pod \"tuned-d2c7p\" (UID: \"b8afa6a4-5899-4409-87d5-de85f2a850ac\") " pod="openshift-cluster-node-tuning-operator/tuned-d2c7p" Jul 06 15:33:45.912556 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:45.912091 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b8afa6a4-5899-4409-87d5-de85f2a850ac-etc-kubernetes\") pod \"tuned-d2c7p\" (UID: \"b8afa6a4-5899-4409-87d5-de85f2a850ac\") " pod="openshift-cluster-node-tuning-operator/tuned-d2c7p" Jul 06 15:33:45.912556 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:45.912116 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/b8afa6a4-5899-4409-87d5-de85f2a850ac-run\") pod \"tuned-d2c7p\" (UID: \"b8afa6a4-5899-4409-87d5-de85f2a850ac\") " pod="openshift-cluster-node-tuning-operator/tuned-d2c7p" Jul 06 15:33:45.912556 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:45.912131 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/cba33cb2-37bb-421e-9c92-e8d5894c6b1d-host-cni-bin\") pod \"ovnkube-node-tv82v\" (UID: \"cba33cb2-37bb-421e-9c92-e8d5894c6b1d\") " pod="openshift-ovn-kubernetes/ovnkube-node-tv82v" Jul 06 15:33:45.913224 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:45.912139 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/cba33cb2-37bb-421e-9c92-e8d5894c6b1d-host-slash\") pod \"ovnkube-node-tv82v\" (UID: \"cba33cb2-37bb-421e-9c92-e8d5894c6b1d\") " pod="openshift-ovn-kubernetes/ovnkube-node-tv82v" Jul 06 15:33:45.913224 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:45.912176 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/e32dedb7-fb62-41a5-9126-63a337f62a53-os-release\") pod \"multus-additional-cni-plugins-nmfg6\" (UID: \"e32dedb7-fb62-41a5-9126-63a337f62a53\") " pod="openshift-multus/multus-additional-cni-plugins-nmfg6" Jul 06 15:33:45.913224 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:45.912185 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/cba33cb2-37bb-421e-9c92-e8d5894c6b1d-host-slash\") pod \"ovnkube-node-tv82v\" (UID: \"cba33cb2-37bb-421e-9c92-e8d5894c6b1d\") " pod="openshift-ovn-kubernetes/ovnkube-node-tv82v" Jul 06 15:33:45.913224 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:45.912214 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/cba33cb2-37bb-421e-9c92-e8d5894c6b1d-log-socket\") pod \"ovnkube-node-tv82v\" (UID: \"cba33cb2-37bb-421e-9c92-e8d5894c6b1d\") " pod="openshift-ovn-kubernetes/ovnkube-node-tv82v" Jul 06 15:33:45.913224 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:45.912245 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/b8afa6a4-5899-4409-87d5-de85f2a850ac-etc-sysctl-d\") pod \"tuned-d2c7p\" (UID: \"b8afa6a4-5899-4409-87d5-de85f2a850ac\") " pod="openshift-cluster-node-tuning-operator/tuned-d2c7p" Jul 06 15:33:45.913224 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:45.912270 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b8afa6a4-5899-4409-87d5-de85f2a850ac-tmp\") pod \"tuned-d2c7p\" (UID: \"b8afa6a4-5899-4409-87d5-de85f2a850ac\") " pod="openshift-cluster-node-tuning-operator/tuned-d2c7p" Jul 06 15:33:45.913224 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:45.912294 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/cba33cb2-37bb-421e-9c92-e8d5894c6b1d-host-cni-netd\") pod \"ovnkube-node-tv82v\" (UID: \"cba33cb2-37bb-421e-9c92-e8d5894c6b1d\") " pod="openshift-ovn-kubernetes/ovnkube-node-tv82v" Jul 06 15:33:45.913224 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:45.912321 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/cba33cb2-37bb-421e-9c92-e8d5894c6b1d-env-overrides\") pod \"ovnkube-node-tv82v\" (UID: \"cba33cb2-37bb-421e-9c92-e8d5894c6b1d\") " pod="openshift-ovn-kubernetes/ovnkube-node-tv82v" Jul 06 15:33:45.913224 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:45.912315 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b8afa6a4-5899-4409-87d5-de85f2a850ac-etc-kubernetes\") pod \"tuned-d2c7p\" (UID: \"b8afa6a4-5899-4409-87d5-de85f2a850ac\") " pod="openshift-cluster-node-tuning-operator/tuned-d2c7p" Jul 06 15:33:45.913224 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:45.912351 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/e32dedb7-fb62-41a5-9126-63a337f62a53-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-nmfg6\" (UID: \"e32dedb7-fb62-41a5-9126-63a337f62a53\") " pod="openshift-multus/multus-additional-cni-plugins-nmfg6" Jul 06 15:33:45.913224 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:45.912407 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/cba33cb2-37bb-421e-9c92-e8d5894c6b1d-log-socket\") pod \"ovnkube-node-tv82v\" (UID: \"cba33cb2-37bb-421e-9c92-e8d5894c6b1d\") " pod="openshift-ovn-kubernetes/ovnkube-node-tv82v" Jul 06 15:33:45.913224 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:45.912415 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/b8afa6a4-5899-4409-87d5-de85f2a850ac-etc-systemd\") pod \"tuned-d2c7p\" (UID: \"b8afa6a4-5899-4409-87d5-de85f2a850ac\") " pod="openshift-cluster-node-tuning-operator/tuned-d2c7p" Jul 06 15:33:45.913224 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:45.912409 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/cba33cb2-37bb-421e-9c92-e8d5894c6b1d-host-cni-netd\") pod \"ovnkube-node-tv82v\" (UID: \"cba33cb2-37bb-421e-9c92-e8d5894c6b1d\") " pod="openshift-ovn-kubernetes/ovnkube-node-tv82v" Jul 06 15:33:45.913224 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:45.912461 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/b8afa6a4-5899-4409-87d5-de85f2a850ac-run\") pod \"tuned-d2c7p\" (UID: \"b8afa6a4-5899-4409-87d5-de85f2a850ac\") " pod="openshift-cluster-node-tuning-operator/tuned-d2c7p" Jul 06 15:33:45.913224 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:45.912479 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b8afa6a4-5899-4409-87d5-de85f2a850ac-var-lib-kubelet\") pod \"tuned-d2c7p\" (UID: \"b8afa6a4-5899-4409-87d5-de85f2a850ac\") " pod="openshift-cluster-node-tuning-operator/tuned-d2c7p" Jul 06 15:33:45.913224 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:45.912485 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/b8afa6a4-5899-4409-87d5-de85f2a850ac-etc-systemd\") pod \"tuned-d2c7p\" (UID: \"b8afa6a4-5899-4409-87d5-de85f2a850ac\") " pod="openshift-cluster-node-tuning-operator/tuned-d2c7p" Jul 06 15:33:45.913224 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:45.912486 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/b8afa6a4-5899-4409-87d5-de85f2a850ac-etc-sysctl-d\") pod \"tuned-d2c7p\" (UID: \"b8afa6a4-5899-4409-87d5-de85f2a850ac\") " pod="openshift-cluster-node-tuning-operator/tuned-d2c7p" Jul 06 15:33:45.913224 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:45.912514 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/cba33cb2-37bb-421e-9c92-e8d5894c6b1d-run-systemd\") pod \"ovnkube-node-tv82v\" (UID: \"cba33cb2-37bb-421e-9c92-e8d5894c6b1d\") " pod="openshift-ovn-kubernetes/ovnkube-node-tv82v" Jul 06 15:33:45.914246 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:45.912537 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b8afa6a4-5899-4409-87d5-de85f2a850ac-var-lib-kubelet\") pod \"tuned-d2c7p\" (UID: \"b8afa6a4-5899-4409-87d5-de85f2a850ac\") " pod="openshift-cluster-node-tuning-operator/tuned-d2c7p" Jul 06 15:33:45.914246 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:45.912551 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/cba33cb2-37bb-421e-9c92-e8d5894c6b1d-ovnkube-script-lib\") pod \"ovnkube-node-tv82v\" (UID: \"cba33cb2-37bb-421e-9c92-e8d5894c6b1d\") " pod="openshift-ovn-kubernetes/ovnkube-node-tv82v" Jul 06 15:33:45.914246 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:45.912563 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/0ce018e7-3a1f-43dc-8988-5d8054f04c69-serviceca\") pod \"node-ca-rmsk7\" (UID: \"0ce018e7-3a1f-43dc-8988-5d8054f04c69\") " pod="openshift-image-registry/node-ca-rmsk7" Jul 06 15:33:45.914246 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:45.912590 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/bce194c9-b363-4021-bb8a-c7cc65facc39-original-pull-secret\") pod \"global-pull-secret-syncer-w9qbt\" (UID: \"bce194c9-b363-4021-bb8a-c7cc65facc39\") " pod="kube-system/global-pull-secret-syncer-w9qbt" Jul 06 15:33:45.914246 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:45.912636 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/a83f40ad-b56d-4d3f-9275-1a09f0c89e0c-os-release\") pod \"multus-8zxfb\" (UID: \"a83f40ad-b56d-4d3f-9275-1a09f0c89e0c\") " pod="openshift-multus/multus-8zxfb" Jul 06 15:33:45.914246 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:45.912624 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/cba33cb2-37bb-421e-9c92-e8d5894c6b1d-run-systemd\") pod \"ovnkube-node-tv82v\" (UID: \"cba33cb2-37bb-421e-9c92-e8d5894c6b1d\") " pod="openshift-ovn-kubernetes/ovnkube-node-tv82v" Jul 06 15:33:45.914246 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:45.912671 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-72wh7\" (UniqueName: \"kubernetes.io/projected/a83f40ad-b56d-4d3f-9275-1a09f0c89e0c-kube-api-access-72wh7\") pod \"multus-8zxfb\" (UID: \"a83f40ad-b56d-4d3f-9275-1a09f0c89e0c\") " pod="openshift-multus/multus-8zxfb" Jul 06 15:33:45.914246 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:45.912703 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/cba33cb2-37bb-421e-9c92-e8d5894c6b1d-ovn-node-metrics-cert\") pod \"ovnkube-node-tv82v\" (UID: \"cba33cb2-37bb-421e-9c92-e8d5894c6b1d\") " pod="openshift-ovn-kubernetes/ovnkube-node-tv82v" Jul 06 15:33:45.914246 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:45.912722 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/e67f3ab6-afb7-47ff-8219-adc98fe79e74-konnectivity-ca\") pod \"konnectivity-agent-hq8kh\" (UID: \"e67f3ab6-afb7-47ff-8219-adc98fe79e74\") " pod="kube-system/konnectivity-agent-hq8kh" Jul 06 15:33:45.914246 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:45.912746 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/1aead9b0-c187-46cb-b965-57f2cb23d421-device-dir\") pod \"aws-ebs-csi-driver-node-7s97z\" (UID: \"1aead9b0-c187-46cb-b965-57f2cb23d421\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7s97z" Jul 06 15:33:45.914246 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:45.912785 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/1aead9b0-c187-46cb-b965-57f2cb23d421-sys-fs\") pod \"aws-ebs-csi-driver-node-7s97z\" (UID: \"1aead9b0-c187-46cb-b965-57f2cb23d421\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7s97z" Jul 06 15:33:45.914246 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:45.912809 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a83f40ad-b56d-4d3f-9275-1a09f0c89e0c-cni-binary-copy\") pod \"multus-8zxfb\" (UID: \"a83f40ad-b56d-4d3f-9275-1a09f0c89e0c\") " pod="openshift-multus/multus-8zxfb" Jul 06 15:33:45.914246 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:45.912813 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/cba33cb2-37bb-421e-9c92-e8d5894c6b1d-env-overrides\") pod \"ovnkube-node-tv82v\" (UID: \"cba33cb2-37bb-421e-9c92-e8d5894c6b1d\") " pod="openshift-ovn-kubernetes/ovnkube-node-tv82v" Jul 06 15:33:45.914246 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:45.912834 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/b8afa6a4-5899-4409-87d5-de85f2a850ac-etc-sysconfig\") pod \"tuned-d2c7p\" (UID: \"b8afa6a4-5899-4409-87d5-de85f2a850ac\") " pod="openshift-cluster-node-tuning-operator/tuned-d2c7p" Jul 06 15:33:45.914246 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:45.912859 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b8afa6a4-5899-4409-87d5-de85f2a850ac-host\") pod \"tuned-d2c7p\" (UID: \"b8afa6a4-5899-4409-87d5-de85f2a850ac\") " pod="openshift-cluster-node-tuning-operator/tuned-d2c7p" Jul 06 15:33:45.914246 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:45.912876 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/b8afa6a4-5899-4409-87d5-de85f2a850ac-etc-sysconfig\") pod \"tuned-d2c7p\" (UID: \"b8afa6a4-5899-4409-87d5-de85f2a850ac\") " pod="openshift-cluster-node-tuning-operator/tuned-d2c7p" Jul 06 15:33:45.914246 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:45.912882 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pgqlp\" (UniqueName: \"kubernetes.io/projected/a3b0e3f1-74a1-4f15-9770-1a8ac2a02c4f-kube-api-access-pgqlp\") pod \"node-resolver-pgkmz\" (UID: \"a3b0e3f1-74a1-4f15-9770-1a8ac2a02c4f\") " pod="openshift-dns/node-resolver-pgkmz" Jul 06 15:33:45.915037 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:45.912925 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jcgpr\" (UniqueName: \"kubernetes.io/projected/1aead9b0-c187-46cb-b965-57f2cb23d421-kube-api-access-jcgpr\") pod \"aws-ebs-csi-driver-node-7s97z\" (UID: \"1aead9b0-c187-46cb-b965-57f2cb23d421\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7s97z" Jul 06 15:33:45.915037 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:45.912956 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a83f40ad-b56d-4d3f-9275-1a09f0c89e0c-cnibin\") pod \"multus-8zxfb\" (UID: \"a83f40ad-b56d-4d3f-9275-1a09f0c89e0c\") " pod="openshift-multus/multus-8zxfb" Jul 06 15:33:45.915037 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:45.912981 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a83f40ad-b56d-4d3f-9275-1a09f0c89e0c-multus-conf-dir\") pod \"multus-8zxfb\" (UID: \"a83f40ad-b56d-4d3f-9275-1a09f0c89e0c\") " pod="openshift-multus/multus-8zxfb" Jul 06 15:33:45.915037 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:45.912927 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b8afa6a4-5899-4409-87d5-de85f2a850ac-host\") pod \"tuned-d2c7p\" (UID: \"b8afa6a4-5899-4409-87d5-de85f2a850ac\") " pod="openshift-cluster-node-tuning-operator/tuned-d2c7p" Jul 06 15:33:45.915037 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:45.913010 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e32dedb7-fb62-41a5-9126-63a337f62a53-tuning-conf-dir\") pod \"multus-additional-cni-plugins-nmfg6\" (UID: \"e32dedb7-fb62-41a5-9126-63a337f62a53\") " pod="openshift-multus/multus-additional-cni-plugins-nmfg6" Jul 06 15:33:45.915037 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:45.913037 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/cba33cb2-37bb-421e-9c92-e8d5894c6b1d-host-kubelet\") pod \"ovnkube-node-tv82v\" (UID: \"cba33cb2-37bb-421e-9c92-e8d5894c6b1d\") " pod="openshift-ovn-kubernetes/ovnkube-node-tv82v" Jul 06 15:33:45.915037 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:45.913205 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/cba33cb2-37bb-421e-9c92-e8d5894c6b1d-host-kubelet\") pod \"ovnkube-node-tv82v\" (UID: \"cba33cb2-37bb-421e-9c92-e8d5894c6b1d\") " pod="openshift-ovn-kubernetes/ovnkube-node-tv82v" Jul 06 15:33:45.915037 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:45.913212 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/cba33cb2-37bb-421e-9c92-e8d5894c6b1d-etc-openvswitch\") pod \"ovnkube-node-tv82v\" (UID: \"cba33cb2-37bb-421e-9c92-e8d5894c6b1d\") " pod="openshift-ovn-kubernetes/ovnkube-node-tv82v" Jul 06 15:33:45.915037 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:45.913243 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/cba33cb2-37bb-421e-9c92-e8d5894c6b1d-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-tv82v\" (UID: \"cba33cb2-37bb-421e-9c92-e8d5894c6b1d\") " pod="openshift-ovn-kubernetes/ovnkube-node-tv82v" Jul 06 15:33:45.915037 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:45.913253 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/e67f3ab6-afb7-47ff-8219-adc98fe79e74-konnectivity-ca\") pod \"konnectivity-agent-hq8kh\" (UID: \"e67f3ab6-afb7-47ff-8219-adc98fe79e74\") " pod="kube-system/konnectivity-agent-hq8kh" Jul 06 15:33:45.915037 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:45.913289 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/cba33cb2-37bb-421e-9c92-e8d5894c6b1d-etc-openvswitch\") pod \"ovnkube-node-tv82v\" (UID: \"cba33cb2-37bb-421e-9c92-e8d5894c6b1d\") " pod="openshift-ovn-kubernetes/ovnkube-node-tv82v" Jul 06 15:33:45.915037 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:45.913304 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/1aead9b0-c187-46cb-b965-57f2cb23d421-socket-dir\") pod \"aws-ebs-csi-driver-node-7s97z\" (UID: \"1aead9b0-c187-46cb-b965-57f2cb23d421\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7s97z" Jul 06 15:33:45.915037 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:45.913320 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/cba33cb2-37bb-421e-9c92-e8d5894c6b1d-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-tv82v\" (UID: \"cba33cb2-37bb-421e-9c92-e8d5894c6b1d\") " pod="openshift-ovn-kubernetes/ovnkube-node-tv82v" Jul 06 15:33:45.915037 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:45.913351 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/0ce018e7-3a1f-43dc-8988-5d8054f04c69-host\") pod \"node-ca-rmsk7\" (UID: \"0ce018e7-3a1f-43dc-8988-5d8054f04c69\") " pod="openshift-image-registry/node-ca-rmsk7" Jul 06 15:33:45.915037 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:45.913374 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/bce194c9-b363-4021-bb8a-c7cc65facc39-dbus\") pod \"global-pull-secret-syncer-w9qbt\" (UID: \"bce194c9-b363-4021-bb8a-c7cc65facc39\") " pod="kube-system/global-pull-secret-syncer-w9qbt" Jul 06 15:33:45.915037 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:45.913411 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/a83f40ad-b56d-4d3f-9275-1a09f0c89e0c-hostroot\") pod \"multus-8zxfb\" (UID: \"a83f40ad-b56d-4d3f-9275-1a09f0c89e0c\") " pod="openshift-multus/multus-8zxfb" Jul 06 15:33:45.915037 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:45.913471 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/e67f3ab6-afb7-47ff-8219-adc98fe79e74-agent-certs\") pod \"konnectivity-agent-hq8kh\" (UID: \"e67f3ab6-afb7-47ff-8219-adc98fe79e74\") " pod="kube-system/konnectivity-agent-hq8kh" Jul 06 15:33:45.915693 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:45.913515 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5771d1de-202e-4920-b0de-5d12a4e4e2fc-metrics-certs\") pod \"network-metrics-daemon-v8fxz\" (UID: \"5771d1de-202e-4920-b0de-5d12a4e4e2fc\") " pod="openshift-multus/network-metrics-daemon-v8fxz" Jul 06 15:33:45.915693 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:45.913611 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/a3b0e3f1-74a1-4f15-9770-1a8ac2a02c4f-hosts-file\") pod \"node-resolver-pgkmz\" (UID: \"a3b0e3f1-74a1-4f15-9770-1a8ac2a02c4f\") " pod="openshift-dns/node-resolver-pgkmz" Jul 06 15:33:45.915693 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:45.913637 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/1aead9b0-c187-46cb-b965-57f2cb23d421-registration-dir\") pod \"aws-ebs-csi-driver-node-7s97z\" (UID: \"1aead9b0-c187-46cb-b965-57f2cb23d421\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7s97z" Jul 06 15:33:45.915693 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:45.913669 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/a83f40ad-b56d-4d3f-9275-1a09f0c89e0c-multus-socket-dir-parent\") pod \"multus-8zxfb\" (UID: \"a83f40ad-b56d-4d3f-9275-1a09f0c89e0c\") " pod="openshift-multus/multus-8zxfb" Jul 06 15:33:45.915693 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:45.913694 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/a83f40ad-b56d-4d3f-9275-1a09f0c89e0c-host-var-lib-cni-multus\") pod \"multus-8zxfb\" (UID: \"a83f40ad-b56d-4d3f-9275-1a09f0c89e0c\") " pod="openshift-multus/multus-8zxfb" Jul 06 15:33:45.915693 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:45.913727 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/cba33cb2-37bb-421e-9c92-e8d5894c6b1d-systemd-units\") pod \"ovnkube-node-tv82v\" (UID: \"cba33cb2-37bb-421e-9c92-e8d5894c6b1d\") " pod="openshift-ovn-kubernetes/ovnkube-node-tv82v" Jul 06 15:33:45.915693 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:45.913755 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/a3b0e3f1-74a1-4f15-9770-1a8ac2a02c4f-hosts-file\") pod \"node-resolver-pgkmz\" (UID: \"a3b0e3f1-74a1-4f15-9770-1a8ac2a02c4f\") " pod="openshift-dns/node-resolver-pgkmz" Jul 06 15:33:45.915693 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:45.913756 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/cba33cb2-37bb-421e-9c92-e8d5894c6b1d-host-run-netns\") pod \"ovnkube-node-tv82v\" (UID: \"cba33cb2-37bb-421e-9c92-e8d5894c6b1d\") " pod="openshift-ovn-kubernetes/ovnkube-node-tv82v" Jul 06 15:33:45.915693 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:45.913791 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/cba33cb2-37bb-421e-9c92-e8d5894c6b1d-host-run-netns\") pod \"ovnkube-node-tv82v\" (UID: \"cba33cb2-37bb-421e-9c92-e8d5894c6b1d\") " pod="openshift-ovn-kubernetes/ovnkube-node-tv82v" Jul 06 15:33:45.915693 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:45.913832 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/cba33cb2-37bb-421e-9c92-e8d5894c6b1d-systemd-units\") pod \"ovnkube-node-tv82v\" (UID: \"cba33cb2-37bb-421e-9c92-e8d5894c6b1d\") " pod="openshift-ovn-kubernetes/ovnkube-node-tv82v" Jul 06 15:33:45.915693 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:45.913861 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/cba33cb2-37bb-421e-9c92-e8d5894c6b1d-ovnkube-config\") pod \"ovnkube-node-tv82v\" (UID: \"cba33cb2-37bb-421e-9c92-e8d5894c6b1d\") " pod="openshift-ovn-kubernetes/ovnkube-node-tv82v" Jul 06 15:33:45.915693 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:45.914244 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/cba33cb2-37bb-421e-9c92-e8d5894c6b1d-ovnkube-config\") pod \"ovnkube-node-tv82v\" (UID: \"cba33cb2-37bb-421e-9c92-e8d5894c6b1d\") " pod="openshift-ovn-kubernetes/ovnkube-node-tv82v" Jul 06 15:33:45.915693 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:45.915598 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b8afa6a4-5899-4409-87d5-de85f2a850ac-tmp\") pod \"tuned-d2c7p\" (UID: \"b8afa6a4-5899-4409-87d5-de85f2a850ac\") " pod="openshift-cluster-node-tuning-operator/tuned-d2c7p" Jul 06 15:33:45.915693 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:45.915684 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/cba33cb2-37bb-421e-9c92-e8d5894c6b1d-ovn-node-metrics-cert\") pod \"ovnkube-node-tv82v\" (UID: \"cba33cb2-37bb-421e-9c92-e8d5894c6b1d\") " pod="openshift-ovn-kubernetes/ovnkube-node-tv82v" Jul 06 15:33:45.916355 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:45.915820 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/b8afa6a4-5899-4409-87d5-de85f2a850ac-etc-tuned\") pod \"tuned-d2c7p\" (UID: \"b8afa6a4-5899-4409-87d5-de85f2a850ac\") " pod="openshift-cluster-node-tuning-operator/tuned-d2c7p" Jul 06 15:33:45.916355 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:45.916205 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/e67f3ab6-afb7-47ff-8219-adc98fe79e74-agent-certs\") pod \"konnectivity-agent-hq8kh\" (UID: \"e67f3ab6-afb7-47ff-8219-adc98fe79e74\") " pod="kube-system/konnectivity-agent-hq8kh" Jul 06 15:33:45.925994 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:45.925970 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gk7r6\" (UniqueName: \"kubernetes.io/projected/cba33cb2-37bb-421e-9c92-e8d5894c6b1d-kube-api-access-gk7r6\") pod \"ovnkube-node-tv82v\" (UID: \"cba33cb2-37bb-421e-9c92-e8d5894c6b1d\") " pod="openshift-ovn-kubernetes/ovnkube-node-tv82v" Jul 06 15:33:45.927037 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:45.927019 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-22rxw\" (UniqueName: \"kubernetes.io/projected/b8afa6a4-5899-4409-87d5-de85f2a850ac-kube-api-access-22rxw\") pod \"tuned-d2c7p\" (UID: \"b8afa6a4-5899-4409-87d5-de85f2a850ac\") " pod="openshift-cluster-node-tuning-operator/tuned-d2c7p" Jul 06 15:33:45.927287 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:45.927269 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pgqlp\" (UniqueName: \"kubernetes.io/projected/a3b0e3f1-74a1-4f15-9770-1a8ac2a02c4f-kube-api-access-pgqlp\") pod \"node-resolver-pgkmz\" (UID: \"a3b0e3f1-74a1-4f15-9770-1a8ac2a02c4f\") " pod="openshift-dns/node-resolver-pgkmz" Jul 06 15:33:45.943632 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:45.943582 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-154.ec2.internal" event={"ID":"aebce7470264e8df37783ba3c605e8cd","Type":"ContainerStarted","Data":"0ae41df9185e56db90ba08e15b0e9631e4ed3a07819d63d9731b288db8a8960c"} Jul 06 15:33:45.945206 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:45.945180 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-154.ec2.internal" event={"ID":"89a933a4336ad4eeb3842cbd7d2c9e02","Type":"ContainerStarted","Data":"801ebe6e5a94f1038bef8891e837349211552bf2bfd62f9753a8032b96e21fba"} Jul 06 15:33:46.014514 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:46.014480 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/1aead9b0-c187-46cb-b965-57f2cb23d421-device-dir\") pod \"aws-ebs-csi-driver-node-7s97z\" (UID: \"1aead9b0-c187-46cb-b965-57f2cb23d421\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7s97z" Jul 06 15:33:46.014514 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:46.014511 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/1aead9b0-c187-46cb-b965-57f2cb23d421-sys-fs\") pod \"aws-ebs-csi-driver-node-7s97z\" (UID: \"1aead9b0-c187-46cb-b965-57f2cb23d421\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7s97z" Jul 06 15:33:46.014694 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:46.014535 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a83f40ad-b56d-4d3f-9275-1a09f0c89e0c-cni-binary-copy\") pod \"multus-8zxfb\" (UID: \"a83f40ad-b56d-4d3f-9275-1a09f0c89e0c\") " pod="openshift-multus/multus-8zxfb" Jul 06 15:33:46.014694 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:46.014563 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jcgpr\" (UniqueName: \"kubernetes.io/projected/1aead9b0-c187-46cb-b965-57f2cb23d421-kube-api-access-jcgpr\") pod \"aws-ebs-csi-driver-node-7s97z\" (UID: \"1aead9b0-c187-46cb-b965-57f2cb23d421\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7s97z" Jul 06 15:33:46.014694 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:46.014589 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a83f40ad-b56d-4d3f-9275-1a09f0c89e0c-cnibin\") pod \"multus-8zxfb\" (UID: \"a83f40ad-b56d-4d3f-9275-1a09f0c89e0c\") " pod="openshift-multus/multus-8zxfb" Jul 06 15:33:46.014694 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:46.014612 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a83f40ad-b56d-4d3f-9275-1a09f0c89e0c-multus-conf-dir\") pod \"multus-8zxfb\" (UID: \"a83f40ad-b56d-4d3f-9275-1a09f0c89e0c\") " pod="openshift-multus/multus-8zxfb" Jul 06 15:33:46.014694 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:46.014615 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/1aead9b0-c187-46cb-b965-57f2cb23d421-device-dir\") pod \"aws-ebs-csi-driver-node-7s97z\" (UID: \"1aead9b0-c187-46cb-b965-57f2cb23d421\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7s97z" Jul 06 15:33:46.014694 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:46.014630 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/1aead9b0-c187-46cb-b965-57f2cb23d421-sys-fs\") pod \"aws-ebs-csi-driver-node-7s97z\" (UID: \"1aead9b0-c187-46cb-b965-57f2cb23d421\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7s97z" Jul 06 15:33:46.014694 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:46.014636 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e32dedb7-fb62-41a5-9126-63a337f62a53-tuning-conf-dir\") pod \"multus-additional-cni-plugins-nmfg6\" (UID: \"e32dedb7-fb62-41a5-9126-63a337f62a53\") " pod="openshift-multus/multus-additional-cni-plugins-nmfg6" Jul 06 15:33:46.014694 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:46.014686 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/1aead9b0-c187-46cb-b965-57f2cb23d421-socket-dir\") pod \"aws-ebs-csi-driver-node-7s97z\" (UID: \"1aead9b0-c187-46cb-b965-57f2cb23d421\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7s97z" Jul 06 15:33:46.014694 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:46.014692 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a83f40ad-b56d-4d3f-9275-1a09f0c89e0c-cnibin\") pod \"multus-8zxfb\" (UID: \"a83f40ad-b56d-4d3f-9275-1a09f0c89e0c\") " pod="openshift-multus/multus-8zxfb" Jul 06 15:33:46.015093 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:46.014708 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/0ce018e7-3a1f-43dc-8988-5d8054f04c69-host\") pod \"node-ca-rmsk7\" (UID: \"0ce018e7-3a1f-43dc-8988-5d8054f04c69\") " pod="openshift-image-registry/node-ca-rmsk7" Jul 06 15:33:46.015093 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:46.014727 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/bce194c9-b363-4021-bb8a-c7cc65facc39-dbus\") pod \"global-pull-secret-syncer-w9qbt\" (UID: \"bce194c9-b363-4021-bb8a-c7cc65facc39\") " pod="kube-system/global-pull-secret-syncer-w9qbt" Jul 06 15:33:46.015093 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:46.014730 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a83f40ad-b56d-4d3f-9275-1a09f0c89e0c-multus-conf-dir\") pod \"multus-8zxfb\" (UID: \"a83f40ad-b56d-4d3f-9275-1a09f0c89e0c\") " pod="openshift-multus/multus-8zxfb" Jul 06 15:33:46.015093 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:46.014760 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/a83f40ad-b56d-4d3f-9275-1a09f0c89e0c-hostroot\") pod \"multus-8zxfb\" (UID: \"a83f40ad-b56d-4d3f-9275-1a09f0c89e0c\") " pod="openshift-multus/multus-8zxfb" Jul 06 15:33:46.015093 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:46.014786 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5771d1de-202e-4920-b0de-5d12a4e4e2fc-metrics-certs\") pod \"network-metrics-daemon-v8fxz\" (UID: \"5771d1de-202e-4920-b0de-5d12a4e4e2fc\") " pod="openshift-multus/network-metrics-daemon-v8fxz" Jul 06 15:33:46.015093 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:46.014812 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e32dedb7-fb62-41a5-9126-63a337f62a53-tuning-conf-dir\") pod \"multus-additional-cni-plugins-nmfg6\" (UID: \"e32dedb7-fb62-41a5-9126-63a337f62a53\") " pod="openshift-multus/multus-additional-cni-plugins-nmfg6" Jul 06 15:33:46.015093 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:46.014825 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/a83f40ad-b56d-4d3f-9275-1a09f0c89e0c-hostroot\") pod \"multus-8zxfb\" (UID: \"a83f40ad-b56d-4d3f-9275-1a09f0c89e0c\") " pod="openshift-multus/multus-8zxfb" Jul 06 15:33:46.015093 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:46.014850 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/1aead9b0-c187-46cb-b965-57f2cb23d421-registration-dir\") pod \"aws-ebs-csi-driver-node-7s97z\" (UID: \"1aead9b0-c187-46cb-b965-57f2cb23d421\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7s97z" Jul 06 15:33:46.015093 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:46.014876 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/a83f40ad-b56d-4d3f-9275-1a09f0c89e0c-multus-socket-dir-parent\") pod \"multus-8zxfb\" (UID: \"a83f40ad-b56d-4d3f-9275-1a09f0c89e0c\") " pod="openshift-multus/multus-8zxfb" Jul 06 15:33:46.015093 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:46.014900 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/a83f40ad-b56d-4d3f-9275-1a09f0c89e0c-host-var-lib-cni-multus\") pod \"multus-8zxfb\" (UID: \"a83f40ad-b56d-4d3f-9275-1a09f0c89e0c\") " pod="openshift-multus/multus-8zxfb" Jul 06 15:33:46.015093 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:46.014917 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/bce194c9-b363-4021-bb8a-c7cc65facc39-kubelet-config\") pod \"global-pull-secret-syncer-w9qbt\" (UID: \"bce194c9-b363-4021-bb8a-c7cc65facc39\") " pod="kube-system/global-pull-secret-syncer-w9qbt" Jul 06 15:33:46.015093 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:46.014940 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e32dedb7-fb62-41a5-9126-63a337f62a53-cni-binary-copy\") pod \"multus-additional-cni-plugins-nmfg6\" (UID: \"e32dedb7-fb62-41a5-9126-63a337f62a53\") " pod="openshift-multus/multus-additional-cni-plugins-nmfg6" Jul 06 15:33:46.015093 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:46.014963 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/0ce018e7-3a1f-43dc-8988-5d8054f04c69-host\") pod \"node-ca-rmsk7\" (UID: \"0ce018e7-3a1f-43dc-8988-5d8054f04c69\") " pod="openshift-image-registry/node-ca-rmsk7" Jul 06 15:33:46.015093 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:46.014966 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a83f40ad-b56d-4d3f-9275-1a09f0c89e0c-host-var-lib-cni-bin\") pod \"multus-8zxfb\" (UID: \"a83f40ad-b56d-4d3f-9275-1a09f0c89e0c\") " pod="openshift-multus/multus-8zxfb" Jul 06 15:33:46.015093 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:46.014999 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a83f40ad-b56d-4d3f-9275-1a09f0c89e0c-host-var-lib-cni-bin\") pod \"multus-8zxfb\" (UID: \"a83f40ad-b56d-4d3f-9275-1a09f0c89e0c\") " pod="openshift-multus/multus-8zxfb" Jul 06 15:33:46.015093 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:46.015001 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/a83f40ad-b56d-4d3f-9275-1a09f0c89e0c-host-run-multus-certs\") pod \"multus-8zxfb\" (UID: \"a83f40ad-b56d-4d3f-9275-1a09f0c89e0c\") " pod="openshift-multus/multus-8zxfb" Jul 06 15:33:46.015093 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:46.015027 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a83f40ad-b56d-4d3f-9275-1a09f0c89e0c-etc-kubernetes\") pod \"multus-8zxfb\" (UID: \"a83f40ad-b56d-4d3f-9275-1a09f0c89e0c\") " pod="openshift-multus/multus-8zxfb" Jul 06 15:33:46.015093 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:46.015033 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/a83f40ad-b56d-4d3f-9275-1a09f0c89e0c-host-run-multus-certs\") pod \"multus-8zxfb\" (UID: \"a83f40ad-b56d-4d3f-9275-1a09f0c89e0c\") " pod="openshift-multus/multus-8zxfb" Jul 06 15:33:46.015810 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:46.015047 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fb6wt\" (UniqueName: \"kubernetes.io/projected/e32dedb7-fb62-41a5-9126-63a337f62a53-kube-api-access-fb6wt\") pod \"multus-additional-cni-plugins-nmfg6\" (UID: \"e32dedb7-fb62-41a5-9126-63a337f62a53\") " pod="openshift-multus/multus-additional-cni-plugins-nmfg6" Jul 06 15:33:46.015810 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:46.015089 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/1aead9b0-c187-46cb-b965-57f2cb23d421-registration-dir\") pod \"aws-ebs-csi-driver-node-7s97z\" (UID: \"1aead9b0-c187-46cb-b965-57f2cb23d421\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7s97z" Jul 06 15:33:46.015810 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:46.015112 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a83f40ad-b56d-4d3f-9275-1a09f0c89e0c-system-cni-dir\") pod \"multus-8zxfb\" (UID: \"a83f40ad-b56d-4d3f-9275-1a09f0c89e0c\") " pod="openshift-multus/multus-8zxfb" Jul 06 15:33:46.015810 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:46.015162 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/a83f40ad-b56d-4d3f-9275-1a09f0c89e0c-multus-socket-dir-parent\") pod \"multus-8zxfb\" (UID: \"a83f40ad-b56d-4d3f-9275-1a09f0c89e0c\") " pod="openshift-multus/multus-8zxfb" Jul 06 15:33:46.015810 ip-10-0-129-154 kubenswrapper[2563]: E0706 15:33:46.015249 2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 15:33:46.015810 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:46.015295 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/bce194c9-b363-4021-bb8a-c7cc65facc39-dbus\") pod \"global-pull-secret-syncer-w9qbt\" (UID: \"bce194c9-b363-4021-bb8a-c7cc65facc39\") " pod="kube-system/global-pull-secret-syncer-w9qbt" Jul 06 15:33:46.015810 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:46.015306 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a83f40ad-b56d-4d3f-9275-1a09f0c89e0c-system-cni-dir\") pod \"multus-8zxfb\" (UID: \"a83f40ad-b56d-4d3f-9275-1a09f0c89e0c\") " pod="openshift-multus/multus-8zxfb" Jul 06 15:33:46.015810 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:46.015314 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/1aead9b0-c187-46cb-b965-57f2cb23d421-socket-dir\") pod \"aws-ebs-csi-driver-node-7s97z\" (UID: \"1aead9b0-c187-46cb-b965-57f2cb23d421\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7s97z" Jul 06 15:33:46.015810 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:46.015332 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/a83f40ad-b56d-4d3f-9275-1a09f0c89e0c-host-var-lib-cni-multus\") pod \"multus-8zxfb\" (UID: \"a83f40ad-b56d-4d3f-9275-1a09f0c89e0c\") " pod="openshift-multus/multus-8zxfb" Jul 06 15:33:46.015810 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:46.015356 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a83f40ad-b56d-4d3f-9275-1a09f0c89e0c-cni-binary-copy\") pod \"multus-8zxfb\" (UID: \"a83f40ad-b56d-4d3f-9275-1a09f0c89e0c\") " pod="openshift-multus/multus-8zxfb" Jul 06 15:33:46.015810 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:46.015368 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a83f40ad-b56d-4d3f-9275-1a09f0c89e0c-etc-kubernetes\") pod \"multus-8zxfb\" (UID: \"a83f40ad-b56d-4d3f-9275-1a09f0c89e0c\") " pod="openshift-multus/multus-8zxfb" Jul 06 15:33:46.015810 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:46.015366 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/bce194c9-b363-4021-bb8a-c7cc65facc39-kubelet-config\") pod \"global-pull-secret-syncer-w9qbt\" (UID: \"bce194c9-b363-4021-bb8a-c7cc65facc39\") " pod="kube-system/global-pull-secret-syncer-w9qbt" Jul 06 15:33:46.015810 ip-10-0-129-154 kubenswrapper[2563]: E0706 15:33:46.015389 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5771d1de-202e-4920-b0de-5d12a4e4e2fc-metrics-certs podName:5771d1de-202e-4920-b0de-5d12a4e4e2fc nodeName:}" failed. No retries permitted until 2026-07-06 15:33:46.515367605 +0000 UTC m=+3.024600780 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5771d1de-202e-4920-b0de-5d12a4e4e2fc-metrics-certs") pod "network-metrics-daemon-v8fxz" (UID: "5771d1de-202e-4920-b0de-5d12a4e4e2fc") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 15:33:46.015810 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:46.015408 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tj49r\" (UniqueName: \"kubernetes.io/projected/0ce018e7-3a1f-43dc-8988-5d8054f04c69-kube-api-access-tj49r\") pod \"node-ca-rmsk7\" (UID: \"0ce018e7-3a1f-43dc-8988-5d8054f04c69\") " pod="openshift-image-registry/node-ca-rmsk7" Jul 06 15:33:46.015810 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:46.015465 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/a83f40ad-b56d-4d3f-9275-1a09f0c89e0c-host-run-k8s-cni-cncf-io\") pod \"multus-8zxfb\" (UID: \"a83f40ad-b56d-4d3f-9275-1a09f0c89e0c\") " pod="openshift-multus/multus-8zxfb" Jul 06 15:33:46.015810 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:46.015491 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a83f40ad-b56d-4d3f-9275-1a09f0c89e0c-host-var-lib-kubelet\") pod \"multus-8zxfb\" (UID: \"a83f40ad-b56d-4d3f-9275-1a09f0c89e0c\") " pod="openshift-multus/multus-8zxfb" Jul 06 15:33:46.015810 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:46.015527 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-twj8p\" (UniqueName: \"kubernetes.io/projected/e6761760-82ef-43c3-99f0-9e027d7c859c-kube-api-access-twj8p\") pod \"network-check-target-pdw4n\" (UID: \"e6761760-82ef-43c3-99f0-9e027d7c859c\") " pod="openshift-network-diagnostics/network-check-target-pdw4n" Jul 06 15:33:46.016540 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:46.015552 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/11b1836e-d028-4749-baf8-3fc1e799d0be-host-slash\") pod \"iptables-alerter-bfx9b\" (UID: \"11b1836e-d028-4749-baf8-3fc1e799d0be\") " pod="openshift-network-operator/iptables-alerter-bfx9b" Jul 06 15:33:46.016540 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:46.015588 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a83f40ad-b56d-4d3f-9275-1a09f0c89e0c-multus-cni-dir\") pod \"multus-8zxfb\" (UID: \"a83f40ad-b56d-4d3f-9275-1a09f0c89e0c\") " pod="openshift-multus/multus-8zxfb" Jul 06 15:33:46.016540 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:46.015594 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a83f40ad-b56d-4d3f-9275-1a09f0c89e0c-host-var-lib-kubelet\") pod \"multus-8zxfb\" (UID: \"a83f40ad-b56d-4d3f-9275-1a09f0c89e0c\") " pod="openshift-multus/multus-8zxfb" Jul 06 15:33:46.016540 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:46.015631 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dqbgv\" (UniqueName: \"kubernetes.io/projected/11b1836e-d028-4749-baf8-3fc1e799d0be-kube-api-access-dqbgv\") pod \"iptables-alerter-bfx9b\" (UID: \"11b1836e-d028-4749-baf8-3fc1e799d0be\") " pod="openshift-network-operator/iptables-alerter-bfx9b" Jul 06 15:33:46.016540 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:46.015654 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/a83f40ad-b56d-4d3f-9275-1a09f0c89e0c-host-run-k8s-cni-cncf-io\") pod \"multus-8zxfb\" (UID: \"a83f40ad-b56d-4d3f-9275-1a09f0c89e0c\") " pod="openshift-multus/multus-8zxfb" Jul 06 15:33:46.016540 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:46.015654 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a83f40ad-b56d-4d3f-9275-1a09f0c89e0c-multus-cni-dir\") pod \"multus-8zxfb\" (UID: \"a83f40ad-b56d-4d3f-9275-1a09f0c89e0c\") " pod="openshift-multus/multus-8zxfb" Jul 06 15:33:46.016540 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:46.015664 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e32dedb7-fb62-41a5-9126-63a337f62a53-system-cni-dir\") pod \"multus-additional-cni-plugins-nmfg6\" (UID: \"e32dedb7-fb62-41a5-9126-63a337f62a53\") " pod="openshift-multus/multus-additional-cni-plugins-nmfg6" Jul 06 15:33:46.016540 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:46.015703 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/11b1836e-d028-4749-baf8-3fc1e799d0be-host-slash\") pod \"iptables-alerter-bfx9b\" (UID: \"11b1836e-d028-4749-baf8-3fc1e799d0be\") " pod="openshift-network-operator/iptables-alerter-bfx9b" Jul 06 15:33:46.016540 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:46.015703 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e32dedb7-fb62-41a5-9126-63a337f62a53-system-cni-dir\") pod \"multus-additional-cni-plugins-nmfg6\" (UID: \"e32dedb7-fb62-41a5-9126-63a337f62a53\") " pod="openshift-multus/multus-additional-cni-plugins-nmfg6" Jul 06 15:33:46.016540 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:46.015740 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e32dedb7-fb62-41a5-9126-63a337f62a53-cni-binary-copy\") pod \"multus-additional-cni-plugins-nmfg6\" (UID: \"e32dedb7-fb62-41a5-9126-63a337f62a53\") " pod="openshift-multus/multus-additional-cni-plugins-nmfg6" Jul 06 15:33:46.016540 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:46.015785 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e32dedb7-fb62-41a5-9126-63a337f62a53-cnibin\") pod \"multus-additional-cni-plugins-nmfg6\" (UID: \"e32dedb7-fb62-41a5-9126-63a337f62a53\") " pod="openshift-multus/multus-additional-cni-plugins-nmfg6" Jul 06 15:33:46.016540 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:46.015821 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/1aead9b0-c187-46cb-b965-57f2cb23d421-etc-selinux\") pod \"aws-ebs-csi-driver-node-7s97z\" (UID: \"1aead9b0-c187-46cb-b965-57f2cb23d421\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7s97z" Jul 06 15:33:46.016540 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:46.015849 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e32dedb7-fb62-41a5-9126-63a337f62a53-cnibin\") pod \"multus-additional-cni-plugins-nmfg6\" (UID: \"e32dedb7-fb62-41a5-9126-63a337f62a53\") " pod="openshift-multus/multus-additional-cni-plugins-nmfg6" Jul 06 15:33:46.016540 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:46.015911 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/1aead9b0-c187-46cb-b965-57f2cb23d421-kubelet-dir\") pod \"aws-ebs-csi-driver-node-7s97z\" (UID: \"1aead9b0-c187-46cb-b965-57f2cb23d421\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7s97z" Jul 06 15:33:46.016540 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:46.015920 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/1aead9b0-c187-46cb-b965-57f2cb23d421-etc-selinux\") pod \"aws-ebs-csi-driver-node-7s97z\" (UID: \"1aead9b0-c187-46cb-b965-57f2cb23d421\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7s97z" Jul 06 15:33:46.016540 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:46.015959 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/1aead9b0-c187-46cb-b965-57f2cb23d421-kubelet-dir\") pod \"aws-ebs-csi-driver-node-7s97z\" (UID: \"1aead9b0-c187-46cb-b965-57f2cb23d421\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7s97z" Jul 06 15:33:46.016540 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:46.015976 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a83f40ad-b56d-4d3f-9275-1a09f0c89e0c-host-run-netns\") pod \"multus-8zxfb\" (UID: \"a83f40ad-b56d-4d3f-9275-1a09f0c89e0c\") " pod="openshift-multus/multus-8zxfb" Jul 06 15:33:46.017178 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:46.016011 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/a83f40ad-b56d-4d3f-9275-1a09f0c89e0c-multus-daemon-config\") pod \"multus-8zxfb\" (UID: \"a83f40ad-b56d-4d3f-9275-1a09f0c89e0c\") " pod="openshift-multus/multus-8zxfb" Jul 06 15:33:46.017178 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:46.016037 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kj6p9\" (UniqueName: \"kubernetes.io/projected/5771d1de-202e-4920-b0de-5d12a4e4e2fc-kube-api-access-kj6p9\") pod \"network-metrics-daemon-v8fxz\" (UID: \"5771d1de-202e-4920-b0de-5d12a4e4e2fc\") " pod="openshift-multus/network-metrics-daemon-v8fxz" Jul 06 15:33:46.017178 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:46.016064 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/11b1836e-d028-4749-baf8-3fc1e799d0be-iptables-alerter-script\") pod \"iptables-alerter-bfx9b\" (UID: \"11b1836e-d028-4749-baf8-3fc1e799d0be\") " pod="openshift-network-operator/iptables-alerter-bfx9b" Jul 06 15:33:46.017178 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:46.016077 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a83f40ad-b56d-4d3f-9275-1a09f0c89e0c-host-run-netns\") pod \"multus-8zxfb\" (UID: \"a83f40ad-b56d-4d3f-9275-1a09f0c89e0c\") " pod="openshift-multus/multus-8zxfb" Jul 06 15:33:46.017178 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:46.016091 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/e32dedb7-fb62-41a5-9126-63a337f62a53-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-nmfg6\" (UID: \"e32dedb7-fb62-41a5-9126-63a337f62a53\") " pod="openshift-multus/multus-additional-cni-plugins-nmfg6" Jul 06 15:33:46.017178 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:46.016121 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/e32dedb7-fb62-41a5-9126-63a337f62a53-os-release\") pod \"multus-additional-cni-plugins-nmfg6\" (UID: \"e32dedb7-fb62-41a5-9126-63a337f62a53\") " pod="openshift-multus/multus-additional-cni-plugins-nmfg6" Jul 06 15:33:46.017178 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:46.016150 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/e32dedb7-fb62-41a5-9126-63a337f62a53-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-nmfg6\" (UID: \"e32dedb7-fb62-41a5-9126-63a337f62a53\") " pod="openshift-multus/multus-additional-cni-plugins-nmfg6" Jul 06 15:33:46.017178 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:46.016178 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/0ce018e7-3a1f-43dc-8988-5d8054f04c69-serviceca\") pod \"node-ca-rmsk7\" (UID: \"0ce018e7-3a1f-43dc-8988-5d8054f04c69\") " pod="openshift-image-registry/node-ca-rmsk7" Jul 06 15:33:46.017178 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:46.016201 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/bce194c9-b363-4021-bb8a-c7cc65facc39-original-pull-secret\") pod \"global-pull-secret-syncer-w9qbt\" (UID: \"bce194c9-b363-4021-bb8a-c7cc65facc39\") " pod="kube-system/global-pull-secret-syncer-w9qbt" Jul 06 15:33:46.017178 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:46.016226 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/a83f40ad-b56d-4d3f-9275-1a09f0c89e0c-os-release\") pod \"multus-8zxfb\" (UID: \"a83f40ad-b56d-4d3f-9275-1a09f0c89e0c\") " pod="openshift-multus/multus-8zxfb" Jul 06 15:33:46.017178 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:46.016255 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-72wh7\" (UniqueName: \"kubernetes.io/projected/a83f40ad-b56d-4d3f-9275-1a09f0c89e0c-kube-api-access-72wh7\") pod \"multus-8zxfb\" (UID: \"a83f40ad-b56d-4d3f-9275-1a09f0c89e0c\") " pod="openshift-multus/multus-8zxfb" Jul 06 15:33:46.017178 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:46.016628 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/a83f40ad-b56d-4d3f-9275-1a09f0c89e0c-multus-daemon-config\") pod \"multus-8zxfb\" (UID: \"a83f40ad-b56d-4d3f-9275-1a09f0c89e0c\") " pod="openshift-multus/multus-8zxfb" Jul 06 15:33:46.017178 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:46.017047 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/e32dedb7-fb62-41a5-9126-63a337f62a53-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-nmfg6\" (UID: \"e32dedb7-fb62-41a5-9126-63a337f62a53\") " pod="openshift-multus/multus-additional-cni-plugins-nmfg6" Jul 06 15:33:46.017178 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:46.017063 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/0ce018e7-3a1f-43dc-8988-5d8054f04c69-serviceca\") pod \"node-ca-rmsk7\" (UID: \"0ce018e7-3a1f-43dc-8988-5d8054f04c69\") " pod="openshift-image-registry/node-ca-rmsk7" Jul 06 15:33:46.017178 ip-10-0-129-154 kubenswrapper[2563]: E0706 15:33:46.017139 2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 06 15:33:46.017178 ip-10-0-129-154 kubenswrapper[2563]: E0706 15:33:46.017186 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bce194c9-b363-4021-bb8a-c7cc65facc39-original-pull-secret podName:bce194c9-b363-4021-bb8a-c7cc65facc39 nodeName:}" failed. No retries permitted until 2026-07-06 15:33:46.517170601 +0000 UTC m=+3.026403776 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/bce194c9-b363-4021-bb8a-c7cc65facc39-original-pull-secret") pod "global-pull-secret-syncer-w9qbt" (UID: "bce194c9-b363-4021-bb8a-c7cc65facc39") : object "kube-system"/"original-pull-secret" not registered Jul 06 15:33:46.017902 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:46.017203 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/e32dedb7-fb62-41a5-9126-63a337f62a53-os-release\") pod \"multus-additional-cni-plugins-nmfg6\" (UID: \"e32dedb7-fb62-41a5-9126-63a337f62a53\") " pod="openshift-multus/multus-additional-cni-plugins-nmfg6" Jul 06 15:33:46.017902 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:46.017253 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/a83f40ad-b56d-4d3f-9275-1a09f0c89e0c-os-release\") pod \"multus-8zxfb\" (UID: \"a83f40ad-b56d-4d3f-9275-1a09f0c89e0c\") " pod="openshift-multus/multus-8zxfb" Jul 06 15:33:46.017902 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:46.017501 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/e32dedb7-fb62-41a5-9126-63a337f62a53-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-nmfg6\" (UID: \"e32dedb7-fb62-41a5-9126-63a337f62a53\") " pod="openshift-multus/multus-additional-cni-plugins-nmfg6" Jul 06 15:33:46.017902 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:46.017607 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/11b1836e-d028-4749-baf8-3fc1e799d0be-iptables-alerter-script\") pod \"iptables-alerter-bfx9b\" (UID: \"11b1836e-d028-4749-baf8-3fc1e799d0be\") " pod="openshift-network-operator/iptables-alerter-bfx9b" Jul 06 15:33:46.024941 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:46.024920 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jcgpr\" (UniqueName: \"kubernetes.io/projected/1aead9b0-c187-46cb-b965-57f2cb23d421-kube-api-access-jcgpr\") pod \"aws-ebs-csi-driver-node-7s97z\" (UID: \"1aead9b0-c187-46cb-b965-57f2cb23d421\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7s97z" Jul 06 15:33:46.032682 ip-10-0-129-154 kubenswrapper[2563]: E0706 15:33:46.032638 2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 06 15:33:46.032682 ip-10-0-129-154 kubenswrapper[2563]: E0706 15:33:46.032661 2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 06 15:33:46.032682 ip-10-0-129-154 kubenswrapper[2563]: E0706 15:33:46.032674 2563 projected.go:196] Error preparing data for projected volume kube-api-access-twj8p for pod openshift-network-diagnostics/network-check-target-pdw4n: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 15:33:46.032892 ip-10-0-129-154 kubenswrapper[2563]: E0706 15:33:46.032763 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e6761760-82ef-43c3-99f0-9e027d7c859c-kube-api-access-twj8p podName:e6761760-82ef-43c3-99f0-9e027d7c859c nodeName:}" failed. No retries permitted until 2026-07-06 15:33:46.53274839 +0000 UTC m=+3.041981576 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-twj8p" (UniqueName: "kubernetes.io/projected/e6761760-82ef-43c3-99f0-9e027d7c859c-kube-api-access-twj8p") pod "network-check-target-pdw4n" (UID: "e6761760-82ef-43c3-99f0-9e027d7c859c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 15:33:46.035881 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:46.035856 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fb6wt\" (UniqueName: \"kubernetes.io/projected/e32dedb7-fb62-41a5-9126-63a337f62a53-kube-api-access-fb6wt\") pod \"multus-additional-cni-plugins-nmfg6\" (UID: \"e32dedb7-fb62-41a5-9126-63a337f62a53\") " pod="openshift-multus/multus-additional-cni-plugins-nmfg6" Jul 06 15:33:46.035980 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:46.035919 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-72wh7\" (UniqueName: \"kubernetes.io/projected/a83f40ad-b56d-4d3f-9275-1a09f0c89e0c-kube-api-access-72wh7\") pod \"multus-8zxfb\" (UID: \"a83f40ad-b56d-4d3f-9275-1a09f0c89e0c\") " pod="openshift-multus/multus-8zxfb" Jul 06 15:33:46.036249 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:46.036232 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tj49r\" (UniqueName: \"kubernetes.io/projected/0ce018e7-3a1f-43dc-8988-5d8054f04c69-kube-api-access-tj49r\") pod \"node-ca-rmsk7\" (UID: \"0ce018e7-3a1f-43dc-8988-5d8054f04c69\") " pod="openshift-image-registry/node-ca-rmsk7" Jul 06 15:33:46.037238 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:46.037214 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dqbgv\" (UniqueName: \"kubernetes.io/projected/11b1836e-d028-4749-baf8-3fc1e799d0be-kube-api-access-dqbgv\") pod \"iptables-alerter-bfx9b\" (UID: \"11b1836e-d028-4749-baf8-3fc1e799d0be\") " pod="openshift-network-operator/iptables-alerter-bfx9b" Jul 06 15:33:46.046164 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:46.046139 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kj6p9\" (UniqueName: \"kubernetes.io/projected/5771d1de-202e-4920-b0de-5d12a4e4e2fc-kube-api-access-kj6p9\") pod \"network-metrics-daemon-v8fxz\" (UID: \"5771d1de-202e-4920-b0de-5d12a4e4e2fc\") " pod="openshift-multus/network-metrics-daemon-v8fxz" Jul 06 15:33:46.103677 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:46.103648 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-d2c7p" Jul 06 15:33:46.111179 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:46.111151 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-tv82v" Jul 06 15:33:46.116707 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:46.116675 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb8afa6a4_5899_4409_87d5_de85f2a850ac.slice/crio-9b13da4aa1a1a0710e6780a849fc256c020053c532dc16d75ff4587a931207b3 WatchSource:0}: Error finding container 9b13da4aa1a1a0710e6780a849fc256c020053c532dc16d75ff4587a931207b3: Status 404 returned error can't find the container with id 9b13da4aa1a1a0710e6780a849fc256c020053c532dc16d75ff4587a931207b3 Jul 06 15:33:46.117798 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:46.117780 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-pgkmz" Jul 06 15:33:46.121966 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:46.121948 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-hq8kh" Jul 06 15:33:46.122324 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:46.122299 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcba33cb2_37bb_421e_9c92_e8d5894c6b1d.slice/crio-6ea152cf36ccfd78fb8dd994857ecacca6cbe14d76440608cb8e7c2ce3408c9c WatchSource:0}: Error finding container 6ea152cf36ccfd78fb8dd994857ecacca6cbe14d76440608cb8e7c2ce3408c9c: Status 404 returned error can't find the container with id 6ea152cf36ccfd78fb8dd994857ecacca6cbe14d76440608cb8e7c2ce3408c9c Jul 06 15:33:46.128627 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:46.128591 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda3b0e3f1_74a1_4f15_9770_1a8ac2a02c4f.slice/crio-a06008b842011cbfe167d825486fa1a0cfffbd025687a70fcbc34e1e7ed8abf4 WatchSource:0}: Error finding container a06008b842011cbfe167d825486fa1a0cfffbd025687a70fcbc34e1e7ed8abf4: Status 404 returned error can't find the container with id a06008b842011cbfe167d825486fa1a0cfffbd025687a70fcbc34e1e7ed8abf4 Jul 06 15:33:46.130206 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:46.130176 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7s97z" Jul 06 15:33:46.131878 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:46.131742 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode67f3ab6_afb7_47ff_8219_adc98fe79e74.slice/crio-3ad5b1edbeb31e475bb255bb4cb5f0b615adda3ddaa0c5d2ae6bb7288618fc11 WatchSource:0}: Error finding container 3ad5b1edbeb31e475bb255bb4cb5f0b615adda3ddaa0c5d2ae6bb7288618fc11: Status 404 returned error can't find the container with id 3ad5b1edbeb31e475bb255bb4cb5f0b615adda3ddaa0c5d2ae6bb7288618fc11 Jul 06 15:33:46.134938 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:46.134918 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-rmsk7" Jul 06 15:33:46.140176 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:46.140143 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1aead9b0_c187_46cb_b965_57f2cb23d421.slice/crio-fa84b7da2be1b8aeaa7c919c474ea1c30762642a5f64a0f61398661ef0ab21b7 WatchSource:0}: Error finding container fa84b7da2be1b8aeaa7c919c474ea1c30762642a5f64a0f61398661ef0ab21b7: Status 404 returned error can't find the container with id fa84b7da2be1b8aeaa7c919c474ea1c30762642a5f64a0f61398661ef0ab21b7 Jul 06 15:33:46.141010 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:46.140990 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-bfx9b" Jul 06 15:33:46.144206 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:46.144186 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0ce018e7_3a1f_43dc_8988_5d8054f04c69.slice/crio-401bd639183e2e32d34ed0af3cbef745550b74741c75e5b96f9846615ad983be WatchSource:0}: Error finding container 401bd639183e2e32d34ed0af3cbef745550b74741c75e5b96f9846615ad983be: Status 404 returned error can't find the container with id 401bd639183e2e32d34ed0af3cbef745550b74741c75e5b96f9846615ad983be Jul 06 15:33:46.146805 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:46.146791 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-8zxfb" Jul 06 15:33:46.150864 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:46.150842 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod11b1836e_d028_4749_baf8_3fc1e799d0be.slice/crio-7dffb62b3ad62db9a3f55921e1239453d724daf7e6c6d40b88cf7e4a408ad865 WatchSource:0}: Error finding container 7dffb62b3ad62db9a3f55921e1239453d724daf7e6c6d40b88cf7e4a408ad865: Status 404 returned error can't find the container with id 7dffb62b3ad62db9a3f55921e1239453d724daf7e6c6d40b88cf7e4a408ad865 Jul 06 15:33:46.151750 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:46.151731 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-nmfg6" Jul 06 15:33:46.156442 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:46.156409 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda83f40ad_b56d_4d3f_9275_1a09f0c89e0c.slice/crio-4cfc94f08913a425c3064341ee39279ed57f3baf4fb2a66f3b899c7d4e2b2bd1 WatchSource:0}: Error finding container 4cfc94f08913a425c3064341ee39279ed57f3baf4fb2a66f3b899c7d4e2b2bd1: Status 404 returned error can't find the container with id 4cfc94f08913a425c3064341ee39279ed57f3baf4fb2a66f3b899c7d4e2b2bd1 Jul 06 15:33:46.161131 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:33:46.161095 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode32dedb7_fb62_41a5_9126_63a337f62a53.slice/crio-78f1911ee4e35c42e239d4140c3502a02dc88ff658b15a446080e62cb54cb5ef WatchSource:0}: Error finding container 78f1911ee4e35c42e239d4140c3502a02dc88ff658b15a446080e62cb54cb5ef: Status 404 returned error can't find the container with id 78f1911ee4e35c42e239d4140c3502a02dc88ff658b15a446080e62cb54cb5ef Jul 06 15:33:46.210072 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:46.210049 2563 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 06 15:33:46.520446 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:46.520346 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/bce194c9-b363-4021-bb8a-c7cc65facc39-original-pull-secret\") pod \"global-pull-secret-syncer-w9qbt\" (UID: \"bce194c9-b363-4021-bb8a-c7cc65facc39\") " pod="kube-system/global-pull-secret-syncer-w9qbt" Jul 06 15:33:46.520446 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:46.520388 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5771d1de-202e-4920-b0de-5d12a4e4e2fc-metrics-certs\") pod \"network-metrics-daemon-v8fxz\" (UID: \"5771d1de-202e-4920-b0de-5d12a4e4e2fc\") " pod="openshift-multus/network-metrics-daemon-v8fxz" Jul 06 15:33:46.520645 ip-10-0-129-154 kubenswrapper[2563]: E0706 15:33:46.520533 2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 06 15:33:46.520645 ip-10-0-129-154 kubenswrapper[2563]: E0706 15:33:46.520606 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bce194c9-b363-4021-bb8a-c7cc65facc39-original-pull-secret podName:bce194c9-b363-4021-bb8a-c7cc65facc39 nodeName:}" failed. No retries permitted until 2026-07-06 15:33:47.520587104 +0000 UTC m=+4.029820269 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/bce194c9-b363-4021-bb8a-c7cc65facc39-original-pull-secret") pod "global-pull-secret-syncer-w9qbt" (UID: "bce194c9-b363-4021-bb8a-c7cc65facc39") : object "kube-system"/"original-pull-secret" not registered Jul 06 15:33:46.520854 ip-10-0-129-154 kubenswrapper[2563]: E0706 15:33:46.520533 2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 15:33:46.520975 ip-10-0-129-154 kubenswrapper[2563]: E0706 15:33:46.520898 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5771d1de-202e-4920-b0de-5d12a4e4e2fc-metrics-certs podName:5771d1de-202e-4920-b0de-5d12a4e4e2fc nodeName:}" failed. No retries permitted until 2026-07-06 15:33:47.520878144 +0000 UTC m=+4.030111304 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5771d1de-202e-4920-b0de-5d12a4e4e2fc-metrics-certs") pod "network-metrics-daemon-v8fxz" (UID: "5771d1de-202e-4920-b0de-5d12a4e4e2fc") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 15:33:46.621042 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:46.621006 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-twj8p\" (UniqueName: \"kubernetes.io/projected/e6761760-82ef-43c3-99f0-9e027d7c859c-kube-api-access-twj8p\") pod \"network-check-target-pdw4n\" (UID: \"e6761760-82ef-43c3-99f0-9e027d7c859c\") " pod="openshift-network-diagnostics/network-check-target-pdw4n" Jul 06 15:33:46.621220 ip-10-0-129-154 kubenswrapper[2563]: E0706 15:33:46.621197 2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 06 15:33:46.621272 ip-10-0-129-154 kubenswrapper[2563]: E0706 15:33:46.621227 2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 06 15:33:46.621272 ip-10-0-129-154 kubenswrapper[2563]: E0706 15:33:46.621239 2563 projected.go:196] Error preparing data for projected volume kube-api-access-twj8p for pod openshift-network-diagnostics/network-check-target-pdw4n: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 15:33:46.621364 ip-10-0-129-154 kubenswrapper[2563]: E0706 15:33:46.621295 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e6761760-82ef-43c3-99f0-9e027d7c859c-kube-api-access-twj8p podName:e6761760-82ef-43c3-99f0-9e027d7c859c nodeName:}" failed. No retries permitted until 2026-07-06 15:33:47.621278584 +0000 UTC m=+4.130511735 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-twj8p" (UniqueName: "kubernetes.io/projected/e6761760-82ef-43c3-99f0-9e027d7c859c-kube-api-access-twj8p") pod "network-check-target-pdw4n" (UID: "e6761760-82ef-43c3-99f0-9e027d7c859c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 15:33:46.889345 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:46.889305 2563 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-05 15:28:45 +0000 UTC" deadline="2027-12-04 13:37:37.692537598 +0000 UTC" Jul 06 15:33:46.889345 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:46.889341 2563 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12382h3m50.803202656s" Jul 06 15:33:46.939933 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:46.939905 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pdw4n" Jul 06 15:33:46.940087 ip-10-0-129-154 kubenswrapper[2563]: E0706 15:33:46.940019 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pdw4n" podUID="e6761760-82ef-43c3-99f0-9e027d7c859c" Jul 06 15:33:46.948029 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:46.947997 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-bfx9b" event={"ID":"11b1836e-d028-4749-baf8-3fc1e799d0be","Type":"ContainerStarted","Data":"7dffb62b3ad62db9a3f55921e1239453d724daf7e6c6d40b88cf7e4a408ad865"} Jul 06 15:33:46.949169 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:46.949146 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7s97z" event={"ID":"1aead9b0-c187-46cb-b965-57f2cb23d421","Type":"ContainerStarted","Data":"fa84b7da2be1b8aeaa7c919c474ea1c30762642a5f64a0f61398661ef0ab21b7"} Jul 06 15:33:46.950287 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:46.950264 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-pgkmz" event={"ID":"a3b0e3f1-74a1-4f15-9770-1a8ac2a02c4f","Type":"ContainerStarted","Data":"a06008b842011cbfe167d825486fa1a0cfffbd025687a70fcbc34e1e7ed8abf4"} Jul 06 15:33:46.951485 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:46.951458 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-tv82v" event={"ID":"cba33cb2-37bb-421e-9c92-e8d5894c6b1d","Type":"ContainerStarted","Data":"6ea152cf36ccfd78fb8dd994857ecacca6cbe14d76440608cb8e7c2ce3408c9c"} Jul 06 15:33:46.952582 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:46.952561 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-d2c7p" event={"ID":"b8afa6a4-5899-4409-87d5-de85f2a850ac","Type":"ContainerStarted","Data":"9b13da4aa1a1a0710e6780a849fc256c020053c532dc16d75ff4587a931207b3"} Jul 06 15:33:46.953684 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:46.953660 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-nmfg6" event={"ID":"e32dedb7-fb62-41a5-9126-63a337f62a53","Type":"ContainerStarted","Data":"78f1911ee4e35c42e239d4140c3502a02dc88ff658b15a446080e62cb54cb5ef"} Jul 06 15:33:46.954892 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:46.954873 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-8zxfb" event={"ID":"a83f40ad-b56d-4d3f-9275-1a09f0c89e0c","Type":"ContainerStarted","Data":"4cfc94f08913a425c3064341ee39279ed57f3baf4fb2a66f3b899c7d4e2b2bd1"} Jul 06 15:33:46.956138 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:46.956119 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-rmsk7" event={"ID":"0ce018e7-3a1f-43dc-8988-5d8054f04c69","Type":"ContainerStarted","Data":"401bd639183e2e32d34ed0af3cbef745550b74741c75e5b96f9846615ad983be"} Jul 06 15:33:46.957270 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:46.957248 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-hq8kh" event={"ID":"e67f3ab6-afb7-47ff-8219-adc98fe79e74","Type":"ContainerStarted","Data":"3ad5b1edbeb31e475bb255bb4cb5f0b615adda3ddaa0c5d2ae6bb7288618fc11"} Jul 06 15:33:47.527751 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:47.527672 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/bce194c9-b363-4021-bb8a-c7cc65facc39-original-pull-secret\") pod \"global-pull-secret-syncer-w9qbt\" (UID: \"bce194c9-b363-4021-bb8a-c7cc65facc39\") " pod="kube-system/global-pull-secret-syncer-w9qbt" Jul 06 15:33:47.527751 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:47.527725 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5771d1de-202e-4920-b0de-5d12a4e4e2fc-metrics-certs\") pod \"network-metrics-daemon-v8fxz\" (UID: \"5771d1de-202e-4920-b0de-5d12a4e4e2fc\") " pod="openshift-multus/network-metrics-daemon-v8fxz" Jul 06 15:33:47.527967 ip-10-0-129-154 kubenswrapper[2563]: E0706 15:33:47.527872 2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 15:33:47.527967 ip-10-0-129-154 kubenswrapper[2563]: E0706 15:33:47.527932 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5771d1de-202e-4920-b0de-5d12a4e4e2fc-metrics-certs podName:5771d1de-202e-4920-b0de-5d12a4e4e2fc nodeName:}" failed. No retries permitted until 2026-07-06 15:33:49.527914205 +0000 UTC m=+6.037147374 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5771d1de-202e-4920-b0de-5d12a4e4e2fc-metrics-certs") pod "network-metrics-daemon-v8fxz" (UID: "5771d1de-202e-4920-b0de-5d12a4e4e2fc") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 15:33:47.528336 ip-10-0-129-154 kubenswrapper[2563]: E0706 15:33:47.528312 2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 06 15:33:47.528465 ip-10-0-129-154 kubenswrapper[2563]: E0706 15:33:47.528367 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bce194c9-b363-4021-bb8a-c7cc65facc39-original-pull-secret podName:bce194c9-b363-4021-bb8a-c7cc65facc39 nodeName:}" failed. No retries permitted until 2026-07-06 15:33:49.528353409 +0000 UTC m=+6.037586576 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/bce194c9-b363-4021-bb8a-c7cc65facc39-original-pull-secret") pod "global-pull-secret-syncer-w9qbt" (UID: "bce194c9-b363-4021-bb8a-c7cc65facc39") : object "kube-system"/"original-pull-secret" not registered Jul 06 15:33:47.600468 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:47.598131 2563 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 06 15:33:47.628242 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:47.628203 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-twj8p\" (UniqueName: \"kubernetes.io/projected/e6761760-82ef-43c3-99f0-9e027d7c859c-kube-api-access-twj8p\") pod \"network-check-target-pdw4n\" (UID: \"e6761760-82ef-43c3-99f0-9e027d7c859c\") " pod="openshift-network-diagnostics/network-check-target-pdw4n" Jul 06 15:33:47.628450 ip-10-0-129-154 kubenswrapper[2563]: E0706 15:33:47.628414 2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 06 15:33:47.628521 ip-10-0-129-154 kubenswrapper[2563]: E0706 15:33:47.628458 2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 06 15:33:47.628521 ip-10-0-129-154 kubenswrapper[2563]: E0706 15:33:47.628471 2563 projected.go:196] Error preparing data for projected volume kube-api-access-twj8p for pod openshift-network-diagnostics/network-check-target-pdw4n: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 15:33:47.628628 ip-10-0-129-154 kubenswrapper[2563]: E0706 15:33:47.628530 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e6761760-82ef-43c3-99f0-9e027d7c859c-kube-api-access-twj8p podName:e6761760-82ef-43c3-99f0-9e027d7c859c nodeName:}" failed. No retries permitted until 2026-07-06 15:33:49.628512183 +0000 UTC m=+6.137745338 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-twj8p" (UniqueName: "kubernetes.io/projected/e6761760-82ef-43c3-99f0-9e027d7c859c-kube-api-access-twj8p") pod "network-check-target-pdw4n" (UID: "e6761760-82ef-43c3-99f0-9e027d7c859c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 15:33:47.890534 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:47.890401 2563 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-05 15:28:45 +0000 UTC" deadline="2028-01-17 02:48:06.086607757 +0000 UTC" Jul 06 15:33:47.890534 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:47.890457 2563 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13427h14m18.196155768s" Jul 06 15:33:47.942956 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:47.942522 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w9qbt" Jul 06 15:33:47.942956 ip-10-0-129-154 kubenswrapper[2563]: E0706 15:33:47.942656 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-w9qbt" podUID="bce194c9-b363-4021-bb8a-c7cc65facc39" Jul 06 15:33:47.952539 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:47.951567 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-v8fxz" Jul 06 15:33:47.952539 ip-10-0-129-154 kubenswrapper[2563]: E0706 15:33:47.951720 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-v8fxz" podUID="5771d1de-202e-4920-b0de-5d12a4e4e2fc" Jul 06 15:33:48.940057 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:48.939981 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pdw4n" Jul 06 15:33:48.940523 ip-10-0-129-154 kubenswrapper[2563]: E0706 15:33:48.940107 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pdw4n" podUID="e6761760-82ef-43c3-99f0-9e027d7c859c" Jul 06 15:33:48.978651 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:48.978612 2563 generic.go:356] "Generic (PLEG): container finished" podID="aebce7470264e8df37783ba3c605e8cd" containerID="dd4f8c54dd2db063f2e5039c62f53e2f875140c139a6c945b1b072916ab7caac" exitCode=0 Jul 06 15:33:48.979223 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:48.979197 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-154.ec2.internal" event={"ID":"aebce7470264e8df37783ba3c605e8cd","Type":"ContainerDied","Data":"dd4f8c54dd2db063f2e5039c62f53e2f875140c139a6c945b1b072916ab7caac"} Jul 06 15:33:49.000811 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:49.000708 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-154.ec2.internal" event={"ID":"89a933a4336ad4eeb3842cbd7d2c9e02","Type":"ContainerStarted","Data":"cf156d714222d2de14d4c24aaa0c1e850179c77c550fe38424efaa039ed824a0"} Jul 06 15:33:49.543924 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:49.543884 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/bce194c9-b363-4021-bb8a-c7cc65facc39-original-pull-secret\") pod \"global-pull-secret-syncer-w9qbt\" (UID: \"bce194c9-b363-4021-bb8a-c7cc65facc39\") " pod="kube-system/global-pull-secret-syncer-w9qbt" Jul 06 15:33:49.544124 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:49.543952 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5771d1de-202e-4920-b0de-5d12a4e4e2fc-metrics-certs\") pod \"network-metrics-daemon-v8fxz\" (UID: \"5771d1de-202e-4920-b0de-5d12a4e4e2fc\") " pod="openshift-multus/network-metrics-daemon-v8fxz" Jul 06 15:33:49.544124 ip-10-0-129-154 kubenswrapper[2563]: E0706 15:33:49.544117 2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 15:33:49.544227 ip-10-0-129-154 kubenswrapper[2563]: E0706 15:33:49.544178 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5771d1de-202e-4920-b0de-5d12a4e4e2fc-metrics-certs podName:5771d1de-202e-4920-b0de-5d12a4e4e2fc nodeName:}" failed. No retries permitted until 2026-07-06 15:33:53.544160326 +0000 UTC m=+10.053393496 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5771d1de-202e-4920-b0de-5d12a4e4e2fc-metrics-certs") pod "network-metrics-daemon-v8fxz" (UID: "5771d1de-202e-4920-b0de-5d12a4e4e2fc") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 15:33:49.544601 ip-10-0-129-154 kubenswrapper[2563]: E0706 15:33:49.544580 2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 06 15:33:49.544689 ip-10-0-129-154 kubenswrapper[2563]: E0706 15:33:49.544637 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bce194c9-b363-4021-bb8a-c7cc65facc39-original-pull-secret podName:bce194c9-b363-4021-bb8a-c7cc65facc39 nodeName:}" failed. No retries permitted until 2026-07-06 15:33:53.544622113 +0000 UTC m=+10.053855282 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/bce194c9-b363-4021-bb8a-c7cc65facc39-original-pull-secret") pod "global-pull-secret-syncer-w9qbt" (UID: "bce194c9-b363-4021-bb8a-c7cc65facc39") : object "kube-system"/"original-pull-secret" not registered Jul 06 15:33:49.644943 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:49.644347 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-twj8p\" (UniqueName: \"kubernetes.io/projected/e6761760-82ef-43c3-99f0-9e027d7c859c-kube-api-access-twj8p\") pod \"network-check-target-pdw4n\" (UID: \"e6761760-82ef-43c3-99f0-9e027d7c859c\") " pod="openshift-network-diagnostics/network-check-target-pdw4n" Jul 06 15:33:49.644943 ip-10-0-129-154 kubenswrapper[2563]: E0706 15:33:49.644514 2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 06 15:33:49.644943 ip-10-0-129-154 kubenswrapper[2563]: E0706 15:33:49.644538 2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 06 15:33:49.644943 ip-10-0-129-154 kubenswrapper[2563]: E0706 15:33:49.644552 2563 projected.go:196] Error preparing data for projected volume kube-api-access-twj8p for pod openshift-network-diagnostics/network-check-target-pdw4n: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 15:33:49.644943 ip-10-0-129-154 kubenswrapper[2563]: E0706 15:33:49.644609 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e6761760-82ef-43c3-99f0-9e027d7c859c-kube-api-access-twj8p podName:e6761760-82ef-43c3-99f0-9e027d7c859c nodeName:}" failed. No retries permitted until 2026-07-06 15:33:53.644590741 +0000 UTC m=+10.153823895 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-twj8p" (UniqueName: "kubernetes.io/projected/e6761760-82ef-43c3-99f0-9e027d7c859c-kube-api-access-twj8p") pod "network-check-target-pdw4n" (UID: "e6761760-82ef-43c3-99f0-9e027d7c859c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 15:33:49.942088 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:49.942001 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-v8fxz" Jul 06 15:33:49.942491 ip-10-0-129-154 kubenswrapper[2563]: E0706 15:33:49.942140 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-v8fxz" podUID="5771d1de-202e-4920-b0de-5d12a4e4e2fc" Jul 06 15:33:49.942546 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:49.942522 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w9qbt" Jul 06 15:33:49.942617 ip-10-0-129-154 kubenswrapper[2563]: E0706 15:33:49.942599 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-w9qbt" podUID="bce194c9-b363-4021-bb8a-c7cc65facc39" Jul 06 15:33:50.939943 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:50.939912 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pdw4n" Jul 06 15:33:50.940111 ip-10-0-129-154 kubenswrapper[2563]: E0706 15:33:50.940017 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pdw4n" podUID="e6761760-82ef-43c3-99f0-9e027d7c859c" Jul 06 15:33:51.939567 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:51.939537 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w9qbt" Jul 06 15:33:51.939934 ip-10-0-129-154 kubenswrapper[2563]: E0706 15:33:51.939680 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-w9qbt" podUID="bce194c9-b363-4021-bb8a-c7cc65facc39" Jul 06 15:33:51.940086 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:51.940063 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-v8fxz" Jul 06 15:33:51.940213 ip-10-0-129-154 kubenswrapper[2563]: E0706 15:33:51.940167 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-v8fxz" podUID="5771d1de-202e-4920-b0de-5d12a4e4e2fc" Jul 06 15:33:52.939900 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:52.939866 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pdw4n" Jul 06 15:33:52.940346 ip-10-0-129-154 kubenswrapper[2563]: E0706 15:33:52.939987 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pdw4n" podUID="e6761760-82ef-43c3-99f0-9e027d7c859c" Jul 06 15:33:53.576044 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:53.576008 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/bce194c9-b363-4021-bb8a-c7cc65facc39-original-pull-secret\") pod \"global-pull-secret-syncer-w9qbt\" (UID: \"bce194c9-b363-4021-bb8a-c7cc65facc39\") " pod="kube-system/global-pull-secret-syncer-w9qbt" Jul 06 15:33:53.576235 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:53.576059 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5771d1de-202e-4920-b0de-5d12a4e4e2fc-metrics-certs\") pod \"network-metrics-daemon-v8fxz\" (UID: \"5771d1de-202e-4920-b0de-5d12a4e4e2fc\") " pod="openshift-multus/network-metrics-daemon-v8fxz" Jul 06 15:33:53.576235 ip-10-0-129-154 kubenswrapper[2563]: E0706 15:33:53.576179 2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 15:33:53.576235 ip-10-0-129-154 kubenswrapper[2563]: E0706 15:33:53.576178 2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 06 15:33:53.576235 ip-10-0-129-154 kubenswrapper[2563]: E0706 15:33:53.576233 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5771d1de-202e-4920-b0de-5d12a4e4e2fc-metrics-certs podName:5771d1de-202e-4920-b0de-5d12a4e4e2fc nodeName:}" failed. No retries permitted until 2026-07-06 15:34:01.576217788 +0000 UTC m=+18.085450940 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5771d1de-202e-4920-b0de-5d12a4e4e2fc-metrics-certs") pod "network-metrics-daemon-v8fxz" (UID: "5771d1de-202e-4920-b0de-5d12a4e4e2fc") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 15:33:53.576477 ip-10-0-129-154 kubenswrapper[2563]: E0706 15:33:53.576251 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bce194c9-b363-4021-bb8a-c7cc65facc39-original-pull-secret podName:bce194c9-b363-4021-bb8a-c7cc65facc39 nodeName:}" failed. No retries permitted until 2026-07-06 15:34:01.576242771 +0000 UTC m=+18.085475926 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/bce194c9-b363-4021-bb8a-c7cc65facc39-original-pull-secret") pod "global-pull-secret-syncer-w9qbt" (UID: "bce194c9-b363-4021-bb8a-c7cc65facc39") : object "kube-system"/"original-pull-secret" not registered Jul 06 15:33:53.676625 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:53.676594 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-twj8p\" (UniqueName: \"kubernetes.io/projected/e6761760-82ef-43c3-99f0-9e027d7c859c-kube-api-access-twj8p\") pod \"network-check-target-pdw4n\" (UID: \"e6761760-82ef-43c3-99f0-9e027d7c859c\") " pod="openshift-network-diagnostics/network-check-target-pdw4n" Jul 06 15:33:53.676813 ip-10-0-129-154 kubenswrapper[2563]: E0706 15:33:53.676738 2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 06 15:33:53.676813 ip-10-0-129-154 kubenswrapper[2563]: E0706 15:33:53.676756 2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 06 15:33:53.676813 ip-10-0-129-154 kubenswrapper[2563]: E0706 15:33:53.676768 2563 projected.go:196] Error preparing data for projected volume kube-api-access-twj8p for pod openshift-network-diagnostics/network-check-target-pdw4n: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 15:33:53.676972 ip-10-0-129-154 kubenswrapper[2563]: E0706 15:33:53.676834 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e6761760-82ef-43c3-99f0-9e027d7c859c-kube-api-access-twj8p podName:e6761760-82ef-43c3-99f0-9e027d7c859c nodeName:}" failed. No retries permitted until 2026-07-06 15:34:01.676814591 +0000 UTC m=+18.186047756 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-twj8p" (UniqueName: "kubernetes.io/projected/e6761760-82ef-43c3-99f0-9e027d7c859c-kube-api-access-twj8p") pod "network-check-target-pdw4n" (UID: "e6761760-82ef-43c3-99f0-9e027d7c859c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 15:33:53.940852 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:53.940780 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w9qbt" Jul 06 15:33:53.941209 ip-10-0-129-154 kubenswrapper[2563]: E0706 15:33:53.940878 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-w9qbt" podUID="bce194c9-b363-4021-bb8a-c7cc65facc39" Jul 06 15:33:53.941209 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:53.941188 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-v8fxz" Jul 06 15:33:53.941283 ip-10-0-129-154 kubenswrapper[2563]: E0706 15:33:53.941254 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-v8fxz" podUID="5771d1de-202e-4920-b0de-5d12a4e4e2fc" Jul 06 15:33:54.939535 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:54.939496 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pdw4n" Jul 06 15:33:54.939731 ip-10-0-129-154 kubenswrapper[2563]: E0706 15:33:54.939614 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pdw4n" podUID="e6761760-82ef-43c3-99f0-9e027d7c859c" Jul 06 15:33:55.942765 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:55.942735 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-v8fxz" Jul 06 15:33:55.943212 ip-10-0-129-154 kubenswrapper[2563]: E0706 15:33:55.942865 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-v8fxz" podUID="5771d1de-202e-4920-b0de-5d12a4e4e2fc" Jul 06 15:33:55.943212 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:55.942905 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w9qbt" Jul 06 15:33:55.943212 ip-10-0-129-154 kubenswrapper[2563]: E0706 15:33:55.942989 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-w9qbt" podUID="bce194c9-b363-4021-bb8a-c7cc65facc39" Jul 06 15:33:56.939647 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:56.939608 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pdw4n" Jul 06 15:33:56.939811 ip-10-0-129-154 kubenswrapper[2563]: E0706 15:33:56.939735 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pdw4n" podUID="e6761760-82ef-43c3-99f0-9e027d7c859c" Jul 06 15:33:57.939566 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:57.939284 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-v8fxz" Jul 06 15:33:57.940444 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:57.939301 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w9qbt" Jul 06 15:33:57.940444 ip-10-0-129-154 kubenswrapper[2563]: E0706 15:33:57.939630 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-v8fxz" podUID="5771d1de-202e-4920-b0de-5d12a4e4e2fc" Jul 06 15:33:57.940444 ip-10-0-129-154 kubenswrapper[2563]: E0706 15:33:57.939677 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-w9qbt" podUID="bce194c9-b363-4021-bb8a-c7cc65facc39" Jul 06 15:33:58.026900 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:58.026870 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7s97z" event={"ID":"1aead9b0-c187-46cb-b965-57f2cb23d421","Type":"ContainerStarted","Data":"6b1e54552c5e00888f880bc398875c7103f7d2a6a977cefae80b7964cc39fadd"} Jul 06 15:33:58.028547 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:58.028490 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-pgkmz" event={"ID":"a3b0e3f1-74a1-4f15-9770-1a8ac2a02c4f","Type":"ContainerStarted","Data":"351f56679cf42e1fd8c6183135da76beb6e482500b5086deaf8cf5ae51945a62"} Jul 06 15:33:58.030149 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:58.030119 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-d2c7p" event={"ID":"b8afa6a4-5899-4409-87d5-de85f2a850ac","Type":"ContainerStarted","Data":"0a0a699edfd559b0475942620e2a05581cdb95adbd31a6674ff42031810241da"} Jul 06 15:33:58.031934 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:58.031907 2563 generic.go:356] "Generic (PLEG): container finished" podID="e32dedb7-fb62-41a5-9126-63a337f62a53" containerID="c2a4156a277c299c77794b296bc94dafef03d7021e438e14f3ca51c7bcfbe103" exitCode=0 Jul 06 15:33:58.032067 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:58.031990 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-nmfg6" event={"ID":"e32dedb7-fb62-41a5-9126-63a337f62a53","Type":"ContainerDied","Data":"c2a4156a277c299c77794b296bc94dafef03d7021e438e14f3ca51c7bcfbe103"} Jul 06 15:33:58.033757 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:58.033723 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-rmsk7" event={"ID":"0ce018e7-3a1f-43dc-8988-5d8054f04c69","Type":"ContainerStarted","Data":"19d7f676937189aa8d3504bd733164e4e1e0b68cf65ba17e09a4ed5330d2090e"} Jul 06 15:33:58.035550 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:58.035530 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-hq8kh" event={"ID":"e67f3ab6-afb7-47ff-8219-adc98fe79e74","Type":"ContainerStarted","Data":"3af7190d8313ef75be507cc2a0320df5c852404449c5cf922414dbb2de6c0d0b"} Jul 06 15:33:58.037691 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:58.037669 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-154.ec2.internal" event={"ID":"aebce7470264e8df37783ba3c605e8cd","Type":"ContainerStarted","Data":"1e9961a81810fbc16008edbbbbf73c95b3de76ad3b25ebe1d660ef87c167fc66"} Jul 06 15:33:58.057000 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:58.056941 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-154.ec2.internal" podStartSLOduration=13.056901304 podStartE2EDuration="13.056901304s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 15:33:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 15:33:49.018788576 +0000 UTC m=+5.528021756" watchObservedRunningTime="2026-07-06 15:33:58.056901304 +0000 UTC m=+14.566134479" Jul 06 15:33:58.057148 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:58.057121 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-pgkmz" podStartSLOduration=2.933397133 podStartE2EDuration="14.057114207s" totalImagesPullingTime="11.123717074s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 15:33:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 15:33:58.057073682 +0000 UTC m=+14.566306858" watchObservedRunningTime="2026-07-06 15:33:58.057114207 +0000 UTC m=+14.566347381" Jul 06 15:33:58.153208 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:58.153153 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-hq8kh" podStartSLOduration=3.031366361 podStartE2EDuration="14.153135292s" totalImagesPullingTime="11.121768931s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 15:33:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 15:33:58.152982695 +0000 UTC m=+14.662215868" watchObservedRunningTime="2026-07-06 15:33:58.153135292 +0000 UTC m=+14.662368467" Jul 06 15:33:58.153537 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:58.153512 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-d2c7p" podStartSLOduration=4.00097902 podStartE2EDuration="15.153501005s" totalImagesPullingTime="11.152521985s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 15:33:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 15:33:58.12467476 +0000 UTC m=+14.633907934" watchObservedRunningTime="2026-07-06 15:33:58.153501005 +0000 UTC m=+14.662734180" Jul 06 15:33:58.205711 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:58.205565 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-rmsk7" podStartSLOduration=3.176875763 podStartE2EDuration="14.205546418s" totalImagesPullingTime="11.028670655s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 15:33:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 15:33:58.204316034 +0000 UTC m=+14.713549209" watchObservedRunningTime="2026-07-06 15:33:58.205546418 +0000 UTC m=+14.714779591" Jul 06 15:33:58.227806 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:58.227751 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-154.ec2.internal" podStartSLOduration=13.227732987 podStartE2EDuration="13.227732987s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 15:33:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 15:33:58.227688769 +0000 UTC m=+14.736921944" watchObservedRunningTime="2026-07-06 15:33:58.227732987 +0000 UTC m=+14.736966162" Jul 06 15:33:58.940052 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:58.940019 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pdw4n" Jul 06 15:33:58.940576 ip-10-0-129-154 kubenswrapper[2563]: E0706 15:33:58.940146 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pdw4n" podUID="e6761760-82ef-43c3-99f0-9e027d7c859c" Jul 06 15:33:59.041789 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:59.041704 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-bfx9b" event={"ID":"11b1836e-d028-4749-baf8-3fc1e799d0be","Type":"ContainerStarted","Data":"7b99264803be4a908cdb297934e0e3f2b9569dd17f9a86bcec13a2c6dde8b9f8"} Jul 06 15:33:59.623323 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:59.623278 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-hq8kh" Jul 06 15:33:59.624213 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:59.624023 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-hq8kh" Jul 06 15:33:59.648569 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:59.648511 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-bfx9b" podStartSLOduration=4.545533178 podStartE2EDuration="15.648489752s" totalImagesPullingTime="11.102956574s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 15:33:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 15:33:59.067659579 +0000 UTC m=+15.576892744" watchObservedRunningTime="2026-07-06 15:33:59.648489752 +0000 UTC m=+16.157722927" Jul 06 15:33:59.939698 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:59.939620 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w9qbt" Jul 06 15:33:59.939861 ip-10-0-129-154 kubenswrapper[2563]: E0706 15:33:59.939737 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-w9qbt" podUID="bce194c9-b363-4021-bb8a-c7cc65facc39" Jul 06 15:33:59.940025 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:33:59.940011 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-v8fxz" Jul 06 15:33:59.940108 ip-10-0-129-154 kubenswrapper[2563]: E0706 15:33:59.940093 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-v8fxz" podUID="5771d1de-202e-4920-b0de-5d12a4e4e2fc" Jul 06 15:34:00.939737 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:00.939690 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pdw4n" Jul 06 15:34:00.939948 ip-10-0-129-154 kubenswrapper[2563]: E0706 15:34:00.939818 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pdw4n" podUID="e6761760-82ef-43c3-99f0-9e027d7c859c" Jul 06 15:34:01.046262 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:01.046234 2563 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Jul 06 15:34:01.639483 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:01.639446 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/bce194c9-b363-4021-bb8a-c7cc65facc39-original-pull-secret\") pod \"global-pull-secret-syncer-w9qbt\" (UID: \"bce194c9-b363-4021-bb8a-c7cc65facc39\") " pod="kube-system/global-pull-secret-syncer-w9qbt" Jul 06 15:34:01.639748 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:01.639501 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5771d1de-202e-4920-b0de-5d12a4e4e2fc-metrics-certs\") pod \"network-metrics-daemon-v8fxz\" (UID: \"5771d1de-202e-4920-b0de-5d12a4e4e2fc\") " pod="openshift-multus/network-metrics-daemon-v8fxz" Jul 06 15:34:01.639748 ip-10-0-129-154 kubenswrapper[2563]: E0706 15:34:01.639623 2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 15:34:01.639748 ip-10-0-129-154 kubenswrapper[2563]: E0706 15:34:01.639701 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5771d1de-202e-4920-b0de-5d12a4e4e2fc-metrics-certs podName:5771d1de-202e-4920-b0de-5d12a4e4e2fc nodeName:}" failed. No retries permitted until 2026-07-06 15:34:17.639678354 +0000 UTC m=+34.148911529 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5771d1de-202e-4920-b0de-5d12a4e4e2fc-metrics-certs") pod "network-metrics-daemon-v8fxz" (UID: "5771d1de-202e-4920-b0de-5d12a4e4e2fc") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 15:34:01.640031 ip-10-0-129-154 kubenswrapper[2563]: E0706 15:34:01.639625 2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 06 15:34:01.640102 ip-10-0-129-154 kubenswrapper[2563]: E0706 15:34:01.640069 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bce194c9-b363-4021-bb8a-c7cc65facc39-original-pull-secret podName:bce194c9-b363-4021-bb8a-c7cc65facc39 nodeName:}" failed. No retries permitted until 2026-07-06 15:34:17.64005232 +0000 UTC m=+34.149285473 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/bce194c9-b363-4021-bb8a-c7cc65facc39-original-pull-secret") pod "global-pull-secret-syncer-w9qbt" (UID: "bce194c9-b363-4021-bb8a-c7cc65facc39") : object "kube-system"/"original-pull-secret" not registered Jul 06 15:34:01.740875 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:01.740841 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-twj8p\" (UniqueName: \"kubernetes.io/projected/e6761760-82ef-43c3-99f0-9e027d7c859c-kube-api-access-twj8p\") pod \"network-check-target-pdw4n\" (UID: \"e6761760-82ef-43c3-99f0-9e027d7c859c\") " pod="openshift-network-diagnostics/network-check-target-pdw4n" Jul 06 15:34:01.741049 ip-10-0-129-154 kubenswrapper[2563]: E0706 15:34:01.740991 2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 06 15:34:01.741049 ip-10-0-129-154 kubenswrapper[2563]: E0706 15:34:01.741007 2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 06 15:34:01.741049 ip-10-0-129-154 kubenswrapper[2563]: E0706 15:34:01.741016 2563 projected.go:196] Error preparing data for projected volume kube-api-access-twj8p for pod openshift-network-diagnostics/network-check-target-pdw4n: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 15:34:01.741193 ip-10-0-129-154 kubenswrapper[2563]: E0706 15:34:01.741076 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e6761760-82ef-43c3-99f0-9e027d7c859c-kube-api-access-twj8p podName:e6761760-82ef-43c3-99f0-9e027d7c859c nodeName:}" failed. No retries permitted until 2026-07-06 15:34:17.741059543 +0000 UTC m=+34.250292718 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-twj8p" (UniqueName: "kubernetes.io/projected/e6761760-82ef-43c3-99f0-9e027d7c859c-kube-api-access-twj8p") pod "network-check-target-pdw4n" (UID: "e6761760-82ef-43c3-99f0-9e027d7c859c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 15:34:01.940062 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:01.939989 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w9qbt" Jul 06 15:34:01.940210 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:01.939990 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-v8fxz" Jul 06 15:34:01.940210 ip-10-0-129-154 kubenswrapper[2563]: E0706 15:34:01.940112 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-w9qbt" podUID="bce194c9-b363-4021-bb8a-c7cc65facc39" Jul 06 15:34:01.940315 ip-10-0-129-154 kubenswrapper[2563]: E0706 15:34:01.940222 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-v8fxz" podUID="5771d1de-202e-4920-b0de-5d12a4e4e2fc" Jul 06 15:34:02.940114 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:02.939675 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pdw4n" Jul 06 15:34:02.940114 ip-10-0-129-154 kubenswrapper[2563]: E0706 15:34:02.939804 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pdw4n" podUID="e6761760-82ef-43c3-99f0-9e027d7c859c" Jul 06 15:34:03.940926 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:03.940883 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-v8fxz" Jul 06 15:34:03.941463 ip-10-0-129-154 kubenswrapper[2563]: E0706 15:34:03.941025 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-v8fxz" podUID="5771d1de-202e-4920-b0de-5d12a4e4e2fc" Jul 06 15:34:03.941463 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:03.941082 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w9qbt" Jul 06 15:34:03.941463 ip-10-0-129-154 kubenswrapper[2563]: E0706 15:34:03.941188 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-w9qbt" podUID="bce194c9-b363-4021-bb8a-c7cc65facc39" Jul 06 15:34:04.940118 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:04.940065 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pdw4n" Jul 06 15:34:04.940307 ip-10-0-129-154 kubenswrapper[2563]: E0706 15:34:04.940194 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pdw4n" podUID="e6761760-82ef-43c3-99f0-9e027d7c859c" Jul 06 15:34:05.311450 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:05.311353 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-hq8kh" Jul 06 15:34:05.311873 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:05.311543 2563 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Jul 06 15:34:05.312054 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:05.312035 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-hq8kh" Jul 06 15:34:05.939166 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:05.939131 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-v8fxz" Jul 06 15:34:05.939166 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:05.939163 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w9qbt" Jul 06 15:34:05.939536 ip-10-0-129-154 kubenswrapper[2563]: E0706 15:34:05.939266 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-v8fxz" podUID="5771d1de-202e-4920-b0de-5d12a4e4e2fc" Jul 06 15:34:05.939536 ip-10-0-129-154 kubenswrapper[2563]: E0706 15:34:05.939402 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-w9qbt" podUID="bce194c9-b363-4021-bb8a-c7cc65facc39" Jul 06 15:34:06.939579 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:06.939547 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pdw4n" Jul 06 15:34:06.940033 ip-10-0-129-154 kubenswrapper[2563]: E0706 15:34:06.939679 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pdw4n" podUID="e6761760-82ef-43c3-99f0-9e027d7c859c" Jul 06 15:34:07.939918 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:07.939548 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w9qbt" Jul 06 15:34:07.939918 ip-10-0-129-154 kubenswrapper[2563]: E0706 15:34:07.939678 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-w9qbt" podUID="bce194c9-b363-4021-bb8a-c7cc65facc39" Jul 06 15:34:07.939918 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:07.939737 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-v8fxz" Jul 06 15:34:07.939918 ip-10-0-129-154 kubenswrapper[2563]: E0706 15:34:07.939866 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-v8fxz" podUID="5771d1de-202e-4920-b0de-5d12a4e4e2fc" Jul 06 15:34:07.951895 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:07.951872 2563 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 06 15:34:08.061463 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:08.061266 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7s97z" event={"ID":"1aead9b0-c187-46cb-b965-57f2cb23d421","Type":"ContainerStarted","Data":"715480d06a6898e37e43f8da854310370c7ee07a25bb5889649afcc40f40dc6e"} Jul 06 15:34:08.063079 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:08.063053 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-tv82v" event={"ID":"cba33cb2-37bb-421e-9c92-e8d5894c6b1d","Type":"ContainerStarted","Data":"c8d3de580b49d880228491983ac58c2f3f90e49dbc5c3a2b3d1d65a082643a55"} Jul 06 15:34:08.063221 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:08.063115 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-tv82v" event={"ID":"cba33cb2-37bb-421e-9c92-e8d5894c6b1d","Type":"ContainerStarted","Data":"9d6f3372e081170ff0c7626eaa7fb360d83e73b853ee072b71a8e5754d76a7b4"} Jul 06 15:34:08.064705 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:08.064678 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-8zxfb" event={"ID":"a83f40ad-b56d-4d3f-9275-1a09f0c89e0c","Type":"ContainerStarted","Data":"00614f7c27fea244327a120ae2a887a88635c45927af4a78f2a22c351a997e97"} Jul 06 15:34:08.089398 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:08.089351 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-8zxfb" podStartSLOduration=2.456224709 podStartE2EDuration="24.089336949s" totalImagesPullingTime="21.63311224s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 15:33:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 15:34:08.088957852 +0000 UTC m=+24.598191074" watchObservedRunningTime="2026-07-06 15:34:08.089336949 +0000 UTC m=+24.598570123" Jul 06 15:34:08.869610 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:08.869528 2563 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-06T15:34:07.951891718Z","UUID":"c886a279-9f82-4f05-a11e-564a19ad6c77","Handler":null,"Name":"","Endpoint":""} Jul 06 15:34:08.870891 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:08.870871 2563 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 06 15:34:08.871060 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:08.870907 2563 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 06 15:34:08.939689 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:08.939657 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pdw4n" Jul 06 15:34:08.939870 ip-10-0-129-154 kubenswrapper[2563]: E0706 15:34:08.939776 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pdw4n" podUID="e6761760-82ef-43c3-99f0-9e027d7c859c" Jul 06 15:34:09.070554 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:09.070514 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-tv82v" event={"ID":"cba33cb2-37bb-421e-9c92-e8d5894c6b1d","Type":"ContainerStarted","Data":"950fdc1387e42158e09602d42fdb3f55fc5a02bd772da0900b1e045a82dc5782"} Jul 06 15:34:09.071549 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:09.070616 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-tv82v" event={"ID":"cba33cb2-37bb-421e-9c92-e8d5894c6b1d","Type":"ContainerStarted","Data":"2b4ab901043e9e89762bc08c614490a63a44a2c911d80a57428fe191763650ef"} Jul 06 15:34:09.071549 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:09.071009 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-tv82v" event={"ID":"cba33cb2-37bb-421e-9c92-e8d5894c6b1d","Type":"ContainerStarted","Data":"f2795931e55638c4819c4b8f81e5cadeb06ba71faa131d901840b64b73f983c7"} Jul 06 15:34:09.071549 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:09.071092 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-tv82v" event={"ID":"cba33cb2-37bb-421e-9c92-e8d5894c6b1d","Type":"ContainerStarted","Data":"fd6794027510846dfd87d8570d0085b9e1760415f2806b357b7baca3d3df4c9c"} Jul 06 15:34:09.939752 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:09.939723 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-v8fxz" Jul 06 15:34:09.939752 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:09.939753 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w9qbt" Jul 06 15:34:09.940000 ip-10-0-129-154 kubenswrapper[2563]: E0706 15:34:09.939857 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-v8fxz" podUID="5771d1de-202e-4920-b0de-5d12a4e4e2fc" Jul 06 15:34:09.940000 ip-10-0-129-154 kubenswrapper[2563]: E0706 15:34:09.939944 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-w9qbt" podUID="bce194c9-b363-4021-bb8a-c7cc65facc39" Jul 06 15:34:10.939692 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:10.939670 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pdw4n" Jul 06 15:34:10.940034 ip-10-0-129-154 kubenswrapper[2563]: E0706 15:34:10.939755 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pdw4n" podUID="e6761760-82ef-43c3-99f0-9e027d7c859c" Jul 06 15:34:11.939578 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:11.939384 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w9qbt" Jul 06 15:34:11.939750 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:11.939469 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-v8fxz" Jul 06 15:34:11.939750 ip-10-0-129-154 kubenswrapper[2563]: E0706 15:34:11.939650 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-w9qbt" podUID="bce194c9-b363-4021-bb8a-c7cc65facc39" Jul 06 15:34:11.940091 ip-10-0-129-154 kubenswrapper[2563]: E0706 15:34:11.939761 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-v8fxz" podUID="5771d1de-202e-4920-b0de-5d12a4e4e2fc" Jul 06 15:34:12.082614 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:12.082523 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7s97z" event={"ID":"1aead9b0-c187-46cb-b965-57f2cb23d421","Type":"ContainerStarted","Data":"f7775f5fa976f21eae8707e3898eacb0bf2eb8b7c75162607b642a45d5de544f"} Jul 06 15:34:12.085780 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:12.085752 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-tv82v" event={"ID":"cba33cb2-37bb-421e-9c92-e8d5894c6b1d","Type":"ContainerStarted","Data":"ac26d9e5a5c7ffd159119f06fedbcadb012b2f69007000f9521b6c0e4186a4a5"} Jul 06 15:34:12.087507 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:12.087482 2563 generic.go:356] "Generic (PLEG): container finished" podID="e32dedb7-fb62-41a5-9126-63a337f62a53" containerID="da48c3f55f008f33758e2163cc7a1a31d10de08616a27af70fa4b437e1becc52" exitCode=0 Jul 06 15:34:12.087598 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:12.087522 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-nmfg6" event={"ID":"e32dedb7-fb62-41a5-9126-63a337f62a53","Type":"ContainerDied","Data":"da48c3f55f008f33758e2163cc7a1a31d10de08616a27af70fa4b437e1becc52"} Jul 06 15:34:12.130741 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:12.130693 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7s97z" podStartSLOduration=3.793910258 podStartE2EDuration="28.13067811s" totalImagesPullingTime="24.336767852s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 15:33:44 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 15:34:12.105728242 +0000 UTC m=+28.614961416" watchObservedRunningTime="2026-07-06 15:34:12.13067811 +0000 UTC m=+28.639911326" Jul 06 15:34:12.939171 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:12.939137 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pdw4n" Jul 06 15:34:12.939482 ip-10-0-129-154 kubenswrapper[2563]: E0706 15:34:12.939243 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pdw4n" podUID="e6761760-82ef-43c3-99f0-9e027d7c859c" Jul 06 15:34:13.943437 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:13.943230 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-v8fxz" Jul 06 15:34:13.943866 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:13.943282 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w9qbt" Jul 06 15:34:13.943866 ip-10-0-129-154 kubenswrapper[2563]: E0706 15:34:13.943540 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-v8fxz" podUID="5771d1de-202e-4920-b0de-5d12a4e4e2fc" Jul 06 15:34:13.943866 ip-10-0-129-154 kubenswrapper[2563]: E0706 15:34:13.943584 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-w9qbt" podUID="bce194c9-b363-4021-bb8a-c7cc65facc39" Jul 06 15:34:14.095082 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:14.095042 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-tv82v" event={"ID":"cba33cb2-37bb-421e-9c92-e8d5894c6b1d","Type":"ContainerStarted","Data":"515b05bc5a66e4175a9ece9837461ef21360dce7ed31da38d215902756335c2c"} Jul 06 15:34:14.095355 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:14.095327 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-tv82v" Jul 06 15:34:14.095509 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:14.095409 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-tv82v" Jul 06 15:34:14.097217 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:14.097193 2563 generic.go:356] "Generic (PLEG): container finished" podID="e32dedb7-fb62-41a5-9126-63a337f62a53" containerID="4d0a4effd874455c1b918380226c29a299b16583c7c213d4a6e380afb4bb0404" exitCode=0 Jul 06 15:34:14.097326 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:14.097229 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-nmfg6" event={"ID":"e32dedb7-fb62-41a5-9126-63a337f62a53","Type":"ContainerDied","Data":"4d0a4effd874455c1b918380226c29a299b16583c7c213d4a6e380afb4bb0404"} Jul 06 15:34:14.111989 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:14.111965 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-tv82v" Jul 06 15:34:14.130678 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:14.130636 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-tv82v" podStartSLOduration=9.442563319 podStartE2EDuration="31.130623737s" totalImagesPullingTime="21.688060418s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 15:33:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 15:34:14.13011121 +0000 UTC m=+30.639344384" watchObservedRunningTime="2026-07-06 15:34:14.130623737 +0000 UTC m=+30.639856945" Jul 06 15:34:14.939236 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:14.939191 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pdw4n" Jul 06 15:34:14.939411 ip-10-0-129-154 kubenswrapper[2563]: E0706 15:34:14.939298 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pdw4n" podUID="e6761760-82ef-43c3-99f0-9e027d7c859c" Jul 06 15:34:15.100161 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:15.100134 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-tv82v" Jul 06 15:34:15.115172 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:15.115150 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-tv82v" Jul 06 15:34:15.504537 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:15.504509 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-pdw4n"] Jul 06 15:34:15.504676 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:15.504629 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pdw4n" Jul 06 15:34:15.504740 ip-10-0-129-154 kubenswrapper[2563]: E0706 15:34:15.504716 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pdw4n" podUID="e6761760-82ef-43c3-99f0-9e027d7c859c" Jul 06 15:34:15.507380 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:15.507359 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-w9qbt"] Jul 06 15:34:15.507491 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:15.507486 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w9qbt" Jul 06 15:34:15.507571 ip-10-0-129-154 kubenswrapper[2563]: E0706 15:34:15.507554 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-w9qbt" podUID="bce194c9-b363-4021-bb8a-c7cc65facc39" Jul 06 15:34:15.510735 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:15.510709 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-v8fxz"] Jul 06 15:34:15.510834 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:15.510821 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-v8fxz" Jul 06 15:34:15.510950 ip-10-0-129-154 kubenswrapper[2563]: E0706 15:34:15.510928 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-v8fxz" podUID="5771d1de-202e-4920-b0de-5d12a4e4e2fc" Jul 06 15:34:16.103793 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:16.103708 2563 generic.go:356] "Generic (PLEG): container finished" podID="e32dedb7-fb62-41a5-9126-63a337f62a53" containerID="3ebc630f207f8170813ce43aeceb4e18211f88d86e76922960ee6bd7b5ccfd27" exitCode=0 Jul 06 15:34:16.104472 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:16.103789 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-nmfg6" event={"ID":"e32dedb7-fb62-41a5-9126-63a337f62a53","Type":"ContainerDied","Data":"3ebc630f207f8170813ce43aeceb4e18211f88d86e76922960ee6bd7b5ccfd27"} Jul 06 15:34:16.939372 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:16.939330 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w9qbt" Jul 06 15:34:16.939372 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:16.939365 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pdw4n" Jul 06 15:34:16.939566 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:16.939467 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-v8fxz" Jul 06 15:34:16.939566 ip-10-0-129-154 kubenswrapper[2563]: E0706 15:34:16.939467 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pdw4n" podUID="e6761760-82ef-43c3-99f0-9e027d7c859c" Jul 06 15:34:16.939566 ip-10-0-129-154 kubenswrapper[2563]: E0706 15:34:16.939518 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-w9qbt" podUID="bce194c9-b363-4021-bb8a-c7cc65facc39" Jul 06 15:34:16.939668 ip-10-0-129-154 kubenswrapper[2563]: E0706 15:34:16.939577 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-v8fxz" podUID="5771d1de-202e-4920-b0de-5d12a4e4e2fc" Jul 06 15:34:17.666614 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:17.666584 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/bce194c9-b363-4021-bb8a-c7cc65facc39-original-pull-secret\") pod \"global-pull-secret-syncer-w9qbt\" (UID: \"bce194c9-b363-4021-bb8a-c7cc65facc39\") " pod="kube-system/global-pull-secret-syncer-w9qbt" Jul 06 15:34:17.667218 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:17.666622 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5771d1de-202e-4920-b0de-5d12a4e4e2fc-metrics-certs\") pod \"network-metrics-daemon-v8fxz\" (UID: \"5771d1de-202e-4920-b0de-5d12a4e4e2fc\") " pod="openshift-multus/network-metrics-daemon-v8fxz" Jul 06 15:34:17.667218 ip-10-0-129-154 kubenswrapper[2563]: E0706 15:34:17.666719 2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 15:34:17.667218 ip-10-0-129-154 kubenswrapper[2563]: E0706 15:34:17.666719 2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 06 15:34:17.667218 ip-10-0-129-154 kubenswrapper[2563]: E0706 15:34:17.666784 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bce194c9-b363-4021-bb8a-c7cc65facc39-original-pull-secret podName:bce194c9-b363-4021-bb8a-c7cc65facc39 nodeName:}" failed. No retries permitted until 2026-07-06 15:34:49.666765742 +0000 UTC m=+66.175998897 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/bce194c9-b363-4021-bb8a-c7cc65facc39-original-pull-secret") pod "global-pull-secret-syncer-w9qbt" (UID: "bce194c9-b363-4021-bb8a-c7cc65facc39") : object "kube-system"/"original-pull-secret" not registered Jul 06 15:34:17.667218 ip-10-0-129-154 kubenswrapper[2563]: E0706 15:34:17.666798 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5771d1de-202e-4920-b0de-5d12a4e4e2fc-metrics-certs podName:5771d1de-202e-4920-b0de-5d12a4e4e2fc nodeName:}" failed. No retries permitted until 2026-07-06 15:34:49.666791659 +0000 UTC m=+66.176024814 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5771d1de-202e-4920-b0de-5d12a4e4e2fc-metrics-certs") pod "network-metrics-daemon-v8fxz" (UID: "5771d1de-202e-4920-b0de-5d12a4e4e2fc") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 15:34:17.767052 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:17.767015 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-twj8p\" (UniqueName: \"kubernetes.io/projected/e6761760-82ef-43c3-99f0-9e027d7c859c-kube-api-access-twj8p\") pod \"network-check-target-pdw4n\" (UID: \"e6761760-82ef-43c3-99f0-9e027d7c859c\") " pod="openshift-network-diagnostics/network-check-target-pdw4n" Jul 06 15:34:17.767226 ip-10-0-129-154 kubenswrapper[2563]: E0706 15:34:17.767193 2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 06 15:34:17.767226 ip-10-0-129-154 kubenswrapper[2563]: E0706 15:34:17.767226 2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 06 15:34:17.767335 ip-10-0-129-154 kubenswrapper[2563]: E0706 15:34:17.767239 2563 projected.go:196] Error preparing data for projected volume kube-api-access-twj8p for pod openshift-network-diagnostics/network-check-target-pdw4n: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 15:34:17.767335 ip-10-0-129-154 kubenswrapper[2563]: E0706 15:34:17.767304 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e6761760-82ef-43c3-99f0-9e027d7c859c-kube-api-access-twj8p podName:e6761760-82ef-43c3-99f0-9e027d7c859c nodeName:}" failed. No retries permitted until 2026-07-06 15:34:49.767285016 +0000 UTC m=+66.276518182 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-twj8p" (UniqueName: "kubernetes.io/projected/e6761760-82ef-43c3-99f0-9e027d7c859c-kube-api-access-twj8p") pod "network-check-target-pdw4n" (UID: "e6761760-82ef-43c3-99f0-9e027d7c859c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 15:34:18.939945 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:18.939905 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w9qbt" Jul 06 15:34:18.939945 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:18.939905 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-v8fxz" Jul 06 15:34:18.940411 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:18.939923 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pdw4n" Jul 06 15:34:18.940411 ip-10-0-129-154 kubenswrapper[2563]: E0706 15:34:18.940030 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-w9qbt" podUID="bce194c9-b363-4021-bb8a-c7cc65facc39" Jul 06 15:34:18.940411 ip-10-0-129-154 kubenswrapper[2563]: E0706 15:34:18.940133 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-v8fxz" podUID="5771d1de-202e-4920-b0de-5d12a4e4e2fc" Jul 06 15:34:18.940411 ip-10-0-129-154 kubenswrapper[2563]: E0706 15:34:18.940229 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pdw4n" podUID="e6761760-82ef-43c3-99f0-9e027d7c859c" Jul 06 15:34:20.939846 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:20.939817 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-v8fxz" Jul 06 15:34:20.940249 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:20.939817 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pdw4n" Jul 06 15:34:20.940249 ip-10-0-129-154 kubenswrapper[2563]: E0706 15:34:20.939931 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-v8fxz" podUID="5771d1de-202e-4920-b0de-5d12a4e4e2fc" Jul 06 15:34:20.940249 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:20.939831 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w9qbt" Jul 06 15:34:20.940249 ip-10-0-129-154 kubenswrapper[2563]: E0706 15:34:20.940027 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pdw4n" podUID="e6761760-82ef-43c3-99f0-9e027d7c859c" Jul 06 15:34:20.940249 ip-10-0-129-154 kubenswrapper[2563]: E0706 15:34:20.940067 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-w9qbt" podUID="bce194c9-b363-4021-bb8a-c7cc65facc39" Jul 06 15:34:21.793125 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:21.792883 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-154.ec2.internal" event="NodeReady" Jul 06 15:34:21.793310 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:21.793258 2563 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 06 15:34:21.840066 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:21.840032 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-d668d48f7-cncvq"] Jul 06 15:34:21.845998 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:21.845975 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-5855858db9-884bq"] Jul 06 15:34:21.846140 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:21.846120 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-d668d48f7-cncvq" Jul 06 15:34:21.849670 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:21.849645 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-28kt7"] Jul 06 15:34:21.849809 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:21.849715 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 06 15:34:21.849809 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:21.849758 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-5855858db9-884bq" Jul 06 15:34:21.850358 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:21.850329 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-98ctb\"" Jul 06 15:34:21.850632 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:21.850615 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 06 15:34:21.850805 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:21.850790 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 06 15:34:21.855880 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:21.854814 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 06 15:34:21.855880 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:21.855103 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-x692s\"" Jul 06 15:34:21.855880 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:21.855161 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\"" Jul 06 15:34:21.855880 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:21.855467 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 06 15:34:21.855880 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:21.855488 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\"" Jul 06 15:34:21.855880 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:21.855556 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\"" Jul 06 15:34:21.855880 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:21.855836 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\"" Jul 06 15:34:21.856243 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:21.856035 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-64786d9cbb-trjfs"] Jul 06 15:34:21.856243 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:21.856158 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-28kt7" Jul 06 15:34:21.860953 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:21.860018 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-trjfs" Jul 06 15:34:21.860953 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:21.860259 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-4grrz"] Jul 06 15:34:21.860953 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:21.860715 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\"" Jul 06 15:34:21.860953 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:21.860742 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\"" Jul 06 15:34:21.860953 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:21.860806 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\"" Jul 06 15:34:21.861234 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:21.861140 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\"" Jul 06 15:34:21.861234 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:21.861178 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-7gpkd\"" Jul 06 15:34:21.861919 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:21.861904 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 06 15:34:21.863627 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:21.863609 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-5c847dcfc4-nvj9g"] Jul 06 15:34:21.863757 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:21.863735 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-4grrz" Jul 06 15:34:21.864644 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:21.864626 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\"" Jul 06 15:34:21.864850 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:21.864835 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-7jvt2\"" Jul 06 15:34:21.865007 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:21.864991 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 06 15:34:21.865139 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:21.865057 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 06 15:34:21.865139 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:21.865086 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\"" Jul 06 15:34:21.866250 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:21.866234 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\"" Jul 06 15:34:21.866858 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:21.866840 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-7prfp"] Jul 06 15:34:21.867060 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:21.866898 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5c847dcfc4-nvj9g" Jul 06 15:34:21.869633 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:21.869559 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\"" Jul 06 15:34:21.869633 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:21.869593 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-6td9x\"" Jul 06 15:34:21.870411 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:21.870393 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\"" Jul 06 15:34:21.872131 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:21.870566 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-662qp"] Jul 06 15:34:21.872131 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:21.870400 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\"" Jul 06 15:34:21.872131 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:21.871696 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 06 15:34:21.872131 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:21.871904 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 06 15:34:21.872325 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:21.872142 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-2fds7\"" Jul 06 15:34:21.872787 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:21.872770 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-7prfp" Jul 06 15:34:21.874377 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:21.874355 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-68b757fb67-l47bf"] Jul 06 15:34:21.874756 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:21.874737 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-662qp" Jul 06 15:34:21.879135 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:21.879113 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-d668d48f7-cncvq"] Jul 06 15:34:21.879238 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:21.879151 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-64786d9cbb-trjfs"] Jul 06 15:34:21.879238 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:21.879174 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-28kt7"] Jul 06 15:34:21.879362 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:21.879265 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-68b757fb67-l47bf" Jul 06 15:34:21.880106 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:21.880086 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-5855858db9-884bq"] Jul 06 15:34:21.880187 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:21.880148 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-lvkj8\"" Jul 06 15:34:21.880265 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:21.880246 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-glbgv\"" Jul 06 15:34:21.880404 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:21.880367 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\"" Jul 06 15:34:21.880489 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:21.880438 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\"" Jul 06 15:34:21.880529 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:21.880486 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\"" Jul 06 15:34:21.880629 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:21.880615 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\"" Jul 06 15:34:21.880676 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:21.880631 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\"" Jul 06 15:34:21.880676 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:21.880659 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\"" Jul 06 15:34:21.880848 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:21.880829 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\"" Jul 06 15:34:21.881042 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:21.881024 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\"" Jul 06 15:34:21.881930 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:21.881908 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\"" Jul 06 15:34:21.885466 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:21.885446 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\"" Jul 06 15:34:21.885701 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:21.885686 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\"" Jul 06 15:34:21.885788 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:21.885726 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\"" Jul 06 15:34:21.885788 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:21.885764 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-gz4md\"" Jul 06 15:34:21.886807 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:21.886764 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-68b757fb67-l47bf"] Jul 06 15:34:21.887630 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:21.887599 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-7prfp"] Jul 06 15:34:21.893403 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:21.889157 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-4grrz"] Jul 06 15:34:21.893403 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:21.891857 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-6bffdf9778-fgktw"] Jul 06 15:34:21.896244 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:21.895941 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\"" Jul 06 15:34:21.896244 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:21.896003 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6bffdf9778-fgktw" Jul 06 15:34:21.897370 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:21.897352 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-989f8b8dc-fkcxv"] Jul 06 15:34:21.899668 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:21.899640 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-z6cjr\"" Jul 06 15:34:21.900217 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:21.900199 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\"" Jul 06 15:34:21.900370 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:21.900349 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\"" Jul 06 15:34:21.901268 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:21.901130 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5c847dcfc4-nvj9g"] Jul 06 15:34:21.901353 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:21.901320 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-fkcxv" Jul 06 15:34:21.902508 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:21.902489 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-qltg7"] Jul 06 15:34:21.903877 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:21.903857 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 06 15:34:21.904659 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:21.904639 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-t8txl\"" Jul 06 15:34:21.904826 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:21.904803 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 06 15:34:21.906092 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:21.906075 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-h86nn"] Jul 06 15:34:21.906252 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:21.906231 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-qltg7" Jul 06 15:34:21.921609 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:21.913216 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 06 15:34:21.921609 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:21.914845 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 06 15:34:21.921609 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:21.915019 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-pk7tw\"" Jul 06 15:34:21.921609 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:21.915046 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 06 15:34:21.928866 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:21.928774 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-662qp"] Jul 06 15:34:21.928866 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:21.928816 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-989f8b8dc-fkcxv"] Jul 06 15:34:21.928866 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:21.928835 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6bffdf9778-fgktw"] Jul 06 15:34:21.928866 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:21.928854 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-h86nn"] Jul 06 15:34:21.929076 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:21.928876 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-qltg7"] Jul 06 15:34:21.929076 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:21.928976 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-h86nn" Jul 06 15:34:21.936661 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:21.936624 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 06 15:34:21.936796 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:21.936679 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 06 15:34:21.936796 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:21.936624 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-8w9vb\"" Jul 06 15:34:22.000629 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:22.000591 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/201d3840-f397-4577-b8ba-03b31295ed3f-metrics-certs\") pod \"router-default-5855858db9-884bq\" (UID: \"201d3840-f397-4577-b8ba-03b31295ed3f\") " pod="openshift-ingress/router-default-5855858db9-884bq" Jul 06 15:34:22.000942 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:22.000655 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/9ad7e968-ed2e-4cbe-9b4b-780bc9d01b88-serving-cert\") pod \"console-operator-68b757fb67-l47bf\" (UID: \"9ad7e968-ed2e-4cbe-9b4b-780bc9d01b88\") " pod="openshift-console-operator/console-operator-68b757fb67-l47bf" Jul 06 15:34:22.000942 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:22.000680 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4vgt4\" (UniqueName: \"kubernetes.io/projected/5967a722-7558-4ab7-98f4-b3a5e367bd7a-kube-api-access-4vgt4\") pod \"ingress-canary-qltg7\" (UID: \"5967a722-7558-4ab7-98f4-b3a5e367bd7a\") " pod="openshift-ingress-canary/ingress-canary-qltg7" Jul 06 15:34:22.000942 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:22.000704 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/929783be-42bc-444c-9c66-a3a17fd9ff04-config\") pod \"service-ca-operator-5d46d9ffd4-28kt7\" (UID: \"929783be-42bc-444c-9c66-a3a17fd9ff04\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-28kt7" Jul 06 15:34:22.000942 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:22.000737 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/9ad7e968-ed2e-4cbe-9b4b-780bc9d01b88-trusted-ca\") pod \"console-operator-68b757fb67-l47bf\" (UID: \"9ad7e968-ed2e-4cbe-9b4b-780bc9d01b88\") " pod="openshift-console-operator/console-operator-68b757fb67-l47bf" Jul 06 15:34:22.000942 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:22.000759 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/0e2e4871-594a-42f5-abb7-867d61453254-snapshots\") pod \"insights-operator-5c847dcfc4-nvj9g\" (UID: \"0e2e4871-594a-42f5-abb7-867d61453254\") " pod="openshift-insights/insights-operator-5c847dcfc4-nvj9g" Jul 06 15:34:22.000942 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:22.000784 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g4rzf\" (UniqueName: \"kubernetes.io/projected/0e2e4871-594a-42f5-abb7-867d61453254-kube-api-access-g4rzf\") pod \"insights-operator-5c847dcfc4-nvj9g\" (UID: \"0e2e4871-594a-42f5-abb7-867d61453254\") " pod="openshift-insights/insights-operator-5c847dcfc4-nvj9g" Jul 06 15:34:22.000942 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:22.000807 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/929783be-42bc-444c-9c66-a3a17fd9ff04-serving-cert\") pod \"service-ca-operator-5d46d9ffd4-28kt7\" (UID: \"929783be-42bc-444c-9c66-a3a17fd9ff04\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-28kt7" Jul 06 15:34:22.000942 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:22.000821 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/1911fa1c-48ce-44ab-8b64-7ae70ce80a0d-ca-trust-extracted\") pod \"image-registry-d668d48f7-cncvq\" (UID: \"1911fa1c-48ce-44ab-8b64-7ae70ce80a0d\") " pod="openshift-image-registry/image-registry-d668d48f7-cncvq" Jul 06 15:34:22.000942 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:22.000838 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/bf5c30e2-bf88-468b-adf2-2e8f73f136b4-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-trjfs\" (UID: \"bf5c30e2-bf88-468b-adf2-2e8f73f136b4\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-trjfs" Jul 06 15:34:22.000942 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:22.000863 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/2e3ce465-22c0-4b33-83f4-8fdc24d06414-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-7prfp\" (UID: \"2e3ce465-22c0-4b33-83f4-8fdc24d06414\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-7prfp" Jul 06 15:34:22.000942 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:22.000879 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/201d3840-f397-4577-b8ba-03b31295ed3f-default-certificate\") pod \"router-default-5855858db9-884bq\" (UID: \"201d3840-f397-4577-b8ba-03b31295ed3f\") " pod="openshift-ingress/router-default-5855858db9-884bq" Jul 06 15:34:22.000942 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:22.000894 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/1911fa1c-48ce-44ab-8b64-7ae70ce80a0d-installation-pull-secrets\") pod \"image-registry-d668d48f7-cncvq\" (UID: \"1911fa1c-48ce-44ab-8b64-7ae70ce80a0d\") " pod="openshift-image-registry/image-registry-d668d48f7-cncvq" Jul 06 15:34:22.000942 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:22.000911 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/1911fa1c-48ce-44ab-8b64-7ae70ce80a0d-registry-certificates\") pod \"image-registry-d668d48f7-cncvq\" (UID: \"1911fa1c-48ce-44ab-8b64-7ae70ce80a0d\") " pod="openshift-image-registry/image-registry-d668d48f7-cncvq" Jul 06 15:34:22.000942 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:22.000932 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0e2e4871-594a-42f5-abb7-867d61453254-trusted-ca-bundle\") pod \"insights-operator-5c847dcfc4-nvj9g\" (UID: \"0e2e4871-594a-42f5-abb7-867d61453254\") " pod="openshift-insights/insights-operator-5c847dcfc4-nvj9g" Jul 06 15:34:22.001458 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:22.000955 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/201d3840-f397-4577-b8ba-03b31295ed3f-stats-auth\") pod \"router-default-5855858db9-884bq\" (UID: \"201d3840-f397-4577-b8ba-03b31295ed3f\") " pod="openshift-ingress/router-default-5855858db9-884bq" Jul 06 15:34:22.001458 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:22.000969 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/f955a985-08b6-49d9-b159-8ec3916d8b20-nginx-conf\") pod \"networking-console-plugin-6bffdf9778-fgktw\" (UID: \"f955a985-08b6-49d9-b159-8ec3916d8b20\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-fgktw" Jul 06 15:34:22.001458 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:22.000983 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/bf5c30e2-bf88-468b-adf2-2e8f73f136b4-telemetry-config\") pod \"cluster-monitoring-operator-64786d9cbb-trjfs\" (UID: \"bf5c30e2-bf88-468b-adf2-2e8f73f136b4\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-trjfs" Jul 06 15:34:22.001458 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:22.001004 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jrmhs\" (UniqueName: \"kubernetes.io/projected/929783be-42bc-444c-9c66-a3a17fd9ff04-kube-api-access-jrmhs\") pod \"service-ca-operator-5d46d9ffd4-28kt7\" (UID: \"929783be-42bc-444c-9c66-a3a17fd9ff04\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-28kt7" Jul 06 15:34:22.001458 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:22.001036 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cc9vl\" (UniqueName: \"kubernetes.io/projected/1911fa1c-48ce-44ab-8b64-7ae70ce80a0d-kube-api-access-cc9vl\") pod \"image-registry-d668d48f7-cncvq\" (UID: \"1911fa1c-48ce-44ab-8b64-7ae70ce80a0d\") " pod="openshift-image-registry/image-registry-d668d48f7-cncvq" Jul 06 15:34:22.001458 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:22.001064 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pjxhb\" (UniqueName: \"kubernetes.io/projected/9ad7e968-ed2e-4cbe-9b4b-780bc9d01b88-kube-api-access-pjxhb\") pod \"console-operator-68b757fb67-l47bf\" (UID: \"9ad7e968-ed2e-4cbe-9b4b-780bc9d01b88\") " pod="openshift-console-operator/console-operator-68b757fb67-l47bf" Jul 06 15:34:22.001458 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:22.001110 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/1911fa1c-48ce-44ab-8b64-7ae70ce80a0d-trusted-ca\") pod \"image-registry-d668d48f7-cncvq\" (UID: \"1911fa1c-48ce-44ab-8b64-7ae70ce80a0d\") " pod="openshift-image-registry/image-registry-d668d48f7-cncvq" Jul 06 15:34:22.001458 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:22.001130 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/966f6ac8-fde8-46e3-b9fe-816b6f334efa-serving-cert\") pod \"kube-storage-version-migrator-operator-846bf668f7-662qp\" (UID: \"966f6ac8-fde8-46e3-b9fe-816b6f334efa\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-662qp" Jul 06 15:34:22.001458 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:22.001145 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0e2e4871-594a-42f5-abb7-867d61453254-service-ca-bundle\") pod \"insights-operator-5c847dcfc4-nvj9g\" (UID: \"0e2e4871-594a-42f5-abb7-867d61453254\") " pod="openshift-insights/insights-operator-5c847dcfc4-nvj9g" Jul 06 15:34:22.001458 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:22.001161 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-95wdn\" (UniqueName: \"kubernetes.io/projected/2e3ce465-22c0-4b33-83f4-8fdc24d06414-kube-api-access-95wdn\") pod \"cluster-samples-operator-56fcf86bf-7prfp\" (UID: \"2e3ce465-22c0-4b33-83f4-8fdc24d06414\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-7prfp" Jul 06 15:34:22.001458 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:22.001188 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/0e2e4871-594a-42f5-abb7-867d61453254-serving-cert\") pod \"insights-operator-5c847dcfc4-nvj9g\" (UID: \"0e2e4871-594a-42f5-abb7-867d61453254\") " pod="openshift-insights/insights-operator-5c847dcfc4-nvj9g" Jul 06 15:34:22.001458 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:22.001220 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/9ad7e968-ed2e-4cbe-9b4b-780bc9d01b88-config\") pod \"console-operator-68b757fb67-l47bf\" (UID: \"9ad7e968-ed2e-4cbe-9b4b-780bc9d01b88\") " pod="openshift-console-operator/console-operator-68b757fb67-l47bf" Jul 06 15:34:22.001458 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:22.001245 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/1911fa1c-48ce-44ab-8b64-7ae70ce80a0d-image-registry-private-configuration\") pod \"image-registry-d668d48f7-cncvq\" (UID: \"1911fa1c-48ce-44ab-8b64-7ae70ce80a0d\") " pod="openshift-image-registry/image-registry-d668d48f7-cncvq" Jul 06 15:34:22.001458 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:22.001279 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pj5jn\" (UniqueName: \"kubernetes.io/projected/201d3840-f397-4577-b8ba-03b31295ed3f-kube-api-access-pj5jn\") pod \"router-default-5855858db9-884bq\" (UID: \"201d3840-f397-4577-b8ba-03b31295ed3f\") " pod="openshift-ingress/router-default-5855858db9-884bq" Jul 06 15:34:22.001458 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:22.001306 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/f955a985-08b6-49d9-b159-8ec3916d8b20-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-fgktw\" (UID: \"f955a985-08b6-49d9-b159-8ec3916d8b20\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-fgktw" Jul 06 15:34:22.002002 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:22.001326 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/201d3840-f397-4577-b8ba-03b31295ed3f-service-ca-bundle\") pod \"router-default-5855858db9-884bq\" (UID: \"201d3840-f397-4577-b8ba-03b31295ed3f\") " pod="openshift-ingress/router-default-5855858db9-884bq" Jul 06 15:34:22.002002 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:22.001353 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/966f6ac8-fde8-46e3-b9fe-816b6f334efa-config\") pod \"kube-storage-version-migrator-operator-846bf668f7-662qp\" (UID: \"966f6ac8-fde8-46e3-b9fe-816b6f334efa\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-662qp" Jul 06 15:34:22.002002 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:22.001379 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9qj6w\" (UniqueName: \"kubernetes.io/projected/b7c1005a-9a43-4116-94cb-df36eaf220fb-kube-api-access-9qj6w\") pod \"volume-data-source-validator-5d456fdbcd-4grrz\" (UID: \"b7c1005a-9a43-4116-94cb-df36eaf220fb\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-4grrz" Jul 06 15:34:22.002002 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:22.001409 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lrcsr\" (UniqueName: \"kubernetes.io/projected/d6fa43bb-e8ee-45ea-9726-ba18d3438f1f-kube-api-access-lrcsr\") pod \"network-check-source-989f8b8dc-fkcxv\" (UID: \"d6fa43bb-e8ee-45ea-9726-ba18d3438f1f\") " pod="openshift-network-diagnostics/network-check-source-989f8b8dc-fkcxv" Jul 06 15:34:22.002002 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:22.001449 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0e2e4871-594a-42f5-abb7-867d61453254-tmp\") pod \"insights-operator-5c847dcfc4-nvj9g\" (UID: \"0e2e4871-594a-42f5-abb7-867d61453254\") " pod="openshift-insights/insights-operator-5c847dcfc4-nvj9g" Jul 06 15:34:22.002002 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:22.001478 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pph54\" (UniqueName: \"kubernetes.io/projected/bf5c30e2-bf88-468b-adf2-2e8f73f136b4-kube-api-access-pph54\") pod \"cluster-monitoring-operator-64786d9cbb-trjfs\" (UID: \"bf5c30e2-bf88-468b-adf2-2e8f73f136b4\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-trjfs" Jul 06 15:34:22.002002 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:22.001509 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rg6vz\" (UniqueName: \"kubernetes.io/projected/966f6ac8-fde8-46e3-b9fe-816b6f334efa-kube-api-access-rg6vz\") pod \"kube-storage-version-migrator-operator-846bf668f7-662qp\" (UID: \"966f6ac8-fde8-46e3-b9fe-816b6f334efa\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-662qp" Jul 06 15:34:22.002002 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:22.001534 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1911fa1c-48ce-44ab-8b64-7ae70ce80a0d-registry-tls\") pod \"image-registry-d668d48f7-cncvq\" (UID: \"1911fa1c-48ce-44ab-8b64-7ae70ce80a0d\") " pod="openshift-image-registry/image-registry-d668d48f7-cncvq" Jul 06 15:34:22.002002 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:22.001557 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/1911fa1c-48ce-44ab-8b64-7ae70ce80a0d-bound-sa-token\") pod \"image-registry-d668d48f7-cncvq\" (UID: \"1911fa1c-48ce-44ab-8b64-7ae70ce80a0d\") " pod="openshift-image-registry/image-registry-d668d48f7-cncvq" Jul 06 15:34:22.002002 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:22.001579 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5967a722-7558-4ab7-98f4-b3a5e367bd7a-cert\") pod \"ingress-canary-qltg7\" (UID: \"5967a722-7558-4ab7-98f4-b3a5e367bd7a\") " pod="openshift-ingress-canary/ingress-canary-qltg7" Jul 06 15:34:22.102382 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:22.102289 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/0e2e4871-594a-42f5-abb7-867d61453254-serving-cert\") pod \"insights-operator-5c847dcfc4-nvj9g\" (UID: \"0e2e4871-594a-42f5-abb7-867d61453254\") " pod="openshift-insights/insights-operator-5c847dcfc4-nvj9g" Jul 06 15:34:22.102382 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:22.102342 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/9ad7e968-ed2e-4cbe-9b4b-780bc9d01b88-config\") pod \"console-operator-68b757fb67-l47bf\" (UID: \"9ad7e968-ed2e-4cbe-9b4b-780bc9d01b88\") " pod="openshift-console-operator/console-operator-68b757fb67-l47bf" Jul 06 15:34:22.102382 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:22.102373 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/1911fa1c-48ce-44ab-8b64-7ae70ce80a0d-image-registry-private-configuration\") pod \"image-registry-d668d48f7-cncvq\" (UID: \"1911fa1c-48ce-44ab-8b64-7ae70ce80a0d\") " pod="openshift-image-registry/image-registry-d668d48f7-cncvq" Jul 06 15:34:22.102657 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:22.102435 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pj5jn\" (UniqueName: \"kubernetes.io/projected/201d3840-f397-4577-b8ba-03b31295ed3f-kube-api-access-pj5jn\") pod \"router-default-5855858db9-884bq\" (UID: \"201d3840-f397-4577-b8ba-03b31295ed3f\") " pod="openshift-ingress/router-default-5855858db9-884bq" Jul 06 15:34:22.102657 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:22.102469 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/f955a985-08b6-49d9-b159-8ec3916d8b20-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-fgktw\" (UID: \"f955a985-08b6-49d9-b159-8ec3916d8b20\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-fgktw" Jul 06 15:34:22.102657 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:22.102492 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/201d3840-f397-4577-b8ba-03b31295ed3f-service-ca-bundle\") pod \"router-default-5855858db9-884bq\" (UID: \"201d3840-f397-4577-b8ba-03b31295ed3f\") " pod="openshift-ingress/router-default-5855858db9-884bq" Jul 06 15:34:22.102657 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:22.102511 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/966f6ac8-fde8-46e3-b9fe-816b6f334efa-config\") pod \"kube-storage-version-migrator-operator-846bf668f7-662qp\" (UID: \"966f6ac8-fde8-46e3-b9fe-816b6f334efa\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-662qp" Jul 06 15:34:22.102657 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:22.102537 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9qj6w\" (UniqueName: \"kubernetes.io/projected/b7c1005a-9a43-4116-94cb-df36eaf220fb-kube-api-access-9qj6w\") pod \"volume-data-source-validator-5d456fdbcd-4grrz\" (UID: \"b7c1005a-9a43-4116-94cb-df36eaf220fb\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-4grrz" Jul 06 15:34:22.102657 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:22.102567 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lrcsr\" (UniqueName: \"kubernetes.io/projected/d6fa43bb-e8ee-45ea-9726-ba18d3438f1f-kube-api-access-lrcsr\") pod \"network-check-source-989f8b8dc-fkcxv\" (UID: \"d6fa43bb-e8ee-45ea-9726-ba18d3438f1f\") " pod="openshift-network-diagnostics/network-check-source-989f8b8dc-fkcxv" Jul 06 15:34:22.102657 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:22.102591 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0e2e4871-594a-42f5-abb7-867d61453254-tmp\") pod \"insights-operator-5c847dcfc4-nvj9g\" (UID: \"0e2e4871-594a-42f5-abb7-867d61453254\") " pod="openshift-insights/insights-operator-5c847dcfc4-nvj9g" Jul 06 15:34:22.102657 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:22.102623 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pph54\" (UniqueName: \"kubernetes.io/projected/bf5c30e2-bf88-468b-adf2-2e8f73f136b4-kube-api-access-pph54\") pod \"cluster-monitoring-operator-64786d9cbb-trjfs\" (UID: \"bf5c30e2-bf88-468b-adf2-2e8f73f136b4\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-trjfs" Jul 06 15:34:22.102657 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:22.102643 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rg6vz\" (UniqueName: \"kubernetes.io/projected/966f6ac8-fde8-46e3-b9fe-816b6f334efa-kube-api-access-rg6vz\") pod \"kube-storage-version-migrator-operator-846bf668f7-662qp\" (UID: \"966f6ac8-fde8-46e3-b9fe-816b6f334efa\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-662qp" Jul 06 15:34:22.103038 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:22.102665 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1911fa1c-48ce-44ab-8b64-7ae70ce80a0d-registry-tls\") pod \"image-registry-d668d48f7-cncvq\" (UID: \"1911fa1c-48ce-44ab-8b64-7ae70ce80a0d\") " pod="openshift-image-registry/image-registry-d668d48f7-cncvq" Jul 06 15:34:22.103038 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:22.102680 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/1911fa1c-48ce-44ab-8b64-7ae70ce80a0d-bound-sa-token\") pod \"image-registry-d668d48f7-cncvq\" (UID: \"1911fa1c-48ce-44ab-8b64-7ae70ce80a0d\") " pod="openshift-image-registry/image-registry-d668d48f7-cncvq" Jul 06 15:34:22.103038 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:22.102696 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5967a722-7558-4ab7-98f4-b3a5e367bd7a-cert\") pod \"ingress-canary-qltg7\" (UID: \"5967a722-7558-4ab7-98f4-b3a5e367bd7a\") " pod="openshift-ingress-canary/ingress-canary-qltg7" Jul 06 15:34:22.103038 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:22.102720 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/201d3840-f397-4577-b8ba-03b31295ed3f-metrics-certs\") pod \"router-default-5855858db9-884bq\" (UID: \"201d3840-f397-4577-b8ba-03b31295ed3f\") " pod="openshift-ingress/router-default-5855858db9-884bq" Jul 06 15:34:22.103038 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:22.102757 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/9ad7e968-ed2e-4cbe-9b4b-780bc9d01b88-serving-cert\") pod \"console-operator-68b757fb67-l47bf\" (UID: \"9ad7e968-ed2e-4cbe-9b4b-780bc9d01b88\") " pod="openshift-console-operator/console-operator-68b757fb67-l47bf" Jul 06 15:34:22.103038 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:22.102784 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4vgt4\" (UniqueName: \"kubernetes.io/projected/5967a722-7558-4ab7-98f4-b3a5e367bd7a-kube-api-access-4vgt4\") pod \"ingress-canary-qltg7\" (UID: \"5967a722-7558-4ab7-98f4-b3a5e367bd7a\") " pod="openshift-ingress-canary/ingress-canary-qltg7" Jul 06 15:34:22.103038 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:22.102824 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/929783be-42bc-444c-9c66-a3a17fd9ff04-config\") pod \"service-ca-operator-5d46d9ffd4-28kt7\" (UID: \"929783be-42bc-444c-9c66-a3a17fd9ff04\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-28kt7" Jul 06 15:34:22.103038 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:22.102852 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/9ad7e968-ed2e-4cbe-9b4b-780bc9d01b88-trusted-ca\") pod \"console-operator-68b757fb67-l47bf\" (UID: \"9ad7e968-ed2e-4cbe-9b4b-780bc9d01b88\") " pod="openshift-console-operator/console-operator-68b757fb67-l47bf" Jul 06 15:34:22.103038 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:22.102884 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/0e2e4871-594a-42f5-abb7-867d61453254-snapshots\") pod \"insights-operator-5c847dcfc4-nvj9g\" (UID: \"0e2e4871-594a-42f5-abb7-867d61453254\") " pod="openshift-insights/insights-operator-5c847dcfc4-nvj9g" Jul 06 15:34:22.103038 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:22.102910 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g4rzf\" (UniqueName: \"kubernetes.io/projected/0e2e4871-594a-42f5-abb7-867d61453254-kube-api-access-g4rzf\") pod \"insights-operator-5c847dcfc4-nvj9g\" (UID: \"0e2e4871-594a-42f5-abb7-867d61453254\") " pod="openshift-insights/insights-operator-5c847dcfc4-nvj9g" Jul 06 15:34:22.103038 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:22.102948 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/929783be-42bc-444c-9c66-a3a17fd9ff04-serving-cert\") pod \"service-ca-operator-5d46d9ffd4-28kt7\" (UID: \"929783be-42bc-444c-9c66-a3a17fd9ff04\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-28kt7" Jul 06 15:34:22.103038 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:22.102975 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/1911fa1c-48ce-44ab-8b64-7ae70ce80a0d-ca-trust-extracted\") pod \"image-registry-d668d48f7-cncvq\" (UID: \"1911fa1c-48ce-44ab-8b64-7ae70ce80a0d\") " pod="openshift-image-registry/image-registry-d668d48f7-cncvq" Jul 06 15:34:22.103038 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:22.103001 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/bf5c30e2-bf88-468b-adf2-2e8f73f136b4-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-trjfs\" (UID: \"bf5c30e2-bf88-468b-adf2-2e8f73f136b4\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-trjfs" Jul 06 15:34:22.103038 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:22.103031 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/2e3ce465-22c0-4b33-83f4-8fdc24d06414-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-7prfp\" (UID: \"2e3ce465-22c0-4b33-83f4-8fdc24d06414\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-7prfp" Jul 06 15:34:22.103685 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:22.103057 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/201d3840-f397-4577-b8ba-03b31295ed3f-default-certificate\") pod \"router-default-5855858db9-884bq\" (UID: \"201d3840-f397-4577-b8ba-03b31295ed3f\") " pod="openshift-ingress/router-default-5855858db9-884bq" Jul 06 15:34:22.103685 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:22.103083 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/1911fa1c-48ce-44ab-8b64-7ae70ce80a0d-installation-pull-secrets\") pod \"image-registry-d668d48f7-cncvq\" (UID: \"1911fa1c-48ce-44ab-8b64-7ae70ce80a0d\") " pod="openshift-image-registry/image-registry-d668d48f7-cncvq" Jul 06 15:34:22.103685 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:22.103113 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/1911fa1c-48ce-44ab-8b64-7ae70ce80a0d-registry-certificates\") pod \"image-registry-d668d48f7-cncvq\" (UID: \"1911fa1c-48ce-44ab-8b64-7ae70ce80a0d\") " pod="openshift-image-registry/image-registry-d668d48f7-cncvq" Jul 06 15:34:22.103685 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:22.103122 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/9ad7e968-ed2e-4cbe-9b4b-780bc9d01b88-config\") pod \"console-operator-68b757fb67-l47bf\" (UID: \"9ad7e968-ed2e-4cbe-9b4b-780bc9d01b88\") " pod="openshift-console-operator/console-operator-68b757fb67-l47bf" Jul 06 15:34:22.103685 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:22.103141 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0e2e4871-594a-42f5-abb7-867d61453254-trusted-ca-bundle\") pod \"insights-operator-5c847dcfc4-nvj9g\" (UID: \"0e2e4871-594a-42f5-abb7-867d61453254\") " pod="openshift-insights/insights-operator-5c847dcfc4-nvj9g" Jul 06 15:34:22.103685 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:22.103182 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kxwdk\" (UniqueName: \"kubernetes.io/projected/17c20e5e-6515-48d6-8c51-425a9e86485d-kube-api-access-kxwdk\") pod \"dns-default-h86nn\" (UID: \"17c20e5e-6515-48d6-8c51-425a9e86485d\") " pod="openshift-dns/dns-default-h86nn" Jul 06 15:34:22.103685 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:22.103239 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/201d3840-f397-4577-b8ba-03b31295ed3f-stats-auth\") pod \"router-default-5855858db9-884bq\" (UID: \"201d3840-f397-4577-b8ba-03b31295ed3f\") " pod="openshift-ingress/router-default-5855858db9-884bq" Jul 06 15:34:22.103685 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:22.103264 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/f955a985-08b6-49d9-b159-8ec3916d8b20-nginx-conf\") pod \"networking-console-plugin-6bffdf9778-fgktw\" (UID: \"f955a985-08b6-49d9-b159-8ec3916d8b20\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-fgktw" Jul 06 15:34:22.103685 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:22.103287 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/bf5c30e2-bf88-468b-adf2-2e8f73f136b4-telemetry-config\") pod \"cluster-monitoring-operator-64786d9cbb-trjfs\" (UID: \"bf5c30e2-bf88-468b-adf2-2e8f73f136b4\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-trjfs" Jul 06 15:34:22.103685 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:22.103311 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jrmhs\" (UniqueName: \"kubernetes.io/projected/929783be-42bc-444c-9c66-a3a17fd9ff04-kube-api-access-jrmhs\") pod \"service-ca-operator-5d46d9ffd4-28kt7\" (UID: \"929783be-42bc-444c-9c66-a3a17fd9ff04\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-28kt7" Jul 06 15:34:22.103685 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:22.103346 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cc9vl\" (UniqueName: \"kubernetes.io/projected/1911fa1c-48ce-44ab-8b64-7ae70ce80a0d-kube-api-access-cc9vl\") pod \"image-registry-d668d48f7-cncvq\" (UID: \"1911fa1c-48ce-44ab-8b64-7ae70ce80a0d\") " pod="openshift-image-registry/image-registry-d668d48f7-cncvq" Jul 06 15:34:22.103685 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:22.103370 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/17c20e5e-6515-48d6-8c51-425a9e86485d-config-volume\") pod \"dns-default-h86nn\" (UID: \"17c20e5e-6515-48d6-8c51-425a9e86485d\") " pod="openshift-dns/dns-default-h86nn" Jul 06 15:34:22.103685 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:22.103401 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pjxhb\" (UniqueName: \"kubernetes.io/projected/9ad7e968-ed2e-4cbe-9b4b-780bc9d01b88-kube-api-access-pjxhb\") pod \"console-operator-68b757fb67-l47bf\" (UID: \"9ad7e968-ed2e-4cbe-9b4b-780bc9d01b88\") " pod="openshift-console-operator/console-operator-68b757fb67-l47bf" Jul 06 15:34:22.103685 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:22.103444 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/1911fa1c-48ce-44ab-8b64-7ae70ce80a0d-trusted-ca\") pod \"image-registry-d668d48f7-cncvq\" (UID: \"1911fa1c-48ce-44ab-8b64-7ae70ce80a0d\") " pod="openshift-image-registry/image-registry-d668d48f7-cncvq" Jul 06 15:34:22.103685 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:22.103474 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/966f6ac8-fde8-46e3-b9fe-816b6f334efa-serving-cert\") pod \"kube-storage-version-migrator-operator-846bf668f7-662qp\" (UID: \"966f6ac8-fde8-46e3-b9fe-816b6f334efa\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-662qp" Jul 06 15:34:22.103685 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:22.103502 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0e2e4871-594a-42f5-abb7-867d61453254-service-ca-bundle\") pod \"insights-operator-5c847dcfc4-nvj9g\" (UID: \"0e2e4871-594a-42f5-abb7-867d61453254\") " pod="openshift-insights/insights-operator-5c847dcfc4-nvj9g" Jul 06 15:34:22.104448 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:22.103526 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/17c20e5e-6515-48d6-8c51-425a9e86485d-metrics-tls\") pod \"dns-default-h86nn\" (UID: \"17c20e5e-6515-48d6-8c51-425a9e86485d\") " pod="openshift-dns/dns-default-h86nn" Jul 06 15:34:22.104448 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:22.103551 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-95wdn\" (UniqueName: \"kubernetes.io/projected/2e3ce465-22c0-4b33-83f4-8fdc24d06414-kube-api-access-95wdn\") pod \"cluster-samples-operator-56fcf86bf-7prfp\" (UID: \"2e3ce465-22c0-4b33-83f4-8fdc24d06414\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-7prfp" Jul 06 15:34:22.104448 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:22.103576 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/17c20e5e-6515-48d6-8c51-425a9e86485d-tmp-dir\") pod \"dns-default-h86nn\" (UID: \"17c20e5e-6515-48d6-8c51-425a9e86485d\") " pod="openshift-dns/dns-default-h86nn" Jul 06 15:34:22.104448 ip-10-0-129-154 kubenswrapper[2563]: E0706 15:34:22.103678 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/201d3840-f397-4577-b8ba-03b31295ed3f-service-ca-bundle podName:201d3840-f397-4577-b8ba-03b31295ed3f nodeName:}" failed. No retries permitted until 2026-07-06 15:34:22.603657515 +0000 UTC m=+39.112890691 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/201d3840-f397-4577-b8ba-03b31295ed3f-service-ca-bundle") pod "router-default-5855858db9-884bq" (UID: "201d3840-f397-4577-b8ba-03b31295ed3f") : configmap references non-existent config key: service-ca.crt Jul 06 15:34:22.104448 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:22.103918 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0e2e4871-594a-42f5-abb7-867d61453254-trusted-ca-bundle\") pod \"insights-operator-5c847dcfc4-nvj9g\" (UID: \"0e2e4871-594a-42f5-abb7-867d61453254\") " pod="openshift-insights/insights-operator-5c847dcfc4-nvj9g" Jul 06 15:34:22.104448 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:22.104372 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/966f6ac8-fde8-46e3-b9fe-816b6f334efa-config\") pod \"kube-storage-version-migrator-operator-846bf668f7-662qp\" (UID: \"966f6ac8-fde8-46e3-b9fe-816b6f334efa\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-662qp" Jul 06 15:34:22.107135 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:22.104799 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/f955a985-08b6-49d9-b159-8ec3916d8b20-nginx-conf\") pod \"networking-console-plugin-6bffdf9778-fgktw\" (UID: \"f955a985-08b6-49d9-b159-8ec3916d8b20\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-fgktw" Jul 06 15:34:22.107135 ip-10-0-129-154 kubenswrapper[2563]: E0706 15:34:22.104883 2563 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 06 15:34:22.107135 ip-10-0-129-154 kubenswrapper[2563]: E0706 15:34:22.104960 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/201d3840-f397-4577-b8ba-03b31295ed3f-metrics-certs podName:201d3840-f397-4577-b8ba-03b31295ed3f nodeName:}" failed. No retries permitted until 2026-07-06 15:34:22.604943162 +0000 UTC m=+39.114176318 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/201d3840-f397-4577-b8ba-03b31295ed3f-metrics-certs") pod "router-default-5855858db9-884bq" (UID: "201d3840-f397-4577-b8ba-03b31295ed3f") : secret "router-metrics-certs-default" not found Jul 06 15:34:22.107135 ip-10-0-129-154 kubenswrapper[2563]: E0706 15:34:22.104991 2563 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 06 15:34:22.107135 ip-10-0-129-154 kubenswrapper[2563]: E0706 15:34:22.105027 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2e3ce465-22c0-4b33-83f4-8fdc24d06414-samples-operator-tls podName:2e3ce465-22c0-4b33-83f4-8fdc24d06414 nodeName:}" failed. No retries permitted until 2026-07-06 15:34:22.605016087 +0000 UTC m=+39.114249239 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/2e3ce465-22c0-4b33-83f4-8fdc24d06414-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-7prfp" (UID: "2e3ce465-22c0-4b33-83f4-8fdc24d06414") : secret "samples-operator-tls" not found Jul 06 15:34:22.107135 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:22.105359 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/1911fa1c-48ce-44ab-8b64-7ae70ce80a0d-ca-trust-extracted\") pod \"image-registry-d668d48f7-cncvq\" (UID: \"1911fa1c-48ce-44ab-8b64-7ae70ce80a0d\") " pod="openshift-image-registry/image-registry-d668d48f7-cncvq" Jul 06 15:34:22.107135 ip-10-0-129-154 kubenswrapper[2563]: E0706 15:34:22.105462 2563 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 06 15:34:22.107135 ip-10-0-129-154 kubenswrapper[2563]: E0706 15:34:22.105508 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bf5c30e2-bf88-468b-adf2-2e8f73f136b4-cluster-monitoring-operator-tls podName:bf5c30e2-bf88-468b-adf2-2e8f73f136b4 nodeName:}" failed. No retries permitted until 2026-07-06 15:34:22.60549298 +0000 UTC m=+39.114726133 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/bf5c30e2-bf88-468b-adf2-2e8f73f136b4-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-trjfs" (UID: "bf5c30e2-bf88-468b-adf2-2e8f73f136b4") : secret "cluster-monitoring-operator-tls" not found Jul 06 15:34:22.107135 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:22.105639 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/1911fa1c-48ce-44ab-8b64-7ae70ce80a0d-trusted-ca\") pod \"image-registry-d668d48f7-cncvq\" (UID: \"1911fa1c-48ce-44ab-8b64-7ae70ce80a0d\") " pod="openshift-image-registry/image-registry-d668d48f7-cncvq" Jul 06 15:34:22.107135 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:22.105741 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/bf5c30e2-bf88-468b-adf2-2e8f73f136b4-telemetry-config\") pod \"cluster-monitoring-operator-64786d9cbb-trjfs\" (UID: \"bf5c30e2-bf88-468b-adf2-2e8f73f136b4\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-trjfs" Jul 06 15:34:22.107135 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:22.106239 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0e2e4871-594a-42f5-abb7-867d61453254-service-ca-bundle\") pod \"insights-operator-5c847dcfc4-nvj9g\" (UID: \"0e2e4871-594a-42f5-abb7-867d61453254\") " pod="openshift-insights/insights-operator-5c847dcfc4-nvj9g" Jul 06 15:34:22.107135 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:22.106435 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0e2e4871-594a-42f5-abb7-867d61453254-tmp\") pod \"insights-operator-5c847dcfc4-nvj9g\" (UID: \"0e2e4871-594a-42f5-abb7-867d61453254\") " pod="openshift-insights/insights-operator-5c847dcfc4-nvj9g" Jul 06 15:34:22.107135 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:22.106707 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/1911fa1c-48ce-44ab-8b64-7ae70ce80a0d-registry-certificates\") pod \"image-registry-d668d48f7-cncvq\" (UID: \"1911fa1c-48ce-44ab-8b64-7ae70ce80a0d\") " pod="openshift-image-registry/image-registry-d668d48f7-cncvq" Jul 06 15:34:22.107135 ip-10-0-129-154 kubenswrapper[2563]: E0706 15:34:22.107050 2563 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 06 15:34:22.107135 ip-10-0-129-154 kubenswrapper[2563]: E0706 15:34:22.107068 2563 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-d668d48f7-cncvq: secret "image-registry-tls" not found Jul 06 15:34:22.107135 ip-10-0-129-154 kubenswrapper[2563]: E0706 15:34:22.107109 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1911fa1c-48ce-44ab-8b64-7ae70ce80a0d-registry-tls podName:1911fa1c-48ce-44ab-8b64-7ae70ce80a0d nodeName:}" failed. No retries permitted until 2026-07-06 15:34:22.607096144 +0000 UTC m=+39.116329320 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/1911fa1c-48ce-44ab-8b64-7ae70ce80a0d-registry-tls") pod "image-registry-d668d48f7-cncvq" (UID: "1911fa1c-48ce-44ab-8b64-7ae70ce80a0d") : secret "image-registry-tls" not found Jul 06 15:34:22.108064 ip-10-0-129-154 kubenswrapper[2563]: E0706 15:34:22.107519 2563 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 06 15:34:22.108064 ip-10-0-129-154 kubenswrapper[2563]: E0706 15:34:22.107576 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f955a985-08b6-49d9-b159-8ec3916d8b20-networking-console-plugin-cert podName:f955a985-08b6-49d9-b159-8ec3916d8b20 nodeName:}" failed. No retries permitted until 2026-07-06 15:34:22.607562466 +0000 UTC m=+39.116795622 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/f955a985-08b6-49d9-b159-8ec3916d8b20-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-fgktw" (UID: "f955a985-08b6-49d9-b159-8ec3916d8b20") : secret "networking-console-plugin-cert" not found Jul 06 15:34:22.108064 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:22.107675 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5967a722-7558-4ab7-98f4-b3a5e367bd7a-cert\") pod \"ingress-canary-qltg7\" (UID: \"5967a722-7558-4ab7-98f4-b3a5e367bd7a\") " pod="openshift-ingress-canary/ingress-canary-qltg7" Jul 06 15:34:22.108064 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:22.108004 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/0e2e4871-594a-42f5-abb7-867d61453254-serving-cert\") pod \"insights-operator-5c847dcfc4-nvj9g\" (UID: \"0e2e4871-594a-42f5-abb7-867d61453254\") " pod="openshift-insights/insights-operator-5c847dcfc4-nvj9g" Jul 06 15:34:22.108064 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:22.108035 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/201d3840-f397-4577-b8ba-03b31295ed3f-stats-auth\") pod \"router-default-5855858db9-884bq\" (UID: \"201d3840-f397-4577-b8ba-03b31295ed3f\") " pod="openshift-ingress/router-default-5855858db9-884bq" Jul 06 15:34:22.108064 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:22.108053 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/1911fa1c-48ce-44ab-8b64-7ae70ce80a0d-image-registry-private-configuration\") pod \"image-registry-d668d48f7-cncvq\" (UID: \"1911fa1c-48ce-44ab-8b64-7ae70ce80a0d\") " pod="openshift-image-registry/image-registry-d668d48f7-cncvq" Jul 06 15:34:22.108445 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:22.108410 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/929783be-42bc-444c-9c66-a3a17fd9ff04-config\") pod \"service-ca-operator-5d46d9ffd4-28kt7\" (UID: \"929783be-42bc-444c-9c66-a3a17fd9ff04\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-28kt7" Jul 06 15:34:22.108573 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:22.108519 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/0e2e4871-594a-42f5-abb7-867d61453254-snapshots\") pod \"insights-operator-5c847dcfc4-nvj9g\" (UID: \"0e2e4871-594a-42f5-abb7-867d61453254\") " pod="openshift-insights/insights-operator-5c847dcfc4-nvj9g" Jul 06 15:34:22.108636 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:22.108617 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/966f6ac8-fde8-46e3-b9fe-816b6f334efa-serving-cert\") pod \"kube-storage-version-migrator-operator-846bf668f7-662qp\" (UID: \"966f6ac8-fde8-46e3-b9fe-816b6f334efa\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-662qp" Jul 06 15:34:22.108789 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:22.108770 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/9ad7e968-ed2e-4cbe-9b4b-780bc9d01b88-serving-cert\") pod \"console-operator-68b757fb67-l47bf\" (UID: \"9ad7e968-ed2e-4cbe-9b4b-780bc9d01b88\") " pod="openshift-console-operator/console-operator-68b757fb67-l47bf" Jul 06 15:34:22.108870 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:22.108848 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/9ad7e968-ed2e-4cbe-9b4b-780bc9d01b88-trusted-ca\") pod \"console-operator-68b757fb67-l47bf\" (UID: \"9ad7e968-ed2e-4cbe-9b4b-780bc9d01b88\") " pod="openshift-console-operator/console-operator-68b757fb67-l47bf" Jul 06 15:34:22.110158 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:22.110121 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/1911fa1c-48ce-44ab-8b64-7ae70ce80a0d-installation-pull-secrets\") pod \"image-registry-d668d48f7-cncvq\" (UID: \"1911fa1c-48ce-44ab-8b64-7ae70ce80a0d\") " pod="openshift-image-registry/image-registry-d668d48f7-cncvq" Jul 06 15:34:22.111053 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:22.111033 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/201d3840-f397-4577-b8ba-03b31295ed3f-default-certificate\") pod \"router-default-5855858db9-884bq\" (UID: \"201d3840-f397-4577-b8ba-03b31295ed3f\") " pod="openshift-ingress/router-default-5855858db9-884bq" Jul 06 15:34:22.113360 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:22.113342 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/929783be-42bc-444c-9c66-a3a17fd9ff04-serving-cert\") pod \"service-ca-operator-5d46d9ffd4-28kt7\" (UID: \"929783be-42bc-444c-9c66-a3a17fd9ff04\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-28kt7" Jul 06 15:34:22.123526 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:22.123502 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/1911fa1c-48ce-44ab-8b64-7ae70ce80a0d-bound-sa-token\") pod \"image-registry-d668d48f7-cncvq\" (UID: \"1911fa1c-48ce-44ab-8b64-7ae70ce80a0d\") " pod="openshift-image-registry/image-registry-d668d48f7-cncvq" Jul 06 15:34:22.124299 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:22.124270 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4vgt4\" (UniqueName: \"kubernetes.io/projected/5967a722-7558-4ab7-98f4-b3a5e367bd7a-kube-api-access-4vgt4\") pod \"ingress-canary-qltg7\" (UID: \"5967a722-7558-4ab7-98f4-b3a5e367bd7a\") " pod="openshift-ingress-canary/ingress-canary-qltg7" Jul 06 15:34:22.125327 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:22.125293 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9qj6w\" (UniqueName: \"kubernetes.io/projected/b7c1005a-9a43-4116-94cb-df36eaf220fb-kube-api-access-9qj6w\") pod \"volume-data-source-validator-5d456fdbcd-4grrz\" (UID: \"b7c1005a-9a43-4116-94cb-df36eaf220fb\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-4grrz" Jul 06 15:34:22.126440 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:22.126339 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lrcsr\" (UniqueName: \"kubernetes.io/projected/d6fa43bb-e8ee-45ea-9726-ba18d3438f1f-kube-api-access-lrcsr\") pod \"network-check-source-989f8b8dc-fkcxv\" (UID: \"d6fa43bb-e8ee-45ea-9726-ba18d3438f1f\") " pod="openshift-network-diagnostics/network-check-source-989f8b8dc-fkcxv" Jul 06 15:34:22.127142 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:22.127121 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rg6vz\" (UniqueName: \"kubernetes.io/projected/966f6ac8-fde8-46e3-b9fe-816b6f334efa-kube-api-access-rg6vz\") pod \"kube-storage-version-migrator-operator-846bf668f7-662qp\" (UID: \"966f6ac8-fde8-46e3-b9fe-816b6f334efa\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-662qp" Jul 06 15:34:22.129707 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:22.129665 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g4rzf\" (UniqueName: \"kubernetes.io/projected/0e2e4871-594a-42f5-abb7-867d61453254-kube-api-access-g4rzf\") pod \"insights-operator-5c847dcfc4-nvj9g\" (UID: \"0e2e4871-594a-42f5-abb7-867d61453254\") " pod="openshift-insights/insights-operator-5c847dcfc4-nvj9g" Jul 06 15:34:22.129993 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:22.129968 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pph54\" (UniqueName: \"kubernetes.io/projected/bf5c30e2-bf88-468b-adf2-2e8f73f136b4-kube-api-access-pph54\") pod \"cluster-monitoring-operator-64786d9cbb-trjfs\" (UID: \"bf5c30e2-bf88-468b-adf2-2e8f73f136b4\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-trjfs" Jul 06 15:34:22.130091 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:22.130076 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-95wdn\" (UniqueName: \"kubernetes.io/projected/2e3ce465-22c0-4b33-83f4-8fdc24d06414-kube-api-access-95wdn\") pod \"cluster-samples-operator-56fcf86bf-7prfp\" (UID: \"2e3ce465-22c0-4b33-83f4-8fdc24d06414\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-7prfp" Jul 06 15:34:22.130300 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:22.130280 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jrmhs\" (UniqueName: \"kubernetes.io/projected/929783be-42bc-444c-9c66-a3a17fd9ff04-kube-api-access-jrmhs\") pod \"service-ca-operator-5d46d9ffd4-28kt7\" (UID: \"929783be-42bc-444c-9c66-a3a17fd9ff04\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-28kt7" Jul 06 15:34:22.130721 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:22.130702 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pjxhb\" (UniqueName: \"kubernetes.io/projected/9ad7e968-ed2e-4cbe-9b4b-780bc9d01b88-kube-api-access-pjxhb\") pod \"console-operator-68b757fb67-l47bf\" (UID: \"9ad7e968-ed2e-4cbe-9b4b-780bc9d01b88\") " pod="openshift-console-operator/console-operator-68b757fb67-l47bf" Jul 06 15:34:22.130867 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:22.130835 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cc9vl\" (UniqueName: \"kubernetes.io/projected/1911fa1c-48ce-44ab-8b64-7ae70ce80a0d-kube-api-access-cc9vl\") pod \"image-registry-d668d48f7-cncvq\" (UID: \"1911fa1c-48ce-44ab-8b64-7ae70ce80a0d\") " pod="openshift-image-registry/image-registry-d668d48f7-cncvq" Jul 06 15:34:22.131238 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:22.131214 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pj5jn\" (UniqueName: \"kubernetes.io/projected/201d3840-f397-4577-b8ba-03b31295ed3f-kube-api-access-pj5jn\") pod \"router-default-5855858db9-884bq\" (UID: \"201d3840-f397-4577-b8ba-03b31295ed3f\") " pod="openshift-ingress/router-default-5855858db9-884bq" Jul 06 15:34:22.184391 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:22.184360 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-28kt7" Jul 06 15:34:22.204495 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:22.204032 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kxwdk\" (UniqueName: \"kubernetes.io/projected/17c20e5e-6515-48d6-8c51-425a9e86485d-kube-api-access-kxwdk\") pod \"dns-default-h86nn\" (UID: \"17c20e5e-6515-48d6-8c51-425a9e86485d\") " pod="openshift-dns/dns-default-h86nn" Jul 06 15:34:22.204495 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:22.204085 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/17c20e5e-6515-48d6-8c51-425a9e86485d-config-volume\") pod \"dns-default-h86nn\" (UID: \"17c20e5e-6515-48d6-8c51-425a9e86485d\") " pod="openshift-dns/dns-default-h86nn" Jul 06 15:34:22.204495 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:22.204115 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/17c20e5e-6515-48d6-8c51-425a9e86485d-metrics-tls\") pod \"dns-default-h86nn\" (UID: \"17c20e5e-6515-48d6-8c51-425a9e86485d\") " pod="openshift-dns/dns-default-h86nn" Jul 06 15:34:22.204495 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:22.204142 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/17c20e5e-6515-48d6-8c51-425a9e86485d-tmp-dir\") pod \"dns-default-h86nn\" (UID: \"17c20e5e-6515-48d6-8c51-425a9e86485d\") " pod="openshift-dns/dns-default-h86nn" Jul 06 15:34:22.204495 ip-10-0-129-154 kubenswrapper[2563]: E0706 15:34:22.204288 2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 06 15:34:22.204495 ip-10-0-129-154 kubenswrapper[2563]: E0706 15:34:22.204399 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/17c20e5e-6515-48d6-8c51-425a9e86485d-metrics-tls podName:17c20e5e-6515-48d6-8c51-425a9e86485d nodeName:}" failed. No retries permitted until 2026-07-06 15:34:22.704378097 +0000 UTC m=+39.213611300 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/17c20e5e-6515-48d6-8c51-425a9e86485d-metrics-tls") pod "dns-default-h86nn" (UID: "17c20e5e-6515-48d6-8c51-425a9e86485d") : secret "dns-default-metrics-tls" not found Jul 06 15:34:22.204495 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:22.204461 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/17c20e5e-6515-48d6-8c51-425a9e86485d-tmp-dir\") pod \"dns-default-h86nn\" (UID: \"17c20e5e-6515-48d6-8c51-425a9e86485d\") " pod="openshift-dns/dns-default-h86nn" Jul 06 15:34:22.204929 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:22.204775 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/17c20e5e-6515-48d6-8c51-425a9e86485d-config-volume\") pod \"dns-default-h86nn\" (UID: \"17c20e5e-6515-48d6-8c51-425a9e86485d\") " pod="openshift-dns/dns-default-h86nn" Jul 06 15:34:22.205386 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:22.205370 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-4grrz" Jul 06 15:34:22.213751 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:22.213732 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kxwdk\" (UniqueName: \"kubernetes.io/projected/17c20e5e-6515-48d6-8c51-425a9e86485d-kube-api-access-kxwdk\") pod \"dns-default-h86nn\" (UID: \"17c20e5e-6515-48d6-8c51-425a9e86485d\") " pod="openshift-dns/dns-default-h86nn" Jul 06 15:34:22.221069 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:22.221051 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5c847dcfc4-nvj9g" Jul 06 15:34:22.247465 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:22.247435 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-662qp" Jul 06 15:34:22.254751 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:22.254734 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-68b757fb67-l47bf" Jul 06 15:34:22.270217 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:22.270197 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-fkcxv" Jul 06 15:34:22.278065 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:22.278037 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-qltg7" Jul 06 15:34:22.608091 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:22.608060 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1911fa1c-48ce-44ab-8b64-7ae70ce80a0d-registry-tls\") pod \"image-registry-d668d48f7-cncvq\" (UID: \"1911fa1c-48ce-44ab-8b64-7ae70ce80a0d\") " pod="openshift-image-registry/image-registry-d668d48f7-cncvq" Jul 06 15:34:22.608091 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:22.608096 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/201d3840-f397-4577-b8ba-03b31295ed3f-metrics-certs\") pod \"router-default-5855858db9-884bq\" (UID: \"201d3840-f397-4577-b8ba-03b31295ed3f\") " pod="openshift-ingress/router-default-5855858db9-884bq" Jul 06 15:34:22.608341 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:22.608131 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/bf5c30e2-bf88-468b-adf2-2e8f73f136b4-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-trjfs\" (UID: \"bf5c30e2-bf88-468b-adf2-2e8f73f136b4\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-trjfs" Jul 06 15:34:22.608341 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:22.608150 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/2e3ce465-22c0-4b33-83f4-8fdc24d06414-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-7prfp\" (UID: \"2e3ce465-22c0-4b33-83f4-8fdc24d06414\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-7prfp" Jul 06 15:34:22.608341 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:22.608208 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/f955a985-08b6-49d9-b159-8ec3916d8b20-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-fgktw\" (UID: \"f955a985-08b6-49d9-b159-8ec3916d8b20\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-fgktw" Jul 06 15:34:22.608341 ip-10-0-129-154 kubenswrapper[2563]: E0706 15:34:22.608212 2563 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 06 15:34:22.608341 ip-10-0-129-154 kubenswrapper[2563]: E0706 15:34:22.608236 2563 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 06 15:34:22.608341 ip-10-0-129-154 kubenswrapper[2563]: E0706 15:34:22.608215 2563 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 06 15:34:22.608341 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:22.608225 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/201d3840-f397-4577-b8ba-03b31295ed3f-service-ca-bundle\") pod \"router-default-5855858db9-884bq\" (UID: \"201d3840-f397-4577-b8ba-03b31295ed3f\") " pod="openshift-ingress/router-default-5855858db9-884bq" Jul 06 15:34:22.608341 ip-10-0-129-154 kubenswrapper[2563]: E0706 15:34:22.608276 2563 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 06 15:34:22.608341 ip-10-0-129-154 kubenswrapper[2563]: E0706 15:34:22.608291 2563 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-d668d48f7-cncvq: secret "image-registry-tls" not found Jul 06 15:34:22.608341 ip-10-0-129-154 kubenswrapper[2563]: E0706 15:34:22.608299 2563 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 06 15:34:22.608341 ip-10-0-129-154 kubenswrapper[2563]: E0706 15:34:22.608273 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/201d3840-f397-4577-b8ba-03b31295ed3f-metrics-certs podName:201d3840-f397-4577-b8ba-03b31295ed3f nodeName:}" failed. No retries permitted until 2026-07-06 15:34:23.608258815 +0000 UTC m=+40.117491966 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/201d3840-f397-4577-b8ba-03b31295ed3f-metrics-certs") pod "router-default-5855858db9-884bq" (UID: "201d3840-f397-4577-b8ba-03b31295ed3f") : secret "router-metrics-certs-default" not found Jul 06 15:34:22.608728 ip-10-0-129-154 kubenswrapper[2563]: E0706 15:34:22.608355 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bf5c30e2-bf88-468b-adf2-2e8f73f136b4-cluster-monitoring-operator-tls podName:bf5c30e2-bf88-468b-adf2-2e8f73f136b4 nodeName:}" failed. No retries permitted until 2026-07-06 15:34:23.608343338 +0000 UTC m=+40.117576490 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/bf5c30e2-bf88-468b-adf2-2e8f73f136b4-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-trjfs" (UID: "bf5c30e2-bf88-468b-adf2-2e8f73f136b4") : secret "cluster-monitoring-operator-tls" not found Jul 06 15:34:22.608728 ip-10-0-129-154 kubenswrapper[2563]: E0706 15:34:22.608371 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/201d3840-f397-4577-b8ba-03b31295ed3f-service-ca-bundle podName:201d3840-f397-4577-b8ba-03b31295ed3f nodeName:}" failed. No retries permitted until 2026-07-06 15:34:23.608362616 +0000 UTC m=+40.117595768 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/201d3840-f397-4577-b8ba-03b31295ed3f-service-ca-bundle") pod "router-default-5855858db9-884bq" (UID: "201d3840-f397-4577-b8ba-03b31295ed3f") : configmap references non-existent config key: service-ca.crt Jul 06 15:34:22.608728 ip-10-0-129-154 kubenswrapper[2563]: E0706 15:34:22.608403 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2e3ce465-22c0-4b33-83f4-8fdc24d06414-samples-operator-tls podName:2e3ce465-22c0-4b33-83f4-8fdc24d06414 nodeName:}" failed. No retries permitted until 2026-07-06 15:34:23.608380376 +0000 UTC m=+40.117613530 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/2e3ce465-22c0-4b33-83f4-8fdc24d06414-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-7prfp" (UID: "2e3ce465-22c0-4b33-83f4-8fdc24d06414") : secret "samples-operator-tls" not found Jul 06 15:34:22.608728 ip-10-0-129-154 kubenswrapper[2563]: E0706 15:34:22.608415 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1911fa1c-48ce-44ab-8b64-7ae70ce80a0d-registry-tls podName:1911fa1c-48ce-44ab-8b64-7ae70ce80a0d nodeName:}" failed. No retries permitted until 2026-07-06 15:34:23.608409719 +0000 UTC m=+40.117642870 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/1911fa1c-48ce-44ab-8b64-7ae70ce80a0d-registry-tls") pod "image-registry-d668d48f7-cncvq" (UID: "1911fa1c-48ce-44ab-8b64-7ae70ce80a0d") : secret "image-registry-tls" not found Jul 06 15:34:22.608728 ip-10-0-129-154 kubenswrapper[2563]: E0706 15:34:22.608470 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f955a985-08b6-49d9-b159-8ec3916d8b20-networking-console-plugin-cert podName:f955a985-08b6-49d9-b159-8ec3916d8b20 nodeName:}" failed. No retries permitted until 2026-07-06 15:34:23.60845936 +0000 UTC m=+40.117692514 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/f955a985-08b6-49d9-b159-8ec3916d8b20-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-fgktw" (UID: "f955a985-08b6-49d9-b159-8ec3916d8b20") : secret "networking-console-plugin-cert" not found Jul 06 15:34:22.709778 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:22.709737 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/17c20e5e-6515-48d6-8c51-425a9e86485d-metrics-tls\") pod \"dns-default-h86nn\" (UID: \"17c20e5e-6515-48d6-8c51-425a9e86485d\") " pod="openshift-dns/dns-default-h86nn" Jul 06 15:34:22.709960 ip-10-0-129-154 kubenswrapper[2563]: E0706 15:34:22.709901 2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 06 15:34:22.710047 ip-10-0-129-154 kubenswrapper[2563]: E0706 15:34:22.709978 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/17c20e5e-6515-48d6-8c51-425a9e86485d-metrics-tls podName:17c20e5e-6515-48d6-8c51-425a9e86485d nodeName:}" failed. No retries permitted until 2026-07-06 15:34:23.709957451 +0000 UTC m=+40.219190625 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/17c20e5e-6515-48d6-8c51-425a9e86485d-metrics-tls") pod "dns-default-h86nn" (UID: "17c20e5e-6515-48d6-8c51-425a9e86485d") : secret "dns-default-metrics-tls" not found Jul 06 15:34:22.901430 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:22.901382 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-662qp"] Jul 06 15:34:22.904244 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:22.904221 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-4grrz"] Jul 06 15:34:22.911492 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:22.911468 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-989f8b8dc-fkcxv"] Jul 06 15:34:22.912281 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:22.912262 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-qltg7"] Jul 06 15:34:22.917300 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:34:22.917277 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod966f6ac8_fde8_46e3_b9fe_816b6f334efa.slice/crio-1ffb4801463c94fdac9e4d4392e4fe5b8d887c403bff14f12b43aba1465ad55c WatchSource:0}: Error finding container 1ffb4801463c94fdac9e4d4392e4fe5b8d887c403bff14f12b43aba1465ad55c: Status 404 returned error can't find the container with id 1ffb4801463c94fdac9e4d4392e4fe5b8d887c403bff14f12b43aba1465ad55c Jul 06 15:34:22.918272 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:34:22.918228 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb7c1005a_9a43_4116_94cb_df36eaf220fb.slice/crio-5d1acc8ce86c97d164e24b5da0893375b7ad669cd90a24badf9b062ae8d0ff83 WatchSource:0}: Error finding container 5d1acc8ce86c97d164e24b5da0893375b7ad669cd90a24badf9b062ae8d0ff83: Status 404 returned error can't find the container with id 5d1acc8ce86c97d164e24b5da0893375b7ad669cd90a24badf9b062ae8d0ff83 Jul 06 15:34:22.919027 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:22.919002 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-68b757fb67-l47bf"] Jul 06 15:34:22.919432 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:34:22.919382 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd6fa43bb_e8ee_45ea_9726_ba18d3438f1f.slice/crio-ffe53baa1f74e327a272b106dadfcec8d00201f305c0e4bae94be5e692ac4bcb WatchSource:0}: Error finding container ffe53baa1f74e327a272b106dadfcec8d00201f305c0e4bae94be5e692ac4bcb: Status 404 returned error can't find the container with id ffe53baa1f74e327a272b106dadfcec8d00201f305c0e4bae94be5e692ac4bcb Jul 06 15:34:22.920589 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:34:22.920515 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5967a722_7558_4ab7_98f4_b3a5e367bd7a.slice/crio-dbfad6cecdf5b3c3af736f669ab741e3cd0a17c5dc7642fb70e5ae4acaba5d6b WatchSource:0}: Error finding container dbfad6cecdf5b3c3af736f669ab741e3cd0a17c5dc7642fb70e5ae4acaba5d6b: Status 404 returned error can't find the container with id dbfad6cecdf5b3c3af736f669ab741e3cd0a17c5dc7642fb70e5ae4acaba5d6b Jul 06 15:34:22.923558 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:34:22.923542 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9ad7e968_ed2e_4cbe_9b4b_780bc9d01b88.slice/crio-0731eaeee9fb0dae97ec221170517f63266042b11d52ac19abb2885ea26ee0b6 WatchSource:0}: Error finding container 0731eaeee9fb0dae97ec221170517f63266042b11d52ac19abb2885ea26ee0b6: Status 404 returned error can't find the container with id 0731eaeee9fb0dae97ec221170517f63266042b11d52ac19abb2885ea26ee0b6 Jul 06 15:34:22.926142 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:22.926112 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5c847dcfc4-nvj9g"] Jul 06 15:34:22.929139 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:34:22.929122 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0e2e4871_594a_42f5_abb7_867d61453254.slice/crio-78f45441fff219de92397eff705f4f2499f38e72fc3474cc387b2e85b5bde0e4 WatchSource:0}: Error finding container 78f45441fff219de92397eff705f4f2499f38e72fc3474cc387b2e85b5bde0e4: Status 404 returned error can't find the container with id 78f45441fff219de92397eff705f4f2499f38e72fc3474cc387b2e85b5bde0e4 Jul 06 15:34:22.932812 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:22.932505 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-28kt7"] Jul 06 15:34:22.935791 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:34:22.935768 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod929783be_42bc_444c_9c66_a3a17fd9ff04.slice/crio-8919df4d131362c9e04111c0fd830de73e3a28e0f3b29f5d8c1a34c1a7ec943b WatchSource:0}: Error finding container 8919df4d131362c9e04111c0fd830de73e3a28e0f3b29f5d8c1a34c1a7ec943b: Status 404 returned error can't find the container with id 8919df4d131362c9e04111c0fd830de73e3a28e0f3b29f5d8c1a34c1a7ec943b Jul 06 15:34:22.939490 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:22.939470 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-v8fxz" Jul 06 15:34:22.939591 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:22.939503 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pdw4n" Jul 06 15:34:22.939591 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:22.939506 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w9qbt" Jul 06 15:34:22.946493 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:22.944339 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 06 15:34:22.946493 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:22.944349 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-v99lt\"" Jul 06 15:34:22.946493 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:22.944395 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-w485s\"" Jul 06 15:34:22.946493 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:22.944519 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 06 15:34:23.124675 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:23.124371 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-nmfg6" event={"ID":"e32dedb7-fb62-41a5-9126-63a337f62a53","Type":"ContainerStarted","Data":"d3aa5a329b85dde5309f25cb040d4c44f841de844737ede0187d4693154e7064"} Jul 06 15:34:23.125649 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:23.125624 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-fkcxv" event={"ID":"d6fa43bb-e8ee-45ea-9726-ba18d3438f1f","Type":"ContainerStarted","Data":"ffe53baa1f74e327a272b106dadfcec8d00201f305c0e4bae94be5e692ac4bcb"} Jul 06 15:34:23.126716 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:23.126691 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-l47bf" event={"ID":"9ad7e968-ed2e-4cbe-9b4b-780bc9d01b88","Type":"ContainerStarted","Data":"0731eaeee9fb0dae97ec221170517f63266042b11d52ac19abb2885ea26ee0b6"} Jul 06 15:34:23.127848 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:23.127817 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-4grrz" event={"ID":"b7c1005a-9a43-4116-94cb-df36eaf220fb","Type":"ContainerStarted","Data":"5d1acc8ce86c97d164e24b5da0893375b7ad669cd90a24badf9b062ae8d0ff83"} Jul 06 15:34:23.128918 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:23.128894 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-662qp" event={"ID":"966f6ac8-fde8-46e3-b9fe-816b6f334efa","Type":"ContainerStarted","Data":"1ffb4801463c94fdac9e4d4392e4fe5b8d887c403bff14f12b43aba1465ad55c"} Jul 06 15:34:23.130145 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:23.130122 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-28kt7" event={"ID":"929783be-42bc-444c-9c66-a3a17fd9ff04","Type":"ContainerStarted","Data":"8919df4d131362c9e04111c0fd830de73e3a28e0f3b29f5d8c1a34c1a7ec943b"} Jul 06 15:34:23.131301 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:23.131281 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-qltg7" event={"ID":"5967a722-7558-4ab7-98f4-b3a5e367bd7a","Type":"ContainerStarted","Data":"dbfad6cecdf5b3c3af736f669ab741e3cd0a17c5dc7642fb70e5ae4acaba5d6b"} Jul 06 15:34:23.132409 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:23.132392 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5c847dcfc4-nvj9g" event={"ID":"0e2e4871-594a-42f5-abb7-867d61453254","Type":"ContainerStarted","Data":"78f45441fff219de92397eff705f4f2499f38e72fc3474cc387b2e85b5bde0e4"} Jul 06 15:34:23.619826 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:23.619785 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/201d3840-f397-4577-b8ba-03b31295ed3f-metrics-certs\") pod \"router-default-5855858db9-884bq\" (UID: \"201d3840-f397-4577-b8ba-03b31295ed3f\") " pod="openshift-ingress/router-default-5855858db9-884bq" Jul 06 15:34:23.620003 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:23.619866 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/bf5c30e2-bf88-468b-adf2-2e8f73f136b4-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-trjfs\" (UID: \"bf5c30e2-bf88-468b-adf2-2e8f73f136b4\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-trjfs" Jul 06 15:34:23.620003 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:23.619899 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/2e3ce465-22c0-4b33-83f4-8fdc24d06414-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-7prfp\" (UID: \"2e3ce465-22c0-4b33-83f4-8fdc24d06414\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-7prfp" Jul 06 15:34:23.620003 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:23.619990 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/f955a985-08b6-49d9-b159-8ec3916d8b20-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-fgktw\" (UID: \"f955a985-08b6-49d9-b159-8ec3916d8b20\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-fgktw" Jul 06 15:34:23.620167 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:23.620016 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/201d3840-f397-4577-b8ba-03b31295ed3f-service-ca-bundle\") pod \"router-default-5855858db9-884bq\" (UID: \"201d3840-f397-4577-b8ba-03b31295ed3f\") " pod="openshift-ingress/router-default-5855858db9-884bq" Jul 06 15:34:23.620167 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:23.620067 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1911fa1c-48ce-44ab-8b64-7ae70ce80a0d-registry-tls\") pod \"image-registry-d668d48f7-cncvq\" (UID: \"1911fa1c-48ce-44ab-8b64-7ae70ce80a0d\") " pod="openshift-image-registry/image-registry-d668d48f7-cncvq" Jul 06 15:34:23.620255 ip-10-0-129-154 kubenswrapper[2563]: E0706 15:34:23.620198 2563 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 06 15:34:23.620255 ip-10-0-129-154 kubenswrapper[2563]: E0706 15:34:23.620213 2563 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-d668d48f7-cncvq: secret "image-registry-tls" not found Jul 06 15:34:23.620344 ip-10-0-129-154 kubenswrapper[2563]: E0706 15:34:23.620284 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1911fa1c-48ce-44ab-8b64-7ae70ce80a0d-registry-tls podName:1911fa1c-48ce-44ab-8b64-7ae70ce80a0d nodeName:}" failed. No retries permitted until 2026-07-06 15:34:25.62026393 +0000 UTC m=+42.129497099 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/1911fa1c-48ce-44ab-8b64-7ae70ce80a0d-registry-tls") pod "image-registry-d668d48f7-cncvq" (UID: "1911fa1c-48ce-44ab-8b64-7ae70ce80a0d") : secret "image-registry-tls" not found Jul 06 15:34:23.620735 ip-10-0-129-154 kubenswrapper[2563]: E0706 15:34:23.620707 2563 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 06 15:34:23.620848 ip-10-0-129-154 kubenswrapper[2563]: E0706 15:34:23.620769 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/201d3840-f397-4577-b8ba-03b31295ed3f-metrics-certs podName:201d3840-f397-4577-b8ba-03b31295ed3f nodeName:}" failed. No retries permitted until 2026-07-06 15:34:25.620753125 +0000 UTC m=+42.129986292 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/201d3840-f397-4577-b8ba-03b31295ed3f-metrics-certs") pod "router-default-5855858db9-884bq" (UID: "201d3840-f397-4577-b8ba-03b31295ed3f") : secret "router-metrics-certs-default" not found Jul 06 15:34:23.620848 ip-10-0-129-154 kubenswrapper[2563]: E0706 15:34:23.620823 2563 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 06 15:34:23.620973 ip-10-0-129-154 kubenswrapper[2563]: E0706 15:34:23.620861 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bf5c30e2-bf88-468b-adf2-2e8f73f136b4-cluster-monitoring-operator-tls podName:bf5c30e2-bf88-468b-adf2-2e8f73f136b4 nodeName:}" failed. No retries permitted until 2026-07-06 15:34:25.620849947 +0000 UTC m=+42.130083114 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/bf5c30e2-bf88-468b-adf2-2e8f73f136b4-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-trjfs" (UID: "bf5c30e2-bf88-468b-adf2-2e8f73f136b4") : secret "cluster-monitoring-operator-tls" not found Jul 06 15:34:23.620973 ip-10-0-129-154 kubenswrapper[2563]: E0706 15:34:23.620920 2563 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 06 15:34:23.620973 ip-10-0-129-154 kubenswrapper[2563]: E0706 15:34:23.620952 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2e3ce465-22c0-4b33-83f4-8fdc24d06414-samples-operator-tls podName:2e3ce465-22c0-4b33-83f4-8fdc24d06414 nodeName:}" failed. No retries permitted until 2026-07-06 15:34:25.620941142 +0000 UTC m=+42.130174308 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/2e3ce465-22c0-4b33-83f4-8fdc24d06414-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-7prfp" (UID: "2e3ce465-22c0-4b33-83f4-8fdc24d06414") : secret "samples-operator-tls" not found Jul 06 15:34:23.621137 ip-10-0-129-154 kubenswrapper[2563]: E0706 15:34:23.621001 2563 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 06 15:34:23.621137 ip-10-0-129-154 kubenswrapper[2563]: E0706 15:34:23.621036 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f955a985-08b6-49d9-b159-8ec3916d8b20-networking-console-plugin-cert podName:f955a985-08b6-49d9-b159-8ec3916d8b20 nodeName:}" failed. No retries permitted until 2026-07-06 15:34:25.621023663 +0000 UTC m=+42.130256825 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/f955a985-08b6-49d9-b159-8ec3916d8b20-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-fgktw" (UID: "f955a985-08b6-49d9-b159-8ec3916d8b20") : secret "networking-console-plugin-cert" not found Jul 06 15:34:23.621137 ip-10-0-129-154 kubenswrapper[2563]: E0706 15:34:23.621100 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/201d3840-f397-4577-b8ba-03b31295ed3f-service-ca-bundle podName:201d3840-f397-4577-b8ba-03b31295ed3f nodeName:}" failed. No retries permitted until 2026-07-06 15:34:25.621090013 +0000 UTC m=+42.130323179 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/201d3840-f397-4577-b8ba-03b31295ed3f-service-ca-bundle") pod "router-default-5855858db9-884bq" (UID: "201d3840-f397-4577-b8ba-03b31295ed3f") : configmap references non-existent config key: service-ca.crt Jul 06 15:34:23.721868 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:23.721296 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/17c20e5e-6515-48d6-8c51-425a9e86485d-metrics-tls\") pod \"dns-default-h86nn\" (UID: \"17c20e5e-6515-48d6-8c51-425a9e86485d\") " pod="openshift-dns/dns-default-h86nn" Jul 06 15:34:23.721868 ip-10-0-129-154 kubenswrapper[2563]: E0706 15:34:23.721461 2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 06 15:34:23.721868 ip-10-0-129-154 kubenswrapper[2563]: E0706 15:34:23.721522 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/17c20e5e-6515-48d6-8c51-425a9e86485d-metrics-tls podName:17c20e5e-6515-48d6-8c51-425a9e86485d nodeName:}" failed. No retries permitted until 2026-07-06 15:34:25.721502372 +0000 UTC m=+42.230735542 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/17c20e5e-6515-48d6-8c51-425a9e86485d-metrics-tls") pod "dns-default-h86nn" (UID: "17c20e5e-6515-48d6-8c51-425a9e86485d") : secret "dns-default-metrics-tls" not found Jul 06 15:34:24.146910 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:24.146088 2563 generic.go:356] "Generic (PLEG): container finished" podID="e32dedb7-fb62-41a5-9126-63a337f62a53" containerID="d3aa5a329b85dde5309f25cb040d4c44f841de844737ede0187d4693154e7064" exitCode=0 Jul 06 15:34:24.146910 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:24.146155 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-nmfg6" event={"ID":"e32dedb7-fb62-41a5-9126-63a337f62a53","Type":"ContainerDied","Data":"d3aa5a329b85dde5309f25cb040d4c44f841de844737ede0187d4693154e7064"} Jul 06 15:34:25.152648 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:25.152614 2563 generic.go:356] "Generic (PLEG): container finished" podID="e32dedb7-fb62-41a5-9126-63a337f62a53" containerID="432274e26857612543eef9237488623d413686d5658f0090b29729c8cc0c42c5" exitCode=0 Jul 06 15:34:25.153274 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:25.152682 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-nmfg6" event={"ID":"e32dedb7-fb62-41a5-9126-63a337f62a53","Type":"ContainerDied","Data":"432274e26857612543eef9237488623d413686d5658f0090b29729c8cc0c42c5"} Jul 06 15:34:25.644046 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:25.643947 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1911fa1c-48ce-44ab-8b64-7ae70ce80a0d-registry-tls\") pod \"image-registry-d668d48f7-cncvq\" (UID: \"1911fa1c-48ce-44ab-8b64-7ae70ce80a0d\") " pod="openshift-image-registry/image-registry-d668d48f7-cncvq" Jul 06 15:34:25.644046 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:25.644002 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/201d3840-f397-4577-b8ba-03b31295ed3f-metrics-certs\") pod \"router-default-5855858db9-884bq\" (UID: \"201d3840-f397-4577-b8ba-03b31295ed3f\") " pod="openshift-ingress/router-default-5855858db9-884bq" Jul 06 15:34:25.644286 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:25.644056 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/bf5c30e2-bf88-468b-adf2-2e8f73f136b4-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-trjfs\" (UID: \"bf5c30e2-bf88-468b-adf2-2e8f73f136b4\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-trjfs" Jul 06 15:34:25.644286 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:25.644083 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/2e3ce465-22c0-4b33-83f4-8fdc24d06414-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-7prfp\" (UID: \"2e3ce465-22c0-4b33-83f4-8fdc24d06414\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-7prfp" Jul 06 15:34:25.644286 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:25.644159 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/f955a985-08b6-49d9-b159-8ec3916d8b20-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-fgktw\" (UID: \"f955a985-08b6-49d9-b159-8ec3916d8b20\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-fgktw" Jul 06 15:34:25.644286 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:25.644184 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/201d3840-f397-4577-b8ba-03b31295ed3f-service-ca-bundle\") pod \"router-default-5855858db9-884bq\" (UID: \"201d3840-f397-4577-b8ba-03b31295ed3f\") " pod="openshift-ingress/router-default-5855858db9-884bq" Jul 06 15:34:25.644518 ip-10-0-129-154 kubenswrapper[2563]: E0706 15:34:25.644307 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/201d3840-f397-4577-b8ba-03b31295ed3f-service-ca-bundle podName:201d3840-f397-4577-b8ba-03b31295ed3f nodeName:}" failed. No retries permitted until 2026-07-06 15:34:29.644291832 +0000 UTC m=+46.153524997 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/201d3840-f397-4577-b8ba-03b31295ed3f-service-ca-bundle") pod "router-default-5855858db9-884bq" (UID: "201d3840-f397-4577-b8ba-03b31295ed3f") : configmap references non-existent config key: service-ca.crt Jul 06 15:34:25.644518 ip-10-0-129-154 kubenswrapper[2563]: E0706 15:34:25.644373 2563 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 06 15:34:25.644518 ip-10-0-129-154 kubenswrapper[2563]: E0706 15:34:25.644382 2563 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-d668d48f7-cncvq: secret "image-registry-tls" not found Jul 06 15:34:25.644518 ip-10-0-129-154 kubenswrapper[2563]: E0706 15:34:25.644406 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1911fa1c-48ce-44ab-8b64-7ae70ce80a0d-registry-tls podName:1911fa1c-48ce-44ab-8b64-7ae70ce80a0d nodeName:}" failed. No retries permitted until 2026-07-06 15:34:29.644398104 +0000 UTC m=+46.153631259 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/1911fa1c-48ce-44ab-8b64-7ae70ce80a0d-registry-tls") pod "image-registry-d668d48f7-cncvq" (UID: "1911fa1c-48ce-44ab-8b64-7ae70ce80a0d") : secret "image-registry-tls" not found Jul 06 15:34:25.644518 ip-10-0-129-154 kubenswrapper[2563]: E0706 15:34:25.644470 2563 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 06 15:34:25.644518 ip-10-0-129-154 kubenswrapper[2563]: E0706 15:34:25.644493 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/201d3840-f397-4577-b8ba-03b31295ed3f-metrics-certs podName:201d3840-f397-4577-b8ba-03b31295ed3f nodeName:}" failed. No retries permitted until 2026-07-06 15:34:29.64448693 +0000 UTC m=+46.153720085 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/201d3840-f397-4577-b8ba-03b31295ed3f-metrics-certs") pod "router-default-5855858db9-884bq" (UID: "201d3840-f397-4577-b8ba-03b31295ed3f") : secret "router-metrics-certs-default" not found Jul 06 15:34:25.644829 ip-10-0-129-154 kubenswrapper[2563]: E0706 15:34:25.644532 2563 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 06 15:34:25.644829 ip-10-0-129-154 kubenswrapper[2563]: E0706 15:34:25.644564 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bf5c30e2-bf88-468b-adf2-2e8f73f136b4-cluster-monitoring-operator-tls podName:bf5c30e2-bf88-468b-adf2-2e8f73f136b4 nodeName:}" failed. No retries permitted until 2026-07-06 15:34:29.644554499 +0000 UTC m=+46.153787666 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/bf5c30e2-bf88-468b-adf2-2e8f73f136b4-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-trjfs" (UID: "bf5c30e2-bf88-468b-adf2-2e8f73f136b4") : secret "cluster-monitoring-operator-tls" not found Jul 06 15:34:25.644829 ip-10-0-129-154 kubenswrapper[2563]: E0706 15:34:25.644612 2563 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 06 15:34:25.644829 ip-10-0-129-154 kubenswrapper[2563]: E0706 15:34:25.644635 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2e3ce465-22c0-4b33-83f4-8fdc24d06414-samples-operator-tls podName:2e3ce465-22c0-4b33-83f4-8fdc24d06414 nodeName:}" failed. No retries permitted until 2026-07-06 15:34:29.644628749 +0000 UTC m=+46.153861903 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/2e3ce465-22c0-4b33-83f4-8fdc24d06414-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-7prfp" (UID: "2e3ce465-22c0-4b33-83f4-8fdc24d06414") : secret "samples-operator-tls" not found Jul 06 15:34:25.644829 ip-10-0-129-154 kubenswrapper[2563]: E0706 15:34:25.644677 2563 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 06 15:34:25.644829 ip-10-0-129-154 kubenswrapper[2563]: E0706 15:34:25.644702 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f955a985-08b6-49d9-b159-8ec3916d8b20-networking-console-plugin-cert podName:f955a985-08b6-49d9-b159-8ec3916d8b20 nodeName:}" failed. No retries permitted until 2026-07-06 15:34:29.644695672 +0000 UTC m=+46.153928826 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/f955a985-08b6-49d9-b159-8ec3916d8b20-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-fgktw" (UID: "f955a985-08b6-49d9-b159-8ec3916d8b20") : secret "networking-console-plugin-cert" not found Jul 06 15:34:25.745816 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:25.745779 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/17c20e5e-6515-48d6-8c51-425a9e86485d-metrics-tls\") pod \"dns-default-h86nn\" (UID: \"17c20e5e-6515-48d6-8c51-425a9e86485d\") " pod="openshift-dns/dns-default-h86nn" Jul 06 15:34:25.745982 ip-10-0-129-154 kubenswrapper[2563]: E0706 15:34:25.745925 2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 06 15:34:25.746053 ip-10-0-129-154 kubenswrapper[2563]: E0706 15:34:25.745986 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/17c20e5e-6515-48d6-8c51-425a9e86485d-metrics-tls podName:17c20e5e-6515-48d6-8c51-425a9e86485d nodeName:}" failed. No retries permitted until 2026-07-06 15:34:29.745971122 +0000 UTC m=+46.255204276 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/17c20e5e-6515-48d6-8c51-425a9e86485d-metrics-tls") pod "dns-default-h86nn" (UID: "17c20e5e-6515-48d6-8c51-425a9e86485d") : secret "dns-default-metrics-tls" not found Jul 06 15:34:26.159136 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:26.158888 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-nmfg6" event={"ID":"e32dedb7-fb62-41a5-9126-63a337f62a53","Type":"ContainerStarted","Data":"3d923ed1780c02a005595440a655d4cad7b706b235eebc4a9b5e353342f4c84b"} Jul 06 15:34:26.160532 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:26.160492 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-4grrz" event={"ID":"b7c1005a-9a43-4116-94cb-df36eaf220fb","Type":"ContainerStarted","Data":"41d7ef01673d3047f5b16d101020fbfad139af0d1d5cfca7f28476b37af468e7"} Jul 06 15:34:26.185242 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:26.185186 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-nmfg6" podStartSLOduration=8.349109128 podStartE2EDuration="42.185167608s" totalImagesPullingTime="33.83605848s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 15:33:44 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 15:34:26.183604347 +0000 UTC m=+42.692837521" watchObservedRunningTime="2026-07-06 15:34:26.185167608 +0000 UTC m=+42.694400783" Jul 06 15:34:26.200461 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:26.200285 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-4grrz" podStartSLOduration=13.017394673 podStartE2EDuration="15.20026797s" totalImagesPullingTime="2.182873297s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 15:34:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 15:34:26.200082285 +0000 UTC m=+42.709315459" watchObservedRunningTime="2026-07-06 15:34:26.20026797 +0000 UTC m=+42.709501145" Jul 06 15:34:29.681958 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:29.681920 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/201d3840-f397-4577-b8ba-03b31295ed3f-metrics-certs\") pod \"router-default-5855858db9-884bq\" (UID: \"201d3840-f397-4577-b8ba-03b31295ed3f\") " pod="openshift-ingress/router-default-5855858db9-884bq" Jul 06 15:34:29.682488 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:29.681995 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/bf5c30e2-bf88-468b-adf2-2e8f73f136b4-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-trjfs\" (UID: \"bf5c30e2-bf88-468b-adf2-2e8f73f136b4\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-trjfs" Jul 06 15:34:29.682488 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:29.682028 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/2e3ce465-22c0-4b33-83f4-8fdc24d06414-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-7prfp\" (UID: \"2e3ce465-22c0-4b33-83f4-8fdc24d06414\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-7prfp" Jul 06 15:34:29.682488 ip-10-0-129-154 kubenswrapper[2563]: E0706 15:34:29.682096 2563 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 06 15:34:29.682488 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:29.682126 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/f955a985-08b6-49d9-b159-8ec3916d8b20-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-fgktw\" (UID: \"f955a985-08b6-49d9-b159-8ec3916d8b20\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-fgktw" Jul 06 15:34:29.682488 ip-10-0-129-154 kubenswrapper[2563]: E0706 15:34:29.682166 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/201d3840-f397-4577-b8ba-03b31295ed3f-metrics-certs podName:201d3840-f397-4577-b8ba-03b31295ed3f nodeName:}" failed. No retries permitted until 2026-07-06 15:34:37.682148045 +0000 UTC m=+54.191381208 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/201d3840-f397-4577-b8ba-03b31295ed3f-metrics-certs") pod "router-default-5855858db9-884bq" (UID: "201d3840-f397-4577-b8ba-03b31295ed3f") : secret "router-metrics-certs-default" not found Jul 06 15:34:29.682488 ip-10-0-129-154 kubenswrapper[2563]: E0706 15:34:29.682327 2563 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 06 15:34:29.682488 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:29.682380 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/201d3840-f397-4577-b8ba-03b31295ed3f-service-ca-bundle\") pod \"router-default-5855858db9-884bq\" (UID: \"201d3840-f397-4577-b8ba-03b31295ed3f\") " pod="openshift-ingress/router-default-5855858db9-884bq" Jul 06 15:34:29.682488 ip-10-0-129-154 kubenswrapper[2563]: E0706 15:34:29.682396 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bf5c30e2-bf88-468b-adf2-2e8f73f136b4-cluster-monitoring-operator-tls podName:bf5c30e2-bf88-468b-adf2-2e8f73f136b4 nodeName:}" failed. No retries permitted until 2026-07-06 15:34:37.682377314 +0000 UTC m=+54.191610485 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/bf5c30e2-bf88-468b-adf2-2e8f73f136b4-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-trjfs" (UID: "bf5c30e2-bf88-468b-adf2-2e8f73f136b4") : secret "cluster-monitoring-operator-tls" not found Jul 06 15:34:29.682488 ip-10-0-129-154 kubenswrapper[2563]: E0706 15:34:29.682493 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/201d3840-f397-4577-b8ba-03b31295ed3f-service-ca-bundle podName:201d3840-f397-4577-b8ba-03b31295ed3f nodeName:}" failed. No retries permitted until 2026-07-06 15:34:37.682478083 +0000 UTC m=+54.191711246 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/201d3840-f397-4577-b8ba-03b31295ed3f-service-ca-bundle") pod "router-default-5855858db9-884bq" (UID: "201d3840-f397-4577-b8ba-03b31295ed3f") : configmap references non-existent config key: service-ca.crt Jul 06 15:34:29.682876 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:29.682518 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1911fa1c-48ce-44ab-8b64-7ae70ce80a0d-registry-tls\") pod \"image-registry-d668d48f7-cncvq\" (UID: \"1911fa1c-48ce-44ab-8b64-7ae70ce80a0d\") " pod="openshift-image-registry/image-registry-d668d48f7-cncvq" Jul 06 15:34:29.682876 ip-10-0-129-154 kubenswrapper[2563]: E0706 15:34:29.682582 2563 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 06 15:34:29.682876 ip-10-0-129-154 kubenswrapper[2563]: E0706 15:34:29.682638 2563 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 06 15:34:29.682876 ip-10-0-129-154 kubenswrapper[2563]: E0706 15:34:29.682640 2563 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 06 15:34:29.682876 ip-10-0-129-154 kubenswrapper[2563]: E0706 15:34:29.682651 2563 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-d668d48f7-cncvq: secret "image-registry-tls" not found Jul 06 15:34:29.682876 ip-10-0-129-154 kubenswrapper[2563]: E0706 15:34:29.682642 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f955a985-08b6-49d9-b159-8ec3916d8b20-networking-console-plugin-cert podName:f955a985-08b6-49d9-b159-8ec3916d8b20 nodeName:}" failed. No retries permitted until 2026-07-06 15:34:37.682625071 +0000 UTC m=+54.191858227 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/f955a985-08b6-49d9-b159-8ec3916d8b20-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-fgktw" (UID: "f955a985-08b6-49d9-b159-8ec3916d8b20") : secret "networking-console-plugin-cert" not found Jul 06 15:34:29.682876 ip-10-0-129-154 kubenswrapper[2563]: E0706 15:34:29.682703 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2e3ce465-22c0-4b33-83f4-8fdc24d06414-samples-operator-tls podName:2e3ce465-22c0-4b33-83f4-8fdc24d06414 nodeName:}" failed. No retries permitted until 2026-07-06 15:34:37.682691054 +0000 UTC m=+54.191924206 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/2e3ce465-22c0-4b33-83f4-8fdc24d06414-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-7prfp" (UID: "2e3ce465-22c0-4b33-83f4-8fdc24d06414") : secret "samples-operator-tls" not found Jul 06 15:34:29.682876 ip-10-0-129-154 kubenswrapper[2563]: E0706 15:34:29.682715 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1911fa1c-48ce-44ab-8b64-7ae70ce80a0d-registry-tls podName:1911fa1c-48ce-44ab-8b64-7ae70ce80a0d nodeName:}" failed. No retries permitted until 2026-07-06 15:34:37.68270808 +0000 UTC m=+54.191941233 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/1911fa1c-48ce-44ab-8b64-7ae70ce80a0d-registry-tls") pod "image-registry-d668d48f7-cncvq" (UID: "1911fa1c-48ce-44ab-8b64-7ae70ce80a0d") : secret "image-registry-tls" not found Jul 06 15:34:29.783627 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:29.783594 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/17c20e5e-6515-48d6-8c51-425a9e86485d-metrics-tls\") pod \"dns-default-h86nn\" (UID: \"17c20e5e-6515-48d6-8c51-425a9e86485d\") " pod="openshift-dns/dns-default-h86nn" Jul 06 15:34:29.783799 ip-10-0-129-154 kubenswrapper[2563]: E0706 15:34:29.783771 2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 06 15:34:29.783870 ip-10-0-129-154 kubenswrapper[2563]: E0706 15:34:29.783841 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/17c20e5e-6515-48d6-8c51-425a9e86485d-metrics-tls podName:17c20e5e-6515-48d6-8c51-425a9e86485d nodeName:}" failed. No retries permitted until 2026-07-06 15:34:37.783820424 +0000 UTC m=+54.293053576 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/17c20e5e-6515-48d6-8c51-425a9e86485d-metrics-tls") pod "dns-default-h86nn" (UID: "17c20e5e-6515-48d6-8c51-425a9e86485d") : secret "dns-default-metrics-tls" not found Jul 06 15:34:31.183827 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:31.183247 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-fkcxv" event={"ID":"d6fa43bb-e8ee-45ea-9726-ba18d3438f1f","Type":"ContainerStarted","Data":"04955c13a11bea922f6d67b9a750631ab838c4ed31f741e8812ecead93b80ffb"} Jul 06 15:34:31.187111 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:31.186395 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-l47bf_9ad7e968-ed2e-4cbe-9b4b-780bc9d01b88/console-operator/0.log" Jul 06 15:34:31.187111 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:31.186460 2563 generic.go:356] "Generic (PLEG): container finished" podID="9ad7e968-ed2e-4cbe-9b4b-780bc9d01b88" containerID="90e1d5e5ca6d7dfbc2f9ee2ebfc197da5c6dd58317bb25f635974e5ad5f1dd86" exitCode=255 Jul 06 15:34:31.187111 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:31.186529 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-l47bf" event={"ID":"9ad7e968-ed2e-4cbe-9b4b-780bc9d01b88","Type":"ContainerDied","Data":"90e1d5e5ca6d7dfbc2f9ee2ebfc197da5c6dd58317bb25f635974e5ad5f1dd86"} Jul 06 15:34:31.187111 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:31.186818 2563 scope.go:117] "RemoveContainer" containerID="90e1d5e5ca6d7dfbc2f9ee2ebfc197da5c6dd58317bb25f635974e5ad5f1dd86" Jul 06 15:34:31.190119 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:31.188771 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-662qp" event={"ID":"966f6ac8-fde8-46e3-b9fe-816b6f334efa","Type":"ContainerStarted","Data":"5c23b23a2eb925f2b22b3d2363322bbe1ba87661591daa2c13ea7c2a2163cf78"} Jul 06 15:34:31.191257 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:31.191214 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-28kt7" event={"ID":"929783be-42bc-444c-9c66-a3a17fd9ff04","Type":"ContainerStarted","Data":"8bda6142bb256e89cb2b5d85c534bc5e29680e3fecf442b0e33d10e13955b2ef"} Jul 06 15:34:31.193744 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:31.193574 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-qltg7" event={"ID":"5967a722-7558-4ab7-98f4-b3a5e367bd7a","Type":"ContainerStarted","Data":"135d21d048c1545748f8732fc1c594b6f8e8060f31caa2d2430b10c9f1d53383"} Jul 06 15:34:31.195741 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:31.195720 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5c847dcfc4-nvj9g" event={"ID":"0e2e4871-594a-42f5-abb7-867d61453254","Type":"ContainerStarted","Data":"8b8d5c03f37cc0a4bda97b51dcbc0fafa10cf71a83f229be05696af5ddb9b797"} Jul 06 15:34:31.207780 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:31.207727 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-fkcxv" podStartSLOduration=10.390630651 podStartE2EDuration="18.20770997s" totalImagesPullingTime="7.817079319s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 15:34:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 15:34:31.205815108 +0000 UTC m=+47.715048285" watchObservedRunningTime="2026-07-06 15:34:31.20770997 +0000 UTC m=+47.716943145" Jul 06 15:34:31.240409 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:31.240354 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-5c847dcfc4-nvj9g" podStartSLOduration=12.422480297 podStartE2EDuration="20.240337979s" totalImagesPullingTime="7.817857682s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 15:34:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 15:34:31.238971441 +0000 UTC m=+47.748204612" watchObservedRunningTime="2026-07-06 15:34:31.240337979 +0000 UTC m=+47.749571150" Jul 06 15:34:31.268731 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:31.267168 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-662qp" podStartSLOduration=12.437478311 podStartE2EDuration="20.267145981s" totalImagesPullingTime="7.82966767s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 15:34:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 15:34:31.266402188 +0000 UTC m=+47.775635363" watchObservedRunningTime="2026-07-06 15:34:31.267145981 +0000 UTC m=+47.776379156" Jul 06 15:34:31.321791 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:31.321722 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-28kt7" podStartSLOduration=12.510448793 podStartE2EDuration="20.321670817s" totalImagesPullingTime="7.811222024s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 15:34:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 15:34:31.320320963 +0000 UTC m=+47.829554139" watchObservedRunningTime="2026-07-06 15:34:31.321670817 +0000 UTC m=+47.830903992" Jul 06 15:34:31.341885 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:31.341768 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-qltg7" podStartSLOduration=2.524204639 podStartE2EDuration="10.341748993s" totalImagesPullingTime="7.817544354s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 15:34:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 15:34:31.340622956 +0000 UTC m=+47.849856131" watchObservedRunningTime="2026-07-06 15:34:31.341748993 +0000 UTC m=+47.850982170" Jul 06 15:34:32.200341 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:32.200314 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-l47bf_9ad7e968-ed2e-4cbe-9b4b-780bc9d01b88/console-operator/1.log" Jul 06 15:34:32.200770 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:32.200732 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-l47bf_9ad7e968-ed2e-4cbe-9b4b-780bc9d01b88/console-operator/0.log" Jul 06 15:34:32.200770 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:32.200765 2563 generic.go:356] "Generic (PLEG): container finished" podID="9ad7e968-ed2e-4cbe-9b4b-780bc9d01b88" containerID="a81099975cdcb5e7127528428fba030cd7fb5b0a69da693709c895b24d4ac2b3" exitCode=255 Jul 06 15:34:32.200892 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:32.200864 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-l47bf" event={"ID":"9ad7e968-ed2e-4cbe-9b4b-780bc9d01b88","Type":"ContainerDied","Data":"a81099975cdcb5e7127528428fba030cd7fb5b0a69da693709c895b24d4ac2b3"} Jul 06 15:34:32.200959 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:32.200940 2563 scope.go:117] "RemoveContainer" containerID="90e1d5e5ca6d7dfbc2f9ee2ebfc197da5c6dd58317bb25f635974e5ad5f1dd86" Jul 06 15:34:32.201271 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:32.201172 2563 scope.go:117] "RemoveContainer" containerID="a81099975cdcb5e7127528428fba030cd7fb5b0a69da693709c895b24d4ac2b3" Jul 06 15:34:32.201397 ip-10-0-129-154 kubenswrapper[2563]: E0706 15:34:32.201367 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-68b757fb67-l47bf_openshift-console-operator(9ad7e968-ed2e-4cbe-9b4b-780bc9d01b88)\"" pod="openshift-console-operator/console-operator-68b757fb67-l47bf" podUID="9ad7e968-ed2e-4cbe-9b4b-780bc9d01b88" Jul 06 15:34:32.255308 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:32.255289 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-68b757fb67-l47bf" Jul 06 15:34:32.255410 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:32.255326 2563 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-68b757fb67-l47bf" Jul 06 15:34:32.980450 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:32.980407 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-pbh67"] Jul 06 15:34:32.990494 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:32.990472 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-pbh67" Jul 06 15:34:32.993808 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:32.993783 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-pbh67"] Jul 06 15:34:32.994740 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:32.994718 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 06 15:34:32.994740 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:32.994725 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-5rtrq\"" Jul 06 15:34:32.997597 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:32.996621 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 06 15:34:33.013825 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:33.013798 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/e6b68ed7-afa1-44b1-acf0-50c8addc4c6c-data-volume\") pod \"insights-runtime-extractor-pbh67\" (UID: \"e6b68ed7-afa1-44b1-acf0-50c8addc4c6c\") " pod="openshift-insights/insights-runtime-extractor-pbh67" Jul 06 15:34:33.013926 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:33.013832 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/e6b68ed7-afa1-44b1-acf0-50c8addc4c6c-crio-socket\") pod \"insights-runtime-extractor-pbh67\" (UID: \"e6b68ed7-afa1-44b1-acf0-50c8addc4c6c\") " pod="openshift-insights/insights-runtime-extractor-pbh67" Jul 06 15:34:33.013926 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:33.013898 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/e6b68ed7-afa1-44b1-acf0-50c8addc4c6c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-pbh67\" (UID: \"e6b68ed7-afa1-44b1-acf0-50c8addc4c6c\") " pod="openshift-insights/insights-runtime-extractor-pbh67" Jul 06 15:34:33.013926 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:33.013914 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9w8c9\" (UniqueName: \"kubernetes.io/projected/e6b68ed7-afa1-44b1-acf0-50c8addc4c6c-kube-api-access-9w8c9\") pod \"insights-runtime-extractor-pbh67\" (UID: \"e6b68ed7-afa1-44b1-acf0-50c8addc4c6c\") " pod="openshift-insights/insights-runtime-extractor-pbh67" Jul 06 15:34:33.014029 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:33.013999 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e6b68ed7-afa1-44b1-acf0-50c8addc4c6c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-pbh67\" (UID: \"e6b68ed7-afa1-44b1-acf0-50c8addc4c6c\") " pod="openshift-insights/insights-runtime-extractor-pbh67" Jul 06 15:34:33.114525 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:33.114491 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/e6b68ed7-afa1-44b1-acf0-50c8addc4c6c-data-volume\") pod \"insights-runtime-extractor-pbh67\" (UID: \"e6b68ed7-afa1-44b1-acf0-50c8addc4c6c\") " pod="openshift-insights/insights-runtime-extractor-pbh67" Jul 06 15:34:33.114680 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:33.114535 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/e6b68ed7-afa1-44b1-acf0-50c8addc4c6c-crio-socket\") pod \"insights-runtime-extractor-pbh67\" (UID: \"e6b68ed7-afa1-44b1-acf0-50c8addc4c6c\") " pod="openshift-insights/insights-runtime-extractor-pbh67" Jul 06 15:34:33.114680 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:33.114582 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/e6b68ed7-afa1-44b1-acf0-50c8addc4c6c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-pbh67\" (UID: \"e6b68ed7-afa1-44b1-acf0-50c8addc4c6c\") " pod="openshift-insights/insights-runtime-extractor-pbh67" Jul 06 15:34:33.114680 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:33.114597 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9w8c9\" (UniqueName: \"kubernetes.io/projected/e6b68ed7-afa1-44b1-acf0-50c8addc4c6c-kube-api-access-9w8c9\") pod \"insights-runtime-extractor-pbh67\" (UID: \"e6b68ed7-afa1-44b1-acf0-50c8addc4c6c\") " pod="openshift-insights/insights-runtime-extractor-pbh67" Jul 06 15:34:33.114839 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:33.114681 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e6b68ed7-afa1-44b1-acf0-50c8addc4c6c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-pbh67\" (UID: \"e6b68ed7-afa1-44b1-acf0-50c8addc4c6c\") " pod="openshift-insights/insights-runtime-extractor-pbh67" Jul 06 15:34:33.114839 ip-10-0-129-154 kubenswrapper[2563]: E0706 15:34:33.114703 2563 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 06 15:34:33.114839 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:33.114826 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/e6b68ed7-afa1-44b1-acf0-50c8addc4c6c-data-volume\") pod \"insights-runtime-extractor-pbh67\" (UID: \"e6b68ed7-afa1-44b1-acf0-50c8addc4c6c\") " pod="openshift-insights/insights-runtime-extractor-pbh67" Jul 06 15:34:33.114979 ip-10-0-129-154 kubenswrapper[2563]: E0706 15:34:33.114853 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e6b68ed7-afa1-44b1-acf0-50c8addc4c6c-kube-rbac-proxy-cm podName:e6b68ed7-afa1-44b1-acf0-50c8addc4c6c nodeName:}" failed. No retries permitted until 2026-07-06 15:34:33.6148148 +0000 UTC m=+50.124047955 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/e6b68ed7-afa1-44b1-acf0-50c8addc4c6c-kube-rbac-proxy-cm") pod "insights-runtime-extractor-pbh67" (UID: "e6b68ed7-afa1-44b1-acf0-50c8addc4c6c") : configmap "kube-rbac-proxy" not found Jul 06 15:34:33.114979 ip-10-0-129-154 kubenswrapper[2563]: E0706 15:34:33.114856 2563 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 06 15:34:33.114979 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:33.114910 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/e6b68ed7-afa1-44b1-acf0-50c8addc4c6c-crio-socket\") pod \"insights-runtime-extractor-pbh67\" (UID: \"e6b68ed7-afa1-44b1-acf0-50c8addc4c6c\") " pod="openshift-insights/insights-runtime-extractor-pbh67" Jul 06 15:34:33.114979 ip-10-0-129-154 kubenswrapper[2563]: E0706 15:34:33.114935 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e6b68ed7-afa1-44b1-acf0-50c8addc4c6c-insights-runtime-extractor-tls podName:e6b68ed7-afa1-44b1-acf0-50c8addc4c6c nodeName:}" failed. No retries permitted until 2026-07-06 15:34:33.614917234 +0000 UTC m=+50.124150407 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/e6b68ed7-afa1-44b1-acf0-50c8addc4c6c-insights-runtime-extractor-tls") pod "insights-runtime-extractor-pbh67" (UID: "e6b68ed7-afa1-44b1-acf0-50c8addc4c6c") : secret "insights-runtime-extractor-tls" not found Jul 06 15:34:33.128801 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:33.128780 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9w8c9\" (UniqueName: \"kubernetes.io/projected/e6b68ed7-afa1-44b1-acf0-50c8addc4c6c-kube-api-access-9w8c9\") pod \"insights-runtime-extractor-pbh67\" (UID: \"e6b68ed7-afa1-44b1-acf0-50c8addc4c6c\") " pod="openshift-insights/insights-runtime-extractor-pbh67" Jul 06 15:34:33.205429 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:33.205402 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-l47bf_9ad7e968-ed2e-4cbe-9b4b-780bc9d01b88/console-operator/1.log" Jul 06 15:34:33.205803 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:33.205763 2563 scope.go:117] "RemoveContainer" containerID="a81099975cdcb5e7127528428fba030cd7fb5b0a69da693709c895b24d4ac2b3" Jul 06 15:34:33.205958 ip-10-0-129-154 kubenswrapper[2563]: E0706 15:34:33.205938 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-68b757fb67-l47bf_openshift-console-operator(9ad7e968-ed2e-4cbe-9b4b-780bc9d01b88)\"" pod="openshift-console-operator/console-operator-68b757fb67-l47bf" podUID="9ad7e968-ed2e-4cbe-9b4b-780bc9d01b88" Jul 06 15:34:33.620375 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:33.620338 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/e6b68ed7-afa1-44b1-acf0-50c8addc4c6c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-pbh67\" (UID: \"e6b68ed7-afa1-44b1-acf0-50c8addc4c6c\") " pod="openshift-insights/insights-runtime-extractor-pbh67" Jul 06 15:34:33.620662 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:33.620402 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e6b68ed7-afa1-44b1-acf0-50c8addc4c6c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-pbh67\" (UID: \"e6b68ed7-afa1-44b1-acf0-50c8addc4c6c\") " pod="openshift-insights/insights-runtime-extractor-pbh67" Jul 06 15:34:33.620662 ip-10-0-129-154 kubenswrapper[2563]: E0706 15:34:33.620491 2563 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 06 15:34:33.620662 ip-10-0-129-154 kubenswrapper[2563]: E0706 15:34:33.620516 2563 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 06 15:34:33.620662 ip-10-0-129-154 kubenswrapper[2563]: E0706 15:34:33.620587 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e6b68ed7-afa1-44b1-acf0-50c8addc4c6c-kube-rbac-proxy-cm podName:e6b68ed7-afa1-44b1-acf0-50c8addc4c6c nodeName:}" failed. No retries permitted until 2026-07-06 15:34:34.620564813 +0000 UTC m=+51.129797995 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/e6b68ed7-afa1-44b1-acf0-50c8addc4c6c-kube-rbac-proxy-cm") pod "insights-runtime-extractor-pbh67" (UID: "e6b68ed7-afa1-44b1-acf0-50c8addc4c6c") : configmap "kube-rbac-proxy" not found Jul 06 15:34:33.620662 ip-10-0-129-154 kubenswrapper[2563]: E0706 15:34:33.620610 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e6b68ed7-afa1-44b1-acf0-50c8addc4c6c-insights-runtime-extractor-tls podName:e6b68ed7-afa1-44b1-acf0-50c8addc4c6c nodeName:}" failed. No retries permitted until 2026-07-06 15:34:34.620599497 +0000 UTC m=+51.129832656 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/e6b68ed7-afa1-44b1-acf0-50c8addc4c6c-insights-runtime-extractor-tls") pod "insights-runtime-extractor-pbh67" (UID: "e6b68ed7-afa1-44b1-acf0-50c8addc4c6c") : secret "insights-runtime-extractor-tls" not found Jul 06 15:34:34.208519 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:34.208492 2563 scope.go:117] "RemoveContainer" containerID="a81099975cdcb5e7127528428fba030cd7fb5b0a69da693709c895b24d4ac2b3" Jul 06 15:34:34.208992 ip-10-0-129-154 kubenswrapper[2563]: E0706 15:34:34.208692 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-68b757fb67-l47bf_openshift-console-operator(9ad7e968-ed2e-4cbe-9b4b-780bc9d01b88)\"" pod="openshift-console-operator/console-operator-68b757fb67-l47bf" podUID="9ad7e968-ed2e-4cbe-9b4b-780bc9d01b88" Jul 06 15:34:34.631033 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:34.630949 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/e6b68ed7-afa1-44b1-acf0-50c8addc4c6c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-pbh67\" (UID: \"e6b68ed7-afa1-44b1-acf0-50c8addc4c6c\") " pod="openshift-insights/insights-runtime-extractor-pbh67" Jul 06 15:34:34.631033 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:34.631015 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e6b68ed7-afa1-44b1-acf0-50c8addc4c6c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-pbh67\" (UID: \"e6b68ed7-afa1-44b1-acf0-50c8addc4c6c\") " pod="openshift-insights/insights-runtime-extractor-pbh67" Jul 06 15:34:34.631257 ip-10-0-129-154 kubenswrapper[2563]: E0706 15:34:34.631105 2563 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 06 15:34:34.631257 ip-10-0-129-154 kubenswrapper[2563]: E0706 15:34:34.631148 2563 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 06 15:34:34.631257 ip-10-0-129-154 kubenswrapper[2563]: E0706 15:34:34.631192 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e6b68ed7-afa1-44b1-acf0-50c8addc4c6c-kube-rbac-proxy-cm podName:e6b68ed7-afa1-44b1-acf0-50c8addc4c6c nodeName:}" failed. No retries permitted until 2026-07-06 15:34:36.631172893 +0000 UTC m=+53.140406056 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/e6b68ed7-afa1-44b1-acf0-50c8addc4c6c-kube-rbac-proxy-cm") pod "insights-runtime-extractor-pbh67" (UID: "e6b68ed7-afa1-44b1-acf0-50c8addc4c6c") : configmap "kube-rbac-proxy" not found Jul 06 15:34:34.631257 ip-10-0-129-154 kubenswrapper[2563]: E0706 15:34:34.631215 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e6b68ed7-afa1-44b1-acf0-50c8addc4c6c-insights-runtime-extractor-tls podName:e6b68ed7-afa1-44b1-acf0-50c8addc4c6c nodeName:}" failed. No retries permitted until 2026-07-06 15:34:36.631208488 +0000 UTC m=+53.140441640 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/e6b68ed7-afa1-44b1-acf0-50c8addc4c6c-insights-runtime-extractor-tls") pod "insights-runtime-extractor-pbh67" (UID: "e6b68ed7-afa1-44b1-acf0-50c8addc4c6c") : secret "insights-runtime-extractor-tls" not found Jul 06 15:34:35.865601 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:35.865572 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-pgkmz_a3b0e3f1-74a1-4f15-9770-1a8ac2a02c4f/dns-node-resolver/0.log" Jul 06 15:34:36.648509 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:36.648470 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e6b68ed7-afa1-44b1-acf0-50c8addc4c6c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-pbh67\" (UID: \"e6b68ed7-afa1-44b1-acf0-50c8addc4c6c\") " pod="openshift-insights/insights-runtime-extractor-pbh67" Jul 06 15:34:36.648792 ip-10-0-129-154 kubenswrapper[2563]: E0706 15:34:36.648625 2563 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 06 15:34:36.648792 ip-10-0-129-154 kubenswrapper[2563]: E0706 15:34:36.648696 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e6b68ed7-afa1-44b1-acf0-50c8addc4c6c-insights-runtime-extractor-tls podName:e6b68ed7-afa1-44b1-acf0-50c8addc4c6c nodeName:}" failed. No retries permitted until 2026-07-06 15:34:40.648678098 +0000 UTC m=+57.157911249 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/e6b68ed7-afa1-44b1-acf0-50c8addc4c6c-insights-runtime-extractor-tls") pod "insights-runtime-extractor-pbh67" (UID: "e6b68ed7-afa1-44b1-acf0-50c8addc4c6c") : secret "insights-runtime-extractor-tls" not found Jul 06 15:34:36.648792 ip-10-0-129-154 kubenswrapper[2563]: E0706 15:34:36.648694 2563 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 06 15:34:36.648792 ip-10-0-129-154 kubenswrapper[2563]: E0706 15:34:36.648752 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e6b68ed7-afa1-44b1-acf0-50c8addc4c6c-kube-rbac-proxy-cm podName:e6b68ed7-afa1-44b1-acf0-50c8addc4c6c nodeName:}" failed. No retries permitted until 2026-07-06 15:34:40.648737534 +0000 UTC m=+57.157970707 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/e6b68ed7-afa1-44b1-acf0-50c8addc4c6c-kube-rbac-proxy-cm") pod "insights-runtime-extractor-pbh67" (UID: "e6b68ed7-afa1-44b1-acf0-50c8addc4c6c") : configmap "kube-rbac-proxy" not found Jul 06 15:34:36.648792 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:36.648634 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/e6b68ed7-afa1-44b1-acf0-50c8addc4c6c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-pbh67\" (UID: \"e6b68ed7-afa1-44b1-acf0-50c8addc4c6c\") " pod="openshift-insights/insights-runtime-extractor-pbh67" Jul 06 15:34:36.865993 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:36.865962 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-rmsk7_0ce018e7-3a1f-43dc-8988-5d8054f04c69/node-ca/0.log" Jul 06 15:34:37.758371 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:37.758331 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/bf5c30e2-bf88-468b-adf2-2e8f73f136b4-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-trjfs\" (UID: \"bf5c30e2-bf88-468b-adf2-2e8f73f136b4\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-trjfs" Jul 06 15:34:37.758371 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:37.758373 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/2e3ce465-22c0-4b33-83f4-8fdc24d06414-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-7prfp\" (UID: \"2e3ce465-22c0-4b33-83f4-8fdc24d06414\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-7prfp" Jul 06 15:34:37.758646 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:37.758481 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/f955a985-08b6-49d9-b159-8ec3916d8b20-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-fgktw\" (UID: \"f955a985-08b6-49d9-b159-8ec3916d8b20\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-fgktw" Jul 06 15:34:37.758646 ip-10-0-129-154 kubenswrapper[2563]: E0706 15:34:37.758510 2563 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 06 15:34:37.758646 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:37.758510 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/201d3840-f397-4577-b8ba-03b31295ed3f-service-ca-bundle\") pod \"router-default-5855858db9-884bq\" (UID: \"201d3840-f397-4577-b8ba-03b31295ed3f\") " pod="openshift-ingress/router-default-5855858db9-884bq" Jul 06 15:34:37.758646 ip-10-0-129-154 kubenswrapper[2563]: E0706 15:34:37.758518 2563 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 06 15:34:37.758646 ip-10-0-129-154 kubenswrapper[2563]: E0706 15:34:37.758587 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bf5c30e2-bf88-468b-adf2-2e8f73f136b4-cluster-monitoring-operator-tls podName:bf5c30e2-bf88-468b-adf2-2e8f73f136b4 nodeName:}" failed. No retries permitted until 2026-07-06 15:34:53.75856805 +0000 UTC m=+70.267801236 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/bf5c30e2-bf88-468b-adf2-2e8f73f136b4-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-trjfs" (UID: "bf5c30e2-bf88-468b-adf2-2e8f73f136b4") : secret "cluster-monitoring-operator-tls" not found Jul 06 15:34:37.758646 ip-10-0-129-154 kubenswrapper[2563]: E0706 15:34:37.758569 2563 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 06 15:34:37.758854 ip-10-0-129-154 kubenswrapper[2563]: E0706 15:34:37.758658 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/201d3840-f397-4577-b8ba-03b31295ed3f-service-ca-bundle podName:201d3840-f397-4577-b8ba-03b31295ed3f nodeName:}" failed. No retries permitted until 2026-07-06 15:34:53.758636987 +0000 UTC m=+70.267870158 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/201d3840-f397-4577-b8ba-03b31295ed3f-service-ca-bundle") pod "router-default-5855858db9-884bq" (UID: "201d3840-f397-4577-b8ba-03b31295ed3f") : configmap references non-existent config key: service-ca.crt Jul 06 15:34:37.758854 ip-10-0-129-154 kubenswrapper[2563]: E0706 15:34:37.758679 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2e3ce465-22c0-4b33-83f4-8fdc24d06414-samples-operator-tls podName:2e3ce465-22c0-4b33-83f4-8fdc24d06414 nodeName:}" failed. No retries permitted until 2026-07-06 15:34:53.758669459 +0000 UTC m=+70.267902626 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/2e3ce465-22c0-4b33-83f4-8fdc24d06414-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-7prfp" (UID: "2e3ce465-22c0-4b33-83f4-8fdc24d06414") : secret "samples-operator-tls" not found Jul 06 15:34:37.758854 ip-10-0-129-154 kubenswrapper[2563]: E0706 15:34:37.758695 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f955a985-08b6-49d9-b159-8ec3916d8b20-networking-console-plugin-cert podName:f955a985-08b6-49d9-b159-8ec3916d8b20 nodeName:}" failed. No retries permitted until 2026-07-06 15:34:53.758687587 +0000 UTC m=+70.267920748 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/f955a985-08b6-49d9-b159-8ec3916d8b20-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-fgktw" (UID: "f955a985-08b6-49d9-b159-8ec3916d8b20") : secret "networking-console-plugin-cert" not found Jul 06 15:34:37.758854 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:37.758772 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1911fa1c-48ce-44ab-8b64-7ae70ce80a0d-registry-tls\") pod \"image-registry-d668d48f7-cncvq\" (UID: \"1911fa1c-48ce-44ab-8b64-7ae70ce80a0d\") " pod="openshift-image-registry/image-registry-d668d48f7-cncvq" Jul 06 15:34:37.758854 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:37.758808 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/201d3840-f397-4577-b8ba-03b31295ed3f-metrics-certs\") pod \"router-default-5855858db9-884bq\" (UID: \"201d3840-f397-4577-b8ba-03b31295ed3f\") " pod="openshift-ingress/router-default-5855858db9-884bq" Jul 06 15:34:37.759039 ip-10-0-129-154 kubenswrapper[2563]: E0706 15:34:37.758907 2563 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 06 15:34:37.759039 ip-10-0-129-154 kubenswrapper[2563]: E0706 15:34:37.758924 2563 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-d668d48f7-cncvq: secret "image-registry-tls" not found Jul 06 15:34:37.759039 ip-10-0-129-154 kubenswrapper[2563]: E0706 15:34:37.758937 2563 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 06 15:34:37.759039 ip-10-0-129-154 kubenswrapper[2563]: E0706 15:34:37.758981 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1911fa1c-48ce-44ab-8b64-7ae70ce80a0d-registry-tls podName:1911fa1c-48ce-44ab-8b64-7ae70ce80a0d nodeName:}" failed. No retries permitted until 2026-07-06 15:34:53.758967342 +0000 UTC m=+70.268200513 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/1911fa1c-48ce-44ab-8b64-7ae70ce80a0d-registry-tls") pod "image-registry-d668d48f7-cncvq" (UID: "1911fa1c-48ce-44ab-8b64-7ae70ce80a0d") : secret "image-registry-tls" not found Jul 06 15:34:37.759039 ip-10-0-129-154 kubenswrapper[2563]: E0706 15:34:37.758994 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/201d3840-f397-4577-b8ba-03b31295ed3f-metrics-certs podName:201d3840-f397-4577-b8ba-03b31295ed3f nodeName:}" failed. No retries permitted until 2026-07-06 15:34:53.758988307 +0000 UTC m=+70.268221461 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/201d3840-f397-4577-b8ba-03b31295ed3f-metrics-certs") pod "router-default-5855858db9-884bq" (UID: "201d3840-f397-4577-b8ba-03b31295ed3f") : secret "router-metrics-certs-default" not found Jul 06 15:34:37.859689 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:37.859650 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/17c20e5e-6515-48d6-8c51-425a9e86485d-metrics-tls\") pod \"dns-default-h86nn\" (UID: \"17c20e5e-6515-48d6-8c51-425a9e86485d\") " pod="openshift-dns/dns-default-h86nn" Jul 06 15:34:37.859880 ip-10-0-129-154 kubenswrapper[2563]: E0706 15:34:37.859814 2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 06 15:34:37.859880 ip-10-0-129-154 kubenswrapper[2563]: E0706 15:34:37.859880 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/17c20e5e-6515-48d6-8c51-425a9e86485d-metrics-tls podName:17c20e5e-6515-48d6-8c51-425a9e86485d nodeName:}" failed. No retries permitted until 2026-07-06 15:34:53.859863442 +0000 UTC m=+70.369096597 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/17c20e5e-6515-48d6-8c51-425a9e86485d-metrics-tls") pod "dns-default-h86nn" (UID: "17c20e5e-6515-48d6-8c51-425a9e86485d") : secret "dns-default-metrics-tls" not found Jul 06 15:34:37.866431 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:37.866403 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-qltg7_5967a722-7558-4ab7-98f4-b3a5e367bd7a/serve-healthcheck-canary/0.log" Jul 06 15:34:38.468890 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:38.468862 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-846bf668f7-662qp_966f6ac8-fde8-46e3-b9fe-816b6f334efa/kube-storage-version-migrator-operator/0.log" Jul 06 15:34:40.683498 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:40.683450 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/e6b68ed7-afa1-44b1-acf0-50c8addc4c6c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-pbh67\" (UID: \"e6b68ed7-afa1-44b1-acf0-50c8addc4c6c\") " pod="openshift-insights/insights-runtime-extractor-pbh67" Jul 06 15:34:40.683895 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:40.683552 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e6b68ed7-afa1-44b1-acf0-50c8addc4c6c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-pbh67\" (UID: \"e6b68ed7-afa1-44b1-acf0-50c8addc4c6c\") " pod="openshift-insights/insights-runtime-extractor-pbh67" Jul 06 15:34:40.683895 ip-10-0-129-154 kubenswrapper[2563]: E0706 15:34:40.683569 2563 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 06 15:34:40.683895 ip-10-0-129-154 kubenswrapper[2563]: E0706 15:34:40.683629 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e6b68ed7-afa1-44b1-acf0-50c8addc4c6c-kube-rbac-proxy-cm podName:e6b68ed7-afa1-44b1-acf0-50c8addc4c6c nodeName:}" failed. No retries permitted until 2026-07-06 15:34:48.683612823 +0000 UTC m=+65.192845974 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/e6b68ed7-afa1-44b1-acf0-50c8addc4c6c-kube-rbac-proxy-cm") pod "insights-runtime-extractor-pbh67" (UID: "e6b68ed7-afa1-44b1-acf0-50c8addc4c6c") : configmap "kube-rbac-proxy" not found Jul 06 15:34:40.683895 ip-10-0-129-154 kubenswrapper[2563]: E0706 15:34:40.683708 2563 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 06 15:34:40.683895 ip-10-0-129-154 kubenswrapper[2563]: E0706 15:34:40.683756 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e6b68ed7-afa1-44b1-acf0-50c8addc4c6c-insights-runtime-extractor-tls podName:e6b68ed7-afa1-44b1-acf0-50c8addc4c6c nodeName:}" failed. No retries permitted until 2026-07-06 15:34:48.683743267 +0000 UTC m=+65.192976422 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/e6b68ed7-afa1-44b1-acf0-50c8addc4c6c-insights-runtime-extractor-tls") pod "insights-runtime-extractor-pbh67" (UID: "e6b68ed7-afa1-44b1-acf0-50c8addc4c6c") : secret "insights-runtime-extractor-tls" not found Jul 06 15:34:47.118033 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:47.118007 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-tv82v" Jul 06 15:34:47.940144 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:47.940109 2563 scope.go:117] "RemoveContainer" containerID="a81099975cdcb5e7127528428fba030cd7fb5b0a69da693709c895b24d4ac2b3" Jul 06 15:34:48.250886 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:48.250815 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-l47bf_9ad7e968-ed2e-4cbe-9b4b-780bc9d01b88/console-operator/2.log" Jul 06 15:34:48.251285 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:48.251250 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-l47bf_9ad7e968-ed2e-4cbe-9b4b-780bc9d01b88/console-operator/1.log" Jul 06 15:34:48.251285 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:48.251280 2563 generic.go:356] "Generic (PLEG): container finished" podID="9ad7e968-ed2e-4cbe-9b4b-780bc9d01b88" containerID="480484ec82135bd625d8d09c98421660c0d1254a0c803ce88adeccc150df00ee" exitCode=255 Jul 06 15:34:48.251362 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:48.251339 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-l47bf" event={"ID":"9ad7e968-ed2e-4cbe-9b4b-780bc9d01b88","Type":"ContainerDied","Data":"480484ec82135bd625d8d09c98421660c0d1254a0c803ce88adeccc150df00ee"} Jul 06 15:34:48.251395 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:48.251385 2563 scope.go:117] "RemoveContainer" containerID="a81099975cdcb5e7127528428fba030cd7fb5b0a69da693709c895b24d4ac2b3" Jul 06 15:34:48.251755 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:48.251735 2563 scope.go:117] "RemoveContainer" containerID="480484ec82135bd625d8d09c98421660c0d1254a0c803ce88adeccc150df00ee" Jul 06 15:34:48.251935 ip-10-0-129-154 kubenswrapper[2563]: E0706 15:34:48.251913 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-68b757fb67-l47bf_openshift-console-operator(9ad7e968-ed2e-4cbe-9b4b-780bc9d01b88)\"" pod="openshift-console-operator/console-operator-68b757fb67-l47bf" podUID="9ad7e968-ed2e-4cbe-9b4b-780bc9d01b88" Jul 06 15:34:48.756305 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:48.756265 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/e6b68ed7-afa1-44b1-acf0-50c8addc4c6c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-pbh67\" (UID: \"e6b68ed7-afa1-44b1-acf0-50c8addc4c6c\") " pod="openshift-insights/insights-runtime-extractor-pbh67" Jul 06 15:34:48.756506 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:48.756337 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e6b68ed7-afa1-44b1-acf0-50c8addc4c6c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-pbh67\" (UID: \"e6b68ed7-afa1-44b1-acf0-50c8addc4c6c\") " pod="openshift-insights/insights-runtime-extractor-pbh67" Jul 06 15:34:48.756558 ip-10-0-129-154 kubenswrapper[2563]: E0706 15:34:48.756515 2563 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 06 15:34:48.756607 ip-10-0-129-154 kubenswrapper[2563]: E0706 15:34:48.756595 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e6b68ed7-afa1-44b1-acf0-50c8addc4c6c-insights-runtime-extractor-tls podName:e6b68ed7-afa1-44b1-acf0-50c8addc4c6c nodeName:}" failed. No retries permitted until 2026-07-06 15:35:04.756573862 +0000 UTC m=+81.265807031 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/e6b68ed7-afa1-44b1-acf0-50c8addc4c6c-insights-runtime-extractor-tls") pod "insights-runtime-extractor-pbh67" (UID: "e6b68ed7-afa1-44b1-acf0-50c8addc4c6c") : secret "insights-runtime-extractor-tls" not found Jul 06 15:34:48.756901 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:48.756882 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/e6b68ed7-afa1-44b1-acf0-50c8addc4c6c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-pbh67\" (UID: \"e6b68ed7-afa1-44b1-acf0-50c8addc4c6c\") " pod="openshift-insights/insights-runtime-extractor-pbh67" Jul 06 15:34:49.256225 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:49.256198 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-l47bf_9ad7e968-ed2e-4cbe-9b4b-780bc9d01b88/console-operator/2.log" Jul 06 15:34:49.765030 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:49.764996 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/bce194c9-b363-4021-bb8a-c7cc65facc39-original-pull-secret\") pod \"global-pull-secret-syncer-w9qbt\" (UID: \"bce194c9-b363-4021-bb8a-c7cc65facc39\") " pod="kube-system/global-pull-secret-syncer-w9qbt" Jul 06 15:34:49.765202 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:49.765113 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5771d1de-202e-4920-b0de-5d12a4e4e2fc-metrics-certs\") pod \"network-metrics-daemon-v8fxz\" (UID: \"5771d1de-202e-4920-b0de-5d12a4e4e2fc\") " pod="openshift-multus/network-metrics-daemon-v8fxz" Jul 06 15:34:49.768485 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:49.768469 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 06 15:34:49.769703 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:49.769688 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 06 15:34:49.777771 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:49.777743 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/bce194c9-b363-4021-bb8a-c7cc65facc39-original-pull-secret\") pod \"global-pull-secret-syncer-w9qbt\" (UID: \"bce194c9-b363-4021-bb8a-c7cc65facc39\") " pod="kube-system/global-pull-secret-syncer-w9qbt" Jul 06 15:34:49.777895 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:49.777877 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5771d1de-202e-4920-b0de-5d12a4e4e2fc-metrics-certs\") pod \"network-metrics-daemon-v8fxz\" (UID: \"5771d1de-202e-4920-b0de-5d12a4e4e2fc\") " pod="openshift-multus/network-metrics-daemon-v8fxz" Jul 06 15:34:49.866536 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:49.866506 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-twj8p\" (UniqueName: \"kubernetes.io/projected/e6761760-82ef-43c3-99f0-9e027d7c859c-kube-api-access-twj8p\") pod \"network-check-target-pdw4n\" (UID: \"e6761760-82ef-43c3-99f0-9e027d7c859c\") " pod="openshift-network-diagnostics/network-check-target-pdw4n" Jul 06 15:34:49.869032 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:49.869012 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-twj8p\" (UniqueName: \"kubernetes.io/projected/e6761760-82ef-43c3-99f0-9e027d7c859c-kube-api-access-twj8p\") pod \"network-check-target-pdw4n\" (UID: \"e6761760-82ef-43c3-99f0-9e027d7c859c\") " pod="openshift-network-diagnostics/network-check-target-pdw4n" Jul 06 15:34:49.955837 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:49.955812 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-v99lt\"" Jul 06 15:34:49.962473 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:49.962455 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-v8fxz" Jul 06 15:34:49.963000 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:49.962986 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-w485s\"" Jul 06 15:34:49.963050 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:49.963036 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w9qbt" Jul 06 15:34:49.969202 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:49.969186 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pdw4n" Jul 06 15:34:50.120170 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:50.120144 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-pdw4n"] Jul 06 15:34:50.122954 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:34:50.122928 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode6761760_82ef_43c3_99f0_9e027d7c859c.slice/crio-6666286bdf1bee629ae1bd011262a3cb903df0be4090f2cbfcb5433b3cb76f15 WatchSource:0}: Error finding container 6666286bdf1bee629ae1bd011262a3cb903df0be4090f2cbfcb5433b3cb76f15: Status 404 returned error can't find the container with id 6666286bdf1bee629ae1bd011262a3cb903df0be4090f2cbfcb5433b3cb76f15 Jul 06 15:34:50.261399 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:50.261364 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-pdw4n" event={"ID":"e6761760-82ef-43c3-99f0-9e027d7c859c","Type":"ContainerStarted","Data":"3a319e4c1a794c8bcf3be263e3ac291d32ce68a165fc6d447e40f9c806f5eada"} Jul 06 15:34:50.261785 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:50.261413 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-pdw4n" event={"ID":"e6761760-82ef-43c3-99f0-9e027d7c859c","Type":"ContainerStarted","Data":"6666286bdf1bee629ae1bd011262a3cb903df0be4090f2cbfcb5433b3cb76f15"} Jul 06 15:34:50.261785 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:50.261481 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-pdw4n" Jul 06 15:34:50.282232 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:50.282147 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-pdw4n" podStartSLOduration=66.282133744 podStartE2EDuration="1m6.282133744s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 15:33:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 15:34:50.281900475 +0000 UTC m=+66.791133651" watchObservedRunningTime="2026-07-06 15:34:50.282133744 +0000 UTC m=+66.791366932" Jul 06 15:34:50.349818 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:50.349782 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-w9qbt"] Jul 06 15:34:50.352538 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:34:50.352514 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbce194c9_b363_4021_bb8a_c7cc65facc39.slice/crio-2c3b1e85c90e65faf6b1a5987f4aaecd8fea2bd24642681110cba3482cbe6fa4 WatchSource:0}: Error finding container 2c3b1e85c90e65faf6b1a5987f4aaecd8fea2bd24642681110cba3482cbe6fa4: Status 404 returned error can't find the container with id 2c3b1e85c90e65faf6b1a5987f4aaecd8fea2bd24642681110cba3482cbe6fa4 Jul 06 15:34:50.355902 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:50.355853 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-v8fxz"] Jul 06 15:34:50.357743 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:34:50.357719 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5771d1de_202e_4920_b0de_5d12a4e4e2fc.slice/crio-743faf7e51d33db25932d0fc4c0610c5181843adcd88d4f8edd7204e30b75038 WatchSource:0}: Error finding container 743faf7e51d33db25932d0fc4c0610c5181843adcd88d4f8edd7204e30b75038: Status 404 returned error can't find the container with id 743faf7e51d33db25932d0fc4c0610c5181843adcd88d4f8edd7204e30b75038 Jul 06 15:34:51.266250 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:51.266210 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-w9qbt" event={"ID":"bce194c9-b363-4021-bb8a-c7cc65facc39","Type":"ContainerStarted","Data":"2c3b1e85c90e65faf6b1a5987f4aaecd8fea2bd24642681110cba3482cbe6fa4"} Jul 06 15:34:51.267833 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:51.267777 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-v8fxz" event={"ID":"5771d1de-202e-4920-b0de-5d12a4e4e2fc","Type":"ContainerStarted","Data":"743faf7e51d33db25932d0fc4c0610c5181843adcd88d4f8edd7204e30b75038"} Jul 06 15:34:52.255791 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:52.255754 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-68b757fb67-l47bf" Jul 06 15:34:52.255964 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:52.255810 2563 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-68b757fb67-l47bf" Jul 06 15:34:52.256127 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:52.256106 2563 scope.go:117] "RemoveContainer" containerID="480484ec82135bd625d8d09c98421660c0d1254a0c803ce88adeccc150df00ee" Jul 06 15:34:52.256301 ip-10-0-129-154 kubenswrapper[2563]: E0706 15:34:52.256281 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-68b757fb67-l47bf_openshift-console-operator(9ad7e968-ed2e-4cbe-9b4b-780bc9d01b88)\"" pod="openshift-console-operator/console-operator-68b757fb67-l47bf" podUID="9ad7e968-ed2e-4cbe-9b4b-780bc9d01b88" Jul 06 15:34:52.272377 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:52.272347 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-v8fxz" event={"ID":"5771d1de-202e-4920-b0de-5d12a4e4e2fc","Type":"ContainerStarted","Data":"0d8894e0c4ff8b474d95e785d34d8de568299b18b04f5124a0866a3ed82c0dcd"} Jul 06 15:34:52.272830 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:52.272395 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-v8fxz" event={"ID":"5771d1de-202e-4920-b0de-5d12a4e4e2fc","Type":"ContainerStarted","Data":"8c9f4d7da598c48fc4dd08a6c0ba224aecdf02301f502f13a8c520163500046a"} Jul 06 15:34:52.316122 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:52.316067 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-v8fxz" podStartSLOduration=67.151255018 podStartE2EDuration="1m8.316045433s" totalImagesPullingTime="1.164790415s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 15:33:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 15:34:52.315019374 +0000 UTC m=+68.824252550" watchObservedRunningTime="2026-07-06 15:34:52.316045433 +0000 UTC m=+68.825278607" Jul 06 15:34:53.804350 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:53.804315 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/f955a985-08b6-49d9-b159-8ec3916d8b20-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-fgktw\" (UID: \"f955a985-08b6-49d9-b159-8ec3916d8b20\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-fgktw" Jul 06 15:34:53.804350 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:53.804360 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/201d3840-f397-4577-b8ba-03b31295ed3f-service-ca-bundle\") pod \"router-default-5855858db9-884bq\" (UID: \"201d3840-f397-4577-b8ba-03b31295ed3f\") " pod="openshift-ingress/router-default-5855858db9-884bq" Jul 06 15:34:53.804951 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:53.804476 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1911fa1c-48ce-44ab-8b64-7ae70ce80a0d-registry-tls\") pod \"image-registry-d668d48f7-cncvq\" (UID: \"1911fa1c-48ce-44ab-8b64-7ae70ce80a0d\") " pod="openshift-image-registry/image-registry-d668d48f7-cncvq" Jul 06 15:34:53.804951 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:53.804528 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/201d3840-f397-4577-b8ba-03b31295ed3f-metrics-certs\") pod \"router-default-5855858db9-884bq\" (UID: \"201d3840-f397-4577-b8ba-03b31295ed3f\") " pod="openshift-ingress/router-default-5855858db9-884bq" Jul 06 15:34:53.804951 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:53.804586 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/bf5c30e2-bf88-468b-adf2-2e8f73f136b4-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-trjfs\" (UID: \"bf5c30e2-bf88-468b-adf2-2e8f73f136b4\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-trjfs" Jul 06 15:34:53.804951 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:53.804618 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/2e3ce465-22c0-4b33-83f4-8fdc24d06414-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-7prfp\" (UID: \"2e3ce465-22c0-4b33-83f4-8fdc24d06414\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-7prfp" Jul 06 15:34:53.805157 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:53.805124 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/201d3840-f397-4577-b8ba-03b31295ed3f-service-ca-bundle\") pod \"router-default-5855858db9-884bq\" (UID: \"201d3840-f397-4577-b8ba-03b31295ed3f\") " pod="openshift-ingress/router-default-5855858db9-884bq" Jul 06 15:34:53.807126 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:53.807093 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/bf5c30e2-bf88-468b-adf2-2e8f73f136b4-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-trjfs\" (UID: \"bf5c30e2-bf88-468b-adf2-2e8f73f136b4\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-trjfs" Jul 06 15:34:53.807579 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:53.807561 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1911fa1c-48ce-44ab-8b64-7ae70ce80a0d-registry-tls\") pod \"image-registry-d668d48f7-cncvq\" (UID: \"1911fa1c-48ce-44ab-8b64-7ae70ce80a0d\") " pod="openshift-image-registry/image-registry-d668d48f7-cncvq" Jul 06 15:34:53.807667 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:53.807606 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/201d3840-f397-4577-b8ba-03b31295ed3f-metrics-certs\") pod \"router-default-5855858db9-884bq\" (UID: \"201d3840-f397-4577-b8ba-03b31295ed3f\") " pod="openshift-ingress/router-default-5855858db9-884bq" Jul 06 15:34:53.807667 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:53.807623 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/f955a985-08b6-49d9-b159-8ec3916d8b20-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-fgktw\" (UID: \"f955a985-08b6-49d9-b159-8ec3916d8b20\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-fgktw" Jul 06 15:34:53.807667 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:53.807625 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/2e3ce465-22c0-4b33-83f4-8fdc24d06414-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-7prfp\" (UID: \"2e3ce465-22c0-4b33-83f4-8fdc24d06414\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-7prfp" Jul 06 15:34:53.905950 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:53.905916 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/17c20e5e-6515-48d6-8c51-425a9e86485d-metrics-tls\") pod \"dns-default-h86nn\" (UID: \"17c20e5e-6515-48d6-8c51-425a9e86485d\") " pod="openshift-dns/dns-default-h86nn" Jul 06 15:34:53.908586 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:53.908559 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/17c20e5e-6515-48d6-8c51-425a9e86485d-metrics-tls\") pod \"dns-default-h86nn\" (UID: \"17c20e5e-6515-48d6-8c51-425a9e86485d\") " pod="openshift-dns/dns-default-h86nn" Jul 06 15:34:53.969290 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:53.969259 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-98ctb\"" Jul 06 15:34:53.976376 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:53.976348 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-d668d48f7-cncvq" Jul 06 15:34:53.976884 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:53.976852 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-x692s\"" Jul 06 15:34:53.984059 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:53.984036 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-5855858db9-884bq" Jul 06 15:34:54.002252 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:54.001818 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-7jvt2\"" Jul 06 15:34:54.011446 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:54.008230 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-trjfs" Jul 06 15:34:54.038967 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:54.038820 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-lvkj8\"" Jul 06 15:34:54.048511 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:54.048399 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-7prfp" Jul 06 15:34:54.069618 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:54.069582 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-z6cjr\"" Jul 06 15:34:54.075475 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:54.075190 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6bffdf9778-fgktw" Jul 06 15:34:54.090035 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:54.089806 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-8w9vb\"" Jul 06 15:34:54.097221 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:54.096952 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-h86nn" Jul 06 15:34:54.178352 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:54.178209 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-d668d48f7-cncvq"] Jul 06 15:34:54.216911 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:54.216818 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-5855858db9-884bq"] Jul 06 15:34:54.255725 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:54.255514 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-64786d9cbb-trjfs"] Jul 06 15:34:54.283843 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:54.283769 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-trjfs" event={"ID":"bf5c30e2-bf88-468b-adf2-2e8f73f136b4","Type":"ContainerStarted","Data":"545cf546b9eaa5b443654e01dc9c2b2406667eed7406e467dc267b9ddf00f800"} Jul 06 15:34:54.285714 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:54.285682 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-5855858db9-884bq" event={"ID":"201d3840-f397-4577-b8ba-03b31295ed3f","Type":"ContainerStarted","Data":"0fca957b4214deae59ce9e94dd2eceac2d9029f77d15be5a540b63424c6c1c2d"} Jul 06 15:34:54.289074 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:54.288994 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-d668d48f7-cncvq" event={"ID":"1911fa1c-48ce-44ab-8b64-7ae70ce80a0d","Type":"ContainerStarted","Data":"1fa93229d8c438423dcde9915280961afb79833fbe88fd71b69ded7d7c453d07"} Jul 06 15:34:54.301033 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:54.301007 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-7prfp"] Jul 06 15:34:54.321708 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:54.321616 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6bffdf9778-fgktw"] Jul 06 15:34:54.326452 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:34:54.326344 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf955a985_08b6_49d9_b159_8ec3916d8b20.slice/crio-6d2f201911d8a619b630f76c245ca2317ba240bc94b0914c19d72746b06824fd WatchSource:0}: Error finding container 6d2f201911d8a619b630f76c245ca2317ba240bc94b0914c19d72746b06824fd: Status 404 returned error can't find the container with id 6d2f201911d8a619b630f76c245ca2317ba240bc94b0914c19d72746b06824fd Jul 06 15:34:54.339634 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:54.339614 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-h86nn"] Jul 06 15:34:54.344116 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:34:54.344085 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod17c20e5e_6515_48d6_8c51_425a9e86485d.slice/crio-5b538a710dba0ded0ee8bd5af816ce6c9c9ebfc8c863f2277b441d398587dc0f WatchSource:0}: Error finding container 5b538a710dba0ded0ee8bd5af816ce6c9c9ebfc8c863f2277b441d398587dc0f: Status 404 returned error can't find the container with id 5b538a710dba0ded0ee8bd5af816ce6c9c9ebfc8c863f2277b441d398587dc0f Jul 06 15:34:55.298794 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:55.298631 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-d668d48f7-cncvq" event={"ID":"1911fa1c-48ce-44ab-8b64-7ae70ce80a0d","Type":"ContainerStarted","Data":"5578602aacd91466dd03f773d33f7690d2421d91505213f790978053b14a929c"} Jul 06 15:34:55.298794 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:55.298736 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-d668d48f7-cncvq" Jul 06 15:34:55.300172 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:55.300138 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-h86nn" event={"ID":"17c20e5e-6515-48d6-8c51-425a9e86485d","Type":"ContainerStarted","Data":"5b538a710dba0ded0ee8bd5af816ce6c9c9ebfc8c863f2277b441d398587dc0f"} Jul 06 15:34:55.301704 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:55.301677 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6bffdf9778-fgktw" event={"ID":"f955a985-08b6-49d9-b159-8ec3916d8b20","Type":"ContainerStarted","Data":"6d2f201911d8a619b630f76c245ca2317ba240bc94b0914c19d72746b06824fd"} Jul 06 15:34:55.303146 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:55.303088 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-7prfp" event={"ID":"2e3ce465-22c0-4b33-83f4-8fdc24d06414","Type":"ContainerStarted","Data":"e90eb322792f30a3eebe45455468ea734e07e692ba78194b0280deb07794ac94"} Jul 06 15:34:55.305105 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:55.305081 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-5855858db9-884bq" event={"ID":"201d3840-f397-4577-b8ba-03b31295ed3f","Type":"ContainerStarted","Data":"8d41100ef70c7f0d4b49f01c2c6e8e3438e3405de6ed2efdbcc06e060ab98a9c"} Jul 06 15:34:55.332903 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:55.332855 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-d668d48f7-cncvq" podStartSLOduration=71.332838659 podStartE2EDuration="1m11.332838659s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 15:33:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 15:34:55.331843388 +0000 UTC m=+71.841076565" watchObservedRunningTime="2026-07-06 15:34:55.332838659 +0000 UTC m=+71.842071833" Jul 06 15:34:55.369724 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:55.369672 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-5855858db9-884bq" podStartSLOduration=44.369656604 podStartE2EDuration="44.369656604s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 15:34:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 15:34:55.366644791 +0000 UTC m=+71.875877965" watchObservedRunningTime="2026-07-06 15:34:55.369656604 +0000 UTC m=+71.878889792" Jul 06 15:34:55.984273 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:55.984236 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-5855858db9-884bq" Jul 06 15:34:55.987095 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:55.987055 2563 patch_prober.go:28] interesting pod/router-default-5855858db9-884bq container/router namespace/openshift-ingress: Startup probe status=failure output="HTTP probe failed with statuscode: 500" start-of-body=[-]backend-proxy-http failed: reason withheld Jul 06 15:34:55.987095 ip-10-0-129-154 kubenswrapper[2563]: [-]has-synced failed: reason withheld Jul 06 15:34:55.987095 ip-10-0-129-154 kubenswrapper[2563]: [+]process-running ok Jul 06 15:34:55.987095 ip-10-0-129-154 kubenswrapper[2563]: healthz check failed Jul 06 15:34:55.987407 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:55.987117 2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/router-default-5855858db9-884bq" podUID="201d3840-f397-4577-b8ba-03b31295ed3f" containerName="router" probeResult="failure" output="HTTP probe failed with statuscode: 500" Jul 06 15:34:56.985648 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:56.985603 2563 patch_prober.go:28] interesting pod/router-default-5855858db9-884bq container/router namespace/openshift-ingress: Startup probe status=failure output="HTTP probe failed with statuscode: 500" start-of-body=[-]backend-proxy-http failed: reason withheld Jul 06 15:34:56.985648 ip-10-0-129-154 kubenswrapper[2563]: [-]has-synced failed: reason withheld Jul 06 15:34:56.985648 ip-10-0-129-154 kubenswrapper[2563]: [+]process-running ok Jul 06 15:34:56.985648 ip-10-0-129-154 kubenswrapper[2563]: healthz check failed Jul 06 15:34:56.986262 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:56.985672 2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/router-default-5855858db9-884bq" podUID="201d3840-f397-4577-b8ba-03b31295ed3f" containerName="router" probeResult="failure" output="HTTP probe failed with statuscode: 500" Jul 06 15:34:57.985719 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:57.985681 2563 patch_prober.go:28] interesting pod/router-default-5855858db9-884bq container/router namespace/openshift-ingress: Startup probe status=failure output="HTTP probe failed with statuscode: 500" start-of-body=[-]backend-proxy-http failed: reason withheld Jul 06 15:34:57.985719 ip-10-0-129-154 kubenswrapper[2563]: [-]has-synced failed: reason withheld Jul 06 15:34:57.985719 ip-10-0-129-154 kubenswrapper[2563]: [+]process-running ok Jul 06 15:34:57.985719 ip-10-0-129-154 kubenswrapper[2563]: healthz check failed Jul 06 15:34:57.986179 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:57.985742 2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/router-default-5855858db9-884bq" podUID="201d3840-f397-4577-b8ba-03b31295ed3f" containerName="router" probeResult="failure" output="HTTP probe failed with statuscode: 500" Jul 06 15:34:58.985660 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:58.985627 2563 patch_prober.go:28] interesting pod/router-default-5855858db9-884bq container/router namespace/openshift-ingress: Startup probe status=failure output="HTTP probe failed with statuscode: 500" start-of-body=[-]backend-proxy-http failed: reason withheld Jul 06 15:34:58.985660 ip-10-0-129-154 kubenswrapper[2563]: [-]has-synced failed: reason withheld Jul 06 15:34:58.985660 ip-10-0-129-154 kubenswrapper[2563]: [+]process-running ok Jul 06 15:34:58.985660 ip-10-0-129-154 kubenswrapper[2563]: healthz check failed Jul 06 15:34:58.986244 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:58.985684 2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/router-default-5855858db9-884bq" podUID="201d3840-f397-4577-b8ba-03b31295ed3f" containerName="router" probeResult="failure" output="HTTP probe failed with statuscode: 500" Jul 06 15:34:59.985816 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:59.985783 2563 patch_prober.go:28] interesting pod/router-default-5855858db9-884bq container/router namespace/openshift-ingress: Startup probe status=failure output="HTTP probe failed with statuscode: 500" start-of-body=[-]backend-proxy-http failed: reason withheld Jul 06 15:34:59.985816 ip-10-0-129-154 kubenswrapper[2563]: [-]has-synced failed: reason withheld Jul 06 15:34:59.985816 ip-10-0-129-154 kubenswrapper[2563]: [+]process-running ok Jul 06 15:34:59.985816 ip-10-0-129-154 kubenswrapper[2563]: healthz check failed Jul 06 15:34:59.986265 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:34:59.985845 2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/router-default-5855858db9-884bq" podUID="201d3840-f397-4577-b8ba-03b31295ed3f" containerName="router" probeResult="failure" output="HTTP probe failed with statuscode: 500" Jul 06 15:35:00.228380 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:00.228338 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-2bbvr"] Jul 06 15:35:00.243706 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:00.243675 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-2bbvr" Jul 06 15:35:00.243864 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:00.243725 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-2bbvr"] Jul 06 15:35:00.247798 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:00.247775 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-8njr5\"" Jul 06 15:35:00.247924 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:00.247774 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\"" Jul 06 15:35:00.324479 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:00.324403 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-w9qbt" event={"ID":"bce194c9-b363-4021-bb8a-c7cc65facc39","Type":"ContainerStarted","Data":"e0ae351696e3ff753731a491a48bbbe5feb7c795210a486af27014142195776a"} Jul 06 15:35:00.326043 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:00.326015 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-trjfs" event={"ID":"bf5c30e2-bf88-468b-adf2-2e8f73f136b4","Type":"ContainerStarted","Data":"add20e8910ca680a19132b6a7ed6646ecc2030e5d29eabfee4be049f730d47ae"} Jul 06 15:35:00.328645 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:00.328103 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-h86nn" event={"ID":"17c20e5e-6515-48d6-8c51-425a9e86485d","Type":"ContainerStarted","Data":"2d49e5285e68c97f97a31849caff709eb019dcea39aca220cd82819e587d9e9e"} Jul 06 15:35:00.328645 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:00.328149 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-h86nn" event={"ID":"17c20e5e-6515-48d6-8c51-425a9e86485d","Type":"ContainerStarted","Data":"7052d901119096975dd6bf81ea6e9f01debd1182948390d3706c19e43c8da9ef"} Jul 06 15:35:00.328645 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:00.328350 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-h86nn" Jul 06 15:35:00.330754 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:00.330727 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6bffdf9778-fgktw" event={"ID":"f955a985-08b6-49d9-b159-8ec3916d8b20","Type":"ContainerStarted","Data":"e009007a5de9e50ce746db45e760291be02ca7e89e110f14710fce520e7f7dab"} Jul 06 15:35:00.332899 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:00.332878 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-7prfp" event={"ID":"2e3ce465-22c0-4b33-83f4-8fdc24d06414","Type":"ContainerStarted","Data":"91e2ab01dd41f35d613aad1346917b9c889db8a2aeddcab46c4c2a776d344389"} Jul 06 15:35:00.332976 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:00.332923 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-7prfp" event={"ID":"2e3ce465-22c0-4b33-83f4-8fdc24d06414","Type":"ContainerStarted","Data":"cfeb725408fc46a075dc8d5fc796d5d0c160dba80149fe5d119b464ca4d78f7a"} Jul 06 15:35:00.347790 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:00.347749 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-w9qbt" podStartSLOduration=67.489411089 podStartE2EDuration="1m16.347736866s" totalImagesPullingTime="8.858325777s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 15:33:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 15:35:00.345503453 +0000 UTC m=+76.854736628" watchObservedRunningTime="2026-07-06 15:35:00.347736866 +0000 UTC m=+76.856970017" Jul 06 15:35:00.368209 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:00.368184 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/e5f102a2-585e-4fc2-9579-2c221f542f83-tls-certificates\") pod \"prometheus-operator-admission-webhook-d559d66d5-2bbvr\" (UID: \"e5f102a2-585e-4fc2-9579-2c221f542f83\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-2bbvr" Jul 06 15:35:00.375909 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:00.375864 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-7prfp" podStartSLOduration=44.196546237 podStartE2EDuration="49.375847749s" totalImagesPullingTime="5.179301512s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 15:34:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 15:35:00.374763938 +0000 UTC m=+76.883997111" watchObservedRunningTime="2026-07-06 15:35:00.375847749 +0000 UTC m=+76.885080913" Jul 06 15:35:00.418827 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:00.418783 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-h86nn" podStartSLOduration=34.21789894 podStartE2EDuration="39.4187649s" totalImagesPullingTime="5.20086596s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 15:34:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 15:35:00.417921241 +0000 UTC m=+76.927154415" watchObservedRunningTime="2026-07-06 15:35:00.4187649 +0000 UTC m=+76.927998076" Jul 06 15:35:00.448820 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:00.448772 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-6bffdf9778-fgktw" podStartSLOduration=37.228217832 podStartE2EDuration="42.448749682s" totalImagesPullingTime="5.22053185s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 15:34:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 15:35:00.447461885 +0000 UTC m=+76.956695058" watchObservedRunningTime="2026-07-06 15:35:00.448749682 +0000 UTC m=+76.957982855" Jul 06 15:35:00.469609 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:00.469581 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/e5f102a2-585e-4fc2-9579-2c221f542f83-tls-certificates\") pod \"prometheus-operator-admission-webhook-d559d66d5-2bbvr\" (UID: \"e5f102a2-585e-4fc2-9579-2c221f542f83\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-2bbvr" Jul 06 15:35:00.472924 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:00.472895 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/e5f102a2-585e-4fc2-9579-2c221f542f83-tls-certificates\") pod \"prometheus-operator-admission-webhook-d559d66d5-2bbvr\" (UID: \"e5f102a2-585e-4fc2-9579-2c221f542f83\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-2bbvr" Jul 06 15:35:00.553599 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:00.553518 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-2bbvr" Jul 06 15:35:00.698738 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:00.698687 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-trjfs" podStartSLOduration=44.7505676 podStartE2EDuration="49.698663584s" totalImagesPullingTime="4.948095984s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 15:34:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 15:35:00.480795112 +0000 UTC m=+76.990028285" watchObservedRunningTime="2026-07-06 15:35:00.698663584 +0000 UTC m=+77.207896759" Jul 06 15:35:00.699578 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:00.699560 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-2bbvr"] Jul 06 15:35:00.702747 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:35:00.702718 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode5f102a2_585e_4fc2_9579_2c221f542f83.slice/crio-f5823ba6a7e3465f471606982cb9b65a326d17d32ac18d0ef41c5a8d2ec1040f WatchSource:0}: Error finding container f5823ba6a7e3465f471606982cb9b65a326d17d32ac18d0ef41c5a8d2ec1040f: Status 404 returned error can't find the container with id f5823ba6a7e3465f471606982cb9b65a326d17d32ac18d0ef41c5a8d2ec1040f Jul 06 15:35:00.985584 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:00.985550 2563 patch_prober.go:28] interesting pod/router-default-5855858db9-884bq container/router namespace/openshift-ingress: Startup probe status=failure output="HTTP probe failed with statuscode: 500" start-of-body=[-]backend-proxy-http failed: reason withheld Jul 06 15:35:00.985584 ip-10-0-129-154 kubenswrapper[2563]: [-]has-synced failed: reason withheld Jul 06 15:35:00.985584 ip-10-0-129-154 kubenswrapper[2563]: [+]process-running ok Jul 06 15:35:00.985584 ip-10-0-129-154 kubenswrapper[2563]: healthz check failed Jul 06 15:35:00.985920 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:00.985607 2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/router-default-5855858db9-884bq" podUID="201d3840-f397-4577-b8ba-03b31295ed3f" containerName="router" probeResult="failure" output="HTTP probe failed with statuscode: 500" Jul 06 15:35:01.343264 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:01.343162 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-2bbvr" event={"ID":"e5f102a2-585e-4fc2-9579-2c221f542f83","Type":"ContainerStarted","Data":"f5823ba6a7e3465f471606982cb9b65a326d17d32ac18d0ef41c5a8d2ec1040f"} Jul 06 15:35:01.985624 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:01.985592 2563 patch_prober.go:28] interesting pod/router-default-5855858db9-884bq container/router namespace/openshift-ingress: Startup probe status=failure output="HTTP probe failed with statuscode: 500" start-of-body=[-]backend-proxy-http failed: reason withheld Jul 06 15:35:01.985624 ip-10-0-129-154 kubenswrapper[2563]: [-]has-synced failed: reason withheld Jul 06 15:35:01.985624 ip-10-0-129-154 kubenswrapper[2563]: [+]process-running ok Jul 06 15:35:01.985624 ip-10-0-129-154 kubenswrapper[2563]: healthz check failed Jul 06 15:35:01.985848 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:01.985650 2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/router-default-5855858db9-884bq" podUID="201d3840-f397-4577-b8ba-03b31295ed3f" containerName="router" probeResult="failure" output="HTTP probe failed with statuscode: 500" Jul 06 15:35:02.985801 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:02.985768 2563 patch_prober.go:28] interesting pod/router-default-5855858db9-884bq container/router namespace/openshift-ingress: Startup probe status=failure output="HTTP probe failed with statuscode: 500" start-of-body=[-]backend-proxy-http failed: reason withheld Jul 06 15:35:02.985801 ip-10-0-129-154 kubenswrapper[2563]: [-]has-synced failed: reason withheld Jul 06 15:35:02.985801 ip-10-0-129-154 kubenswrapper[2563]: [+]process-running ok Jul 06 15:35:02.985801 ip-10-0-129-154 kubenswrapper[2563]: healthz check failed Jul 06 15:35:02.986281 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:02.985829 2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/router-default-5855858db9-884bq" podUID="201d3840-f397-4577-b8ba-03b31295ed3f" containerName="router" probeResult="failure" output="HTTP probe failed with statuscode: 500" Jul 06 15:35:03.353463 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:03.353357 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-2bbvr" event={"ID":"e5f102a2-585e-4fc2-9579-2c221f542f83","Type":"ContainerStarted","Data":"335f1986972225aa8b038946c7e15cd4c0028210a26dc0170bdbdf92b703fa2c"} Jul 06 15:35:03.353612 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:03.353573 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-2bbvr" Jul 06 15:35:03.358168 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:03.358145 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-2bbvr" Jul 06 15:35:03.381433 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:03.381371 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-2bbvr" podStartSLOduration=1.744138182 podStartE2EDuration="3.381355174s" totalImagesPullingTime="1.637216992s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 15:35:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 15:35:03.38047668 +0000 UTC m=+79.889709855" watchObservedRunningTime="2026-07-06 15:35:03.381355174 +0000 UTC m=+79.890588348" Jul 06 15:35:03.941412 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:03.941383 2563 scope.go:117] "RemoveContainer" containerID="480484ec82135bd625d8d09c98421660c0d1254a0c803ce88adeccc150df00ee" Jul 06 15:35:03.941590 ip-10-0-129-154 kubenswrapper[2563]: E0706 15:35:03.941552 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-68b757fb67-l47bf_openshift-console-operator(9ad7e968-ed2e-4cbe-9b4b-780bc9d01b88)\"" pod="openshift-console-operator/console-operator-68b757fb67-l47bf" podUID="9ad7e968-ed2e-4cbe-9b4b-780bc9d01b88" Jul 06 15:35:03.984622 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:03.984589 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-5855858db9-884bq" Jul 06 15:35:03.985772 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:03.985751 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-5855858db9-884bq" Jul 06 15:35:04.358021 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:04.357941 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-5855858db9-884bq" Jul 06 15:35:04.439313 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:04.439280 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-567f7b8b7c-b7s4w"] Jul 06 15:35:04.456132 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:04.456100 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-567f7b8b7c-b7s4w"] Jul 06 15:35:04.456283 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:04.456218 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-b7s4w" Jul 06 15:35:04.460237 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:04.460210 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-k7xq9\"" Jul 06 15:35:04.460549 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:04.460532 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\"" Jul 06 15:35:04.460649 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:04.460586 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\"" Jul 06 15:35:04.460827 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:04.460806 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 06 15:35:04.605076 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:04.605033 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/eeb3e868-088a-4b4d-80a1-143a15ad91c8-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-567f7b8b7c-b7s4w\" (UID: \"eeb3e868-088a-4b4d-80a1-143a15ad91c8\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-b7s4w" Jul 06 15:35:04.605076 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:04.605078 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/eeb3e868-088a-4b4d-80a1-143a15ad91c8-prometheus-operator-tls\") pod \"prometheus-operator-567f7b8b7c-b7s4w\" (UID: \"eeb3e868-088a-4b4d-80a1-143a15ad91c8\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-b7s4w" Jul 06 15:35:04.605275 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:04.605101 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/eeb3e868-088a-4b4d-80a1-143a15ad91c8-metrics-client-ca\") pod \"prometheus-operator-567f7b8b7c-b7s4w\" (UID: \"eeb3e868-088a-4b4d-80a1-143a15ad91c8\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-b7s4w" Jul 06 15:35:04.605275 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:04.605184 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qdrw9\" (UniqueName: \"kubernetes.io/projected/eeb3e868-088a-4b4d-80a1-143a15ad91c8-kube-api-access-qdrw9\") pod \"prometheus-operator-567f7b8b7c-b7s4w\" (UID: \"eeb3e868-088a-4b4d-80a1-143a15ad91c8\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-b7s4w" Jul 06 15:35:04.705995 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:04.705868 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qdrw9\" (UniqueName: \"kubernetes.io/projected/eeb3e868-088a-4b4d-80a1-143a15ad91c8-kube-api-access-qdrw9\") pod \"prometheus-operator-567f7b8b7c-b7s4w\" (UID: \"eeb3e868-088a-4b4d-80a1-143a15ad91c8\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-b7s4w" Jul 06 15:35:04.705995 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:04.705945 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/eeb3e868-088a-4b4d-80a1-143a15ad91c8-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-567f7b8b7c-b7s4w\" (UID: \"eeb3e868-088a-4b4d-80a1-143a15ad91c8\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-b7s4w" Jul 06 15:35:04.705995 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:04.705975 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/eeb3e868-088a-4b4d-80a1-143a15ad91c8-prometheus-operator-tls\") pod \"prometheus-operator-567f7b8b7c-b7s4w\" (UID: \"eeb3e868-088a-4b4d-80a1-143a15ad91c8\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-b7s4w" Jul 06 15:35:04.705995 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:04.705991 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/eeb3e868-088a-4b4d-80a1-143a15ad91c8-metrics-client-ca\") pod \"prometheus-operator-567f7b8b7c-b7s4w\" (UID: \"eeb3e868-088a-4b4d-80a1-143a15ad91c8\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-b7s4w" Jul 06 15:35:04.706686 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:04.706665 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/eeb3e868-088a-4b4d-80a1-143a15ad91c8-metrics-client-ca\") pod \"prometheus-operator-567f7b8b7c-b7s4w\" (UID: \"eeb3e868-088a-4b4d-80a1-143a15ad91c8\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-b7s4w" Jul 06 15:35:04.708537 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:04.708499 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/eeb3e868-088a-4b4d-80a1-143a15ad91c8-prometheus-operator-tls\") pod \"prometheus-operator-567f7b8b7c-b7s4w\" (UID: \"eeb3e868-088a-4b4d-80a1-143a15ad91c8\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-b7s4w" Jul 06 15:35:04.708643 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:04.708619 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/eeb3e868-088a-4b4d-80a1-143a15ad91c8-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-567f7b8b7c-b7s4w\" (UID: \"eeb3e868-088a-4b4d-80a1-143a15ad91c8\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-b7s4w" Jul 06 15:35:04.715516 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:04.715490 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qdrw9\" (UniqueName: \"kubernetes.io/projected/eeb3e868-088a-4b4d-80a1-143a15ad91c8-kube-api-access-qdrw9\") pod \"prometheus-operator-567f7b8b7c-b7s4w\" (UID: \"eeb3e868-088a-4b4d-80a1-143a15ad91c8\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-b7s4w" Jul 06 15:35:04.772550 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:04.772512 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-b7s4w" Jul 06 15:35:04.806992 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:04.806962 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e6b68ed7-afa1-44b1-acf0-50c8addc4c6c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-pbh67\" (UID: \"e6b68ed7-afa1-44b1-acf0-50c8addc4c6c\") " pod="openshift-insights/insights-runtime-extractor-pbh67" Jul 06 15:35:04.809209 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:04.809187 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e6b68ed7-afa1-44b1-acf0-50c8addc4c6c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-pbh67\" (UID: \"e6b68ed7-afa1-44b1-acf0-50c8addc4c6c\") " pod="openshift-insights/insights-runtime-extractor-pbh67" Jul 06 15:35:04.922216 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:04.922189 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-567f7b8b7c-b7s4w"] Jul 06 15:35:04.924298 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:35:04.924274 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeeb3e868_088a_4b4d_80a1_143a15ad91c8.slice/crio-78a7ecd7cd87b8ca8c8f7f0cec66da68b8965ab2041eb7dec322f8abb0106d09 WatchSource:0}: Error finding container 78a7ecd7cd87b8ca8c8f7f0cec66da68b8965ab2041eb7dec322f8abb0106d09: Status 404 returned error can't find the container with id 78a7ecd7cd87b8ca8c8f7f0cec66da68b8965ab2041eb7dec322f8abb0106d09 Jul 06 15:35:05.106387 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:05.106313 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-5rtrq\"" Jul 06 15:35:05.113990 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:05.113960 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-pbh67" Jul 06 15:35:05.280716 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:05.280690 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-pbh67"] Jul 06 15:35:05.283179 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:35:05.283148 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode6b68ed7_afa1_44b1_acf0_50c8addc4c6c.slice/crio-7909726292f8a4e8e59cb7913eddb639f193d7f56a1e02e07ff0fdb0e7a2686a WatchSource:0}: Error finding container 7909726292f8a4e8e59cb7913eddb639f193d7f56a1e02e07ff0fdb0e7a2686a: Status 404 returned error can't find the container with id 7909726292f8a4e8e59cb7913eddb639f193d7f56a1e02e07ff0fdb0e7a2686a Jul 06 15:35:05.361098 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:05.361005 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-b7s4w" event={"ID":"eeb3e868-088a-4b4d-80a1-143a15ad91c8","Type":"ContainerStarted","Data":"78a7ecd7cd87b8ca8c8f7f0cec66da68b8965ab2041eb7dec322f8abb0106d09"} Jul 06 15:35:05.362035 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:05.362006 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-pbh67" event={"ID":"e6b68ed7-afa1-44b1-acf0-50c8addc4c6c","Type":"ContainerStarted","Data":"7909726292f8a4e8e59cb7913eddb639f193d7f56a1e02e07ff0fdb0e7a2686a"} Jul 06 15:35:07.373231 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:07.373194 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-b7s4w" event={"ID":"eeb3e868-088a-4b4d-80a1-143a15ad91c8","Type":"ContainerStarted","Data":"cbb475441b0bfa275bf084967bf97bb5cbdc7341cd6b54dea5d4eff880aba5b6"} Jul 06 15:35:07.373708 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:07.373254 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-b7s4w" event={"ID":"eeb3e868-088a-4b4d-80a1-143a15ad91c8","Type":"ContainerStarted","Data":"93ac496ea6ca2ad1740267b3cb01778e1efd7e0f025f1261ec7419aaeb1c63c5"} Jul 06 15:35:07.398305 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:07.398223 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-b7s4w" podStartSLOduration=1.625726065 podStartE2EDuration="3.398202169s" totalImagesPullingTime="1.772476104s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 15:35:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 15:35:07.397230041 +0000 UTC m=+83.906463215" watchObservedRunningTime="2026-07-06 15:35:07.398202169 +0000 UTC m=+83.907435345" Jul 06 15:35:09.385884 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:09.385852 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-pbh67" event={"ID":"e6b68ed7-afa1-44b1-acf0-50c8addc4c6c","Type":"ContainerStarted","Data":"3ea9f9bad645d2fc8c09e36398f00ae77a0f00ab19013b6b5f7c08bac50b0ca2"} Jul 06 15:35:10.346624 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:10.346552 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-h86nn" Jul 06 15:35:10.392122 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:10.392086 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-pbh67" event={"ID":"e6b68ed7-afa1-44b1-acf0-50c8addc4c6c","Type":"ContainerStarted","Data":"278b419124bc7eb47b94fcf1ec5e98c25296b4d681a0fc04ce0a24805908d493"} Jul 06 15:35:10.394172 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:10.394152 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-64786d9cbb-trjfs_bf5c30e2-bf88-468b-adf2-2e8f73f136b4/cluster-monitoring-operator/0.log" Jul 06 15:35:10.394314 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:10.394192 2563 generic.go:356] "Generic (PLEG): container finished" podID="bf5c30e2-bf88-468b-adf2-2e8f73f136b4" containerID="add20e8910ca680a19132b6a7ed6646ecc2030e5d29eabfee4be049f730d47ae" exitCode=2 Jul 06 15:35:10.394314 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:10.394238 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-trjfs" event={"ID":"bf5c30e2-bf88-468b-adf2-2e8f73f136b4","Type":"ContainerDied","Data":"add20e8910ca680a19132b6a7ed6646ecc2030e5d29eabfee4be049f730d47ae"} Jul 06 15:35:10.394620 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:10.394598 2563 scope.go:117] "RemoveContainer" containerID="add20e8910ca680a19132b6a7ed6646ecc2030e5d29eabfee4be049f730d47ae" Jul 06 15:35:11.401007 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:11.400980 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-64786d9cbb-trjfs_bf5c30e2-bf88-468b-adf2-2e8f73f136b4/cluster-monitoring-operator/0.log" Jul 06 15:35:11.401448 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:11.401030 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-trjfs" event={"ID":"bf5c30e2-bf88-468b-adf2-2e8f73f136b4","Type":"ContainerStarted","Data":"5c0d10ac6a6bcb9d5eea9ef051d2d5963bc91656279cce54d95439ce2383d074"} Jul 06 15:35:13.409834 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:13.409798 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-pbh67" event={"ID":"e6b68ed7-afa1-44b1-acf0-50c8addc4c6c","Type":"ContainerStarted","Data":"9b36940cd0cd796c5f158a9507ef302ce11786a02fe607b494a65b2eb1013650"} Jul 06 15:35:13.435213 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:13.435162 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-pbh67" podStartSLOduration=34.239409837 podStartE2EDuration="41.435143979s" totalImagesPullingTime="7.195734142s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 15:34:32 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 15:35:13.43431479 +0000 UTC m=+89.943547965" watchObservedRunningTime="2026-07-06 15:35:13.435143979 +0000 UTC m=+89.944377152" Jul 06 15:35:14.270466 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:14.270430 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-sjxxv"] Jul 06 15:35:14.274051 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:14.274032 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-sjxxv" Jul 06 15:35:14.278051 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:14.278027 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 06 15:35:14.278343 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:14.278327 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 06 15:35:14.278442 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:14.278364 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-7t9r2\"" Jul 06 15:35:14.278442 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:14.278402 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 06 15:35:14.387640 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:14.387498 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c55d32ae-8d34-4ca2-8d07-a3a374ecf12d-metrics-client-ca\") pod \"node-exporter-sjxxv\" (UID: \"c55d32ae-8d34-4ca2-8d07-a3a374ecf12d\") " pod="openshift-monitoring/node-exporter-sjxxv" Jul 06 15:35:14.387640 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:14.387553 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c55d32ae-8d34-4ca2-8d07-a3a374ecf12d-sys\") pod \"node-exporter-sjxxv\" (UID: \"c55d32ae-8d34-4ca2-8d07-a3a374ecf12d\") " pod="openshift-monitoring/node-exporter-sjxxv" Jul 06 15:35:14.387640 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:14.387611 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/c55d32ae-8d34-4ca2-8d07-a3a374ecf12d-node-exporter-textfile\") pod \"node-exporter-sjxxv\" (UID: \"c55d32ae-8d34-4ca2-8d07-a3a374ecf12d\") " pod="openshift-monitoring/node-exporter-sjxxv" Jul 06 15:35:14.387874 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:14.387685 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c55d32ae-8d34-4ca2-8d07-a3a374ecf12d-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-sjxxv\" (UID: \"c55d32ae-8d34-4ca2-8d07-a3a374ecf12d\") " pod="openshift-monitoring/node-exporter-sjxxv" Jul 06 15:35:14.387874 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:14.387764 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/c55d32ae-8d34-4ca2-8d07-a3a374ecf12d-root\") pod \"node-exporter-sjxxv\" (UID: \"c55d32ae-8d34-4ca2-8d07-a3a374ecf12d\") " pod="openshift-monitoring/node-exporter-sjxxv" Jul 06 15:35:14.387874 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:14.387793 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/c55d32ae-8d34-4ca2-8d07-a3a374ecf12d-node-exporter-wtmp\") pod \"node-exporter-sjxxv\" (UID: \"c55d32ae-8d34-4ca2-8d07-a3a374ecf12d\") " pod="openshift-monitoring/node-exporter-sjxxv" Jul 06 15:35:14.387874 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:14.387819 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/c55d32ae-8d34-4ca2-8d07-a3a374ecf12d-node-exporter-accelerators-collector-config\") pod \"node-exporter-sjxxv\" (UID: \"c55d32ae-8d34-4ca2-8d07-a3a374ecf12d\") " pod="openshift-monitoring/node-exporter-sjxxv" Jul 06 15:35:14.388010 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:14.387876 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zjmfc\" (UniqueName: \"kubernetes.io/projected/c55d32ae-8d34-4ca2-8d07-a3a374ecf12d-kube-api-access-zjmfc\") pod \"node-exporter-sjxxv\" (UID: \"c55d32ae-8d34-4ca2-8d07-a3a374ecf12d\") " pod="openshift-monitoring/node-exporter-sjxxv" Jul 06 15:35:14.388010 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:14.387898 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/c55d32ae-8d34-4ca2-8d07-a3a374ecf12d-node-exporter-tls\") pod \"node-exporter-sjxxv\" (UID: \"c55d32ae-8d34-4ca2-8d07-a3a374ecf12d\") " pod="openshift-monitoring/node-exporter-sjxxv" Jul 06 15:35:14.488928 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:14.488896 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c55d32ae-8d34-4ca2-8d07-a3a374ecf12d-sys\") pod \"node-exporter-sjxxv\" (UID: \"c55d32ae-8d34-4ca2-8d07-a3a374ecf12d\") " pod="openshift-monitoring/node-exporter-sjxxv" Jul 06 15:35:14.488928 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:14.488928 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/c55d32ae-8d34-4ca2-8d07-a3a374ecf12d-node-exporter-textfile\") pod \"node-exporter-sjxxv\" (UID: \"c55d32ae-8d34-4ca2-8d07-a3a374ecf12d\") " pod="openshift-monitoring/node-exporter-sjxxv" Jul 06 15:35:14.489392 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:14.488947 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c55d32ae-8d34-4ca2-8d07-a3a374ecf12d-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-sjxxv\" (UID: \"c55d32ae-8d34-4ca2-8d07-a3a374ecf12d\") " pod="openshift-monitoring/node-exporter-sjxxv" Jul 06 15:35:14.489392 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:14.488976 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/c55d32ae-8d34-4ca2-8d07-a3a374ecf12d-root\") pod \"node-exporter-sjxxv\" (UID: \"c55d32ae-8d34-4ca2-8d07-a3a374ecf12d\") " pod="openshift-monitoring/node-exporter-sjxxv" Jul 06 15:35:14.489392 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:14.488998 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/c55d32ae-8d34-4ca2-8d07-a3a374ecf12d-node-exporter-wtmp\") pod \"node-exporter-sjxxv\" (UID: \"c55d32ae-8d34-4ca2-8d07-a3a374ecf12d\") " pod="openshift-monitoring/node-exporter-sjxxv" Jul 06 15:35:14.489392 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:14.489018 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c55d32ae-8d34-4ca2-8d07-a3a374ecf12d-sys\") pod \"node-exporter-sjxxv\" (UID: \"c55d32ae-8d34-4ca2-8d07-a3a374ecf12d\") " pod="openshift-monitoring/node-exporter-sjxxv" Jul 06 15:35:14.489392 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:14.489028 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/c55d32ae-8d34-4ca2-8d07-a3a374ecf12d-node-exporter-accelerators-collector-config\") pod \"node-exporter-sjxxv\" (UID: \"c55d32ae-8d34-4ca2-8d07-a3a374ecf12d\") " pod="openshift-monitoring/node-exporter-sjxxv" Jul 06 15:35:14.489392 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:14.489054 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/c55d32ae-8d34-4ca2-8d07-a3a374ecf12d-root\") pod \"node-exporter-sjxxv\" (UID: \"c55d32ae-8d34-4ca2-8d07-a3a374ecf12d\") " pod="openshift-monitoring/node-exporter-sjxxv" Jul 06 15:35:14.489392 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:14.489119 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zjmfc\" (UniqueName: \"kubernetes.io/projected/c55d32ae-8d34-4ca2-8d07-a3a374ecf12d-kube-api-access-zjmfc\") pod \"node-exporter-sjxxv\" (UID: \"c55d32ae-8d34-4ca2-8d07-a3a374ecf12d\") " pod="openshift-monitoring/node-exporter-sjxxv" Jul 06 15:35:14.489392 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:14.489154 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/c55d32ae-8d34-4ca2-8d07-a3a374ecf12d-node-exporter-tls\") pod \"node-exporter-sjxxv\" (UID: \"c55d32ae-8d34-4ca2-8d07-a3a374ecf12d\") " pod="openshift-monitoring/node-exporter-sjxxv" Jul 06 15:35:14.489392 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:14.489188 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c55d32ae-8d34-4ca2-8d07-a3a374ecf12d-metrics-client-ca\") pod \"node-exporter-sjxxv\" (UID: \"c55d32ae-8d34-4ca2-8d07-a3a374ecf12d\") " pod="openshift-monitoring/node-exporter-sjxxv" Jul 06 15:35:14.489392 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:14.489257 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/c55d32ae-8d34-4ca2-8d07-a3a374ecf12d-node-exporter-wtmp\") pod \"node-exporter-sjxxv\" (UID: \"c55d32ae-8d34-4ca2-8d07-a3a374ecf12d\") " pod="openshift-monitoring/node-exporter-sjxxv" Jul 06 15:35:14.489392 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:14.489308 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/c55d32ae-8d34-4ca2-8d07-a3a374ecf12d-node-exporter-textfile\") pod \"node-exporter-sjxxv\" (UID: \"c55d32ae-8d34-4ca2-8d07-a3a374ecf12d\") " pod="openshift-monitoring/node-exporter-sjxxv" Jul 06 15:35:14.489819 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:14.489740 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c55d32ae-8d34-4ca2-8d07-a3a374ecf12d-metrics-client-ca\") pod \"node-exporter-sjxxv\" (UID: \"c55d32ae-8d34-4ca2-8d07-a3a374ecf12d\") " pod="openshift-monitoring/node-exporter-sjxxv" Jul 06 15:35:14.489819 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:14.489743 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/c55d32ae-8d34-4ca2-8d07-a3a374ecf12d-node-exporter-accelerators-collector-config\") pod \"node-exporter-sjxxv\" (UID: \"c55d32ae-8d34-4ca2-8d07-a3a374ecf12d\") " pod="openshift-monitoring/node-exporter-sjxxv" Jul 06 15:35:14.491480 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:14.491460 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c55d32ae-8d34-4ca2-8d07-a3a374ecf12d-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-sjxxv\" (UID: \"c55d32ae-8d34-4ca2-8d07-a3a374ecf12d\") " pod="openshift-monitoring/node-exporter-sjxxv" Jul 06 15:35:14.491579 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:14.491542 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/c55d32ae-8d34-4ca2-8d07-a3a374ecf12d-node-exporter-tls\") pod \"node-exporter-sjxxv\" (UID: \"c55d32ae-8d34-4ca2-8d07-a3a374ecf12d\") " pod="openshift-monitoring/node-exporter-sjxxv" Jul 06 15:35:14.504334 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:14.504308 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zjmfc\" (UniqueName: \"kubernetes.io/projected/c55d32ae-8d34-4ca2-8d07-a3a374ecf12d-kube-api-access-zjmfc\") pod \"node-exporter-sjxxv\" (UID: \"c55d32ae-8d34-4ca2-8d07-a3a374ecf12d\") " pod="openshift-monitoring/node-exporter-sjxxv" Jul 06 15:35:14.583060 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:14.582988 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-sjxxv" Jul 06 15:35:14.597906 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:35:14.597877 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc55d32ae_8d34_4ca2_8d07_a3a374ecf12d.slice/crio-e1181cbbedd07fb1a2dbaee7d84e52e41890a85e7f760b9edaf505f5af0cfeb4 WatchSource:0}: Error finding container e1181cbbedd07fb1a2dbaee7d84e52e41890a85e7f760b9edaf505f5af0cfeb4: Status 404 returned error can't find the container with id e1181cbbedd07fb1a2dbaee7d84e52e41890a85e7f760b9edaf505f5af0cfeb4 Jul 06 15:35:15.418893 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:15.418861 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-sjxxv" event={"ID":"c55d32ae-8d34-4ca2-8d07-a3a374ecf12d","Type":"ContainerStarted","Data":"e1181cbbedd07fb1a2dbaee7d84e52e41890a85e7f760b9edaf505f5af0cfeb4"} Jul 06 15:35:16.313175 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:16.313146 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-d668d48f7-cncvq" Jul 06 15:35:16.424817 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:16.424785 2563 generic.go:356] "Generic (PLEG): container finished" podID="c55d32ae-8d34-4ca2-8d07-a3a374ecf12d" containerID="d044d6e13d22c9e1af8d1b3bbd72c95a37a2d38559fa07d1349f4c2e1976df76" exitCode=0 Jul 06 15:35:16.424967 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:16.424840 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-sjxxv" event={"ID":"c55d32ae-8d34-4ca2-8d07-a3a374ecf12d","Type":"ContainerDied","Data":"d044d6e13d22c9e1af8d1b3bbd72c95a37a2d38559fa07d1349f4c2e1976df76"} Jul 06 15:35:17.429672 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:17.429644 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-54dc4c49bb-mj926"] Jul 06 15:35:17.433530 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:17.433504 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-sjxxv" event={"ID":"c55d32ae-8d34-4ca2-8d07-a3a374ecf12d","Type":"ContainerStarted","Data":"76a9f406bca21c665c16d373dd96131b9a93c8253f4ec9582f10fada7b1f4e9d"} Jul 06 15:35:17.433667 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:17.433535 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-sjxxv" event={"ID":"c55d32ae-8d34-4ca2-8d07-a3a374ecf12d","Type":"ContainerStarted","Data":"4daadd7b39faa17d47bd17d1b84e1fc60cc50f942daf63192b8e485524fda379"} Jul 06 15:35:17.433667 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:17.433618 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-54dc4c49bb-mj926" Jul 06 15:35:17.440140 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:17.440111 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\"" Jul 06 15:35:17.440259 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:17.440188 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\"" Jul 06 15:35:17.443455 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:17.443431 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\"" Jul 06 15:35:17.443455 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:17.443413 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\"" Jul 06 15:35:17.443631 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:17.443525 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-b7fv5\"" Jul 06 15:35:17.443631 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:17.443560 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-cv8pctk3kh5ds\"" Jul 06 15:35:17.447639 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:17.447620 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\"" Jul 06 15:35:17.469094 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:17.469068 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-54dc4c49bb-mj926"] Jul 06 15:35:17.472816 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:17.472775 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-sjxxv" podStartSLOduration=2.252864709 podStartE2EDuration="3.472761655s" totalImagesPullingTime="1.146968422s" totalInitContainerRuntime="72.928524ms" isStatefulPod=true podCreationTimestamp="2026-07-06 15:35:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 15:35:17.472382608 +0000 UTC m=+93.981615789" watchObservedRunningTime="2026-07-06 15:35:17.472761655 +0000 UTC m=+93.981994829" Jul 06 15:35:17.614794 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:17.614759 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2sdd6\" (UniqueName: \"kubernetes.io/projected/206bbf71-dc24-4a17-bae2-8d17e7488a47-kube-api-access-2sdd6\") pod \"thanos-querier-54dc4c49bb-mj926\" (UID: \"206bbf71-dc24-4a17-bae2-8d17e7488a47\") " pod="openshift-monitoring/thanos-querier-54dc4c49bb-mj926" Jul 06 15:35:17.614959 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:17.614809 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/206bbf71-dc24-4a17-bae2-8d17e7488a47-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-54dc4c49bb-mj926\" (UID: \"206bbf71-dc24-4a17-bae2-8d17e7488a47\") " pod="openshift-monitoring/thanos-querier-54dc4c49bb-mj926" Jul 06 15:35:17.614959 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:17.614860 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/206bbf71-dc24-4a17-bae2-8d17e7488a47-metrics-client-ca\") pod \"thanos-querier-54dc4c49bb-mj926\" (UID: \"206bbf71-dc24-4a17-bae2-8d17e7488a47\") " pod="openshift-monitoring/thanos-querier-54dc4c49bb-mj926" Jul 06 15:35:17.614959 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:17.614915 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/206bbf71-dc24-4a17-bae2-8d17e7488a47-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-54dc4c49bb-mj926\" (UID: \"206bbf71-dc24-4a17-bae2-8d17e7488a47\") " pod="openshift-monitoring/thanos-querier-54dc4c49bb-mj926" Jul 06 15:35:17.614959 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:17.614935 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/206bbf71-dc24-4a17-bae2-8d17e7488a47-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-54dc4c49bb-mj926\" (UID: \"206bbf71-dc24-4a17-bae2-8d17e7488a47\") " pod="openshift-monitoring/thanos-querier-54dc4c49bb-mj926" Jul 06 15:35:17.615119 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:17.614961 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/206bbf71-dc24-4a17-bae2-8d17e7488a47-secret-thanos-querier-tls\") pod \"thanos-querier-54dc4c49bb-mj926\" (UID: \"206bbf71-dc24-4a17-bae2-8d17e7488a47\") " pod="openshift-monitoring/thanos-querier-54dc4c49bb-mj926" Jul 06 15:35:17.615119 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:17.614977 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/206bbf71-dc24-4a17-bae2-8d17e7488a47-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-54dc4c49bb-mj926\" (UID: \"206bbf71-dc24-4a17-bae2-8d17e7488a47\") " pod="openshift-monitoring/thanos-querier-54dc4c49bb-mj926" Jul 06 15:35:17.615119 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:17.615018 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/206bbf71-dc24-4a17-bae2-8d17e7488a47-secret-grpc-tls\") pod \"thanos-querier-54dc4c49bb-mj926\" (UID: \"206bbf71-dc24-4a17-bae2-8d17e7488a47\") " pod="openshift-monitoring/thanos-querier-54dc4c49bb-mj926" Jul 06 15:35:17.716202 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:17.716127 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2sdd6\" (UniqueName: \"kubernetes.io/projected/206bbf71-dc24-4a17-bae2-8d17e7488a47-kube-api-access-2sdd6\") pod \"thanos-querier-54dc4c49bb-mj926\" (UID: \"206bbf71-dc24-4a17-bae2-8d17e7488a47\") " pod="openshift-monitoring/thanos-querier-54dc4c49bb-mj926" Jul 06 15:35:17.716202 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:17.716183 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/206bbf71-dc24-4a17-bae2-8d17e7488a47-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-54dc4c49bb-mj926\" (UID: \"206bbf71-dc24-4a17-bae2-8d17e7488a47\") " pod="openshift-monitoring/thanos-querier-54dc4c49bb-mj926" Jul 06 15:35:17.716443 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:17.716213 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/206bbf71-dc24-4a17-bae2-8d17e7488a47-metrics-client-ca\") pod \"thanos-querier-54dc4c49bb-mj926\" (UID: \"206bbf71-dc24-4a17-bae2-8d17e7488a47\") " pod="openshift-monitoring/thanos-querier-54dc4c49bb-mj926" Jul 06 15:35:17.716443 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:17.716260 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/206bbf71-dc24-4a17-bae2-8d17e7488a47-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-54dc4c49bb-mj926\" (UID: \"206bbf71-dc24-4a17-bae2-8d17e7488a47\") " pod="openshift-monitoring/thanos-querier-54dc4c49bb-mj926" Jul 06 15:35:17.716443 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:17.716287 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/206bbf71-dc24-4a17-bae2-8d17e7488a47-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-54dc4c49bb-mj926\" (UID: \"206bbf71-dc24-4a17-bae2-8d17e7488a47\") " pod="openshift-monitoring/thanos-querier-54dc4c49bb-mj926" Jul 06 15:35:17.716443 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:17.716319 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/206bbf71-dc24-4a17-bae2-8d17e7488a47-secret-thanos-querier-tls\") pod \"thanos-querier-54dc4c49bb-mj926\" (UID: \"206bbf71-dc24-4a17-bae2-8d17e7488a47\") " pod="openshift-monitoring/thanos-querier-54dc4c49bb-mj926" Jul 06 15:35:17.716443 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:17.716343 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/206bbf71-dc24-4a17-bae2-8d17e7488a47-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-54dc4c49bb-mj926\" (UID: \"206bbf71-dc24-4a17-bae2-8d17e7488a47\") " pod="openshift-monitoring/thanos-querier-54dc4c49bb-mj926" Jul 06 15:35:17.716443 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:17.716394 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/206bbf71-dc24-4a17-bae2-8d17e7488a47-secret-grpc-tls\") pod \"thanos-querier-54dc4c49bb-mj926\" (UID: \"206bbf71-dc24-4a17-bae2-8d17e7488a47\") " pod="openshift-monitoring/thanos-querier-54dc4c49bb-mj926" Jul 06 15:35:17.717107 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:17.717083 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/206bbf71-dc24-4a17-bae2-8d17e7488a47-metrics-client-ca\") pod \"thanos-querier-54dc4c49bb-mj926\" (UID: \"206bbf71-dc24-4a17-bae2-8d17e7488a47\") " pod="openshift-monitoring/thanos-querier-54dc4c49bb-mj926" Jul 06 15:35:17.719525 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:17.719492 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/206bbf71-dc24-4a17-bae2-8d17e7488a47-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-54dc4c49bb-mj926\" (UID: \"206bbf71-dc24-4a17-bae2-8d17e7488a47\") " pod="openshift-monitoring/thanos-querier-54dc4c49bb-mj926" Jul 06 15:35:17.719628 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:17.719496 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/206bbf71-dc24-4a17-bae2-8d17e7488a47-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-54dc4c49bb-mj926\" (UID: \"206bbf71-dc24-4a17-bae2-8d17e7488a47\") " pod="openshift-monitoring/thanos-querier-54dc4c49bb-mj926" Jul 06 15:35:17.719628 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:17.719537 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/206bbf71-dc24-4a17-bae2-8d17e7488a47-secret-thanos-querier-tls\") pod \"thanos-querier-54dc4c49bb-mj926\" (UID: \"206bbf71-dc24-4a17-bae2-8d17e7488a47\") " pod="openshift-monitoring/thanos-querier-54dc4c49bb-mj926" Jul 06 15:35:17.719700 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:17.719674 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/206bbf71-dc24-4a17-bae2-8d17e7488a47-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-54dc4c49bb-mj926\" (UID: \"206bbf71-dc24-4a17-bae2-8d17e7488a47\") " pod="openshift-monitoring/thanos-querier-54dc4c49bb-mj926" Jul 06 15:35:17.719734 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:17.719702 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/206bbf71-dc24-4a17-bae2-8d17e7488a47-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-54dc4c49bb-mj926\" (UID: \"206bbf71-dc24-4a17-bae2-8d17e7488a47\") " pod="openshift-monitoring/thanos-querier-54dc4c49bb-mj926" Jul 06 15:35:17.719734 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:17.719726 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/206bbf71-dc24-4a17-bae2-8d17e7488a47-secret-grpc-tls\") pod \"thanos-querier-54dc4c49bb-mj926\" (UID: \"206bbf71-dc24-4a17-bae2-8d17e7488a47\") " pod="openshift-monitoring/thanos-querier-54dc4c49bb-mj926" Jul 06 15:35:17.723978 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:17.723956 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2sdd6\" (UniqueName: \"kubernetes.io/projected/206bbf71-dc24-4a17-bae2-8d17e7488a47-kube-api-access-2sdd6\") pod \"thanos-querier-54dc4c49bb-mj926\" (UID: \"206bbf71-dc24-4a17-bae2-8d17e7488a47\") " pod="openshift-monitoring/thanos-querier-54dc4c49bb-mj926" Jul 06 15:35:17.744017 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:17.743992 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-54dc4c49bb-mj926" Jul 06 15:35:17.897818 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:17.897789 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-54dc4c49bb-mj926"] Jul 06 15:35:17.899117 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:35:17.899088 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod206bbf71_dc24_4a17_bae2_8d17e7488a47.slice/crio-b1ff42d05b25e0151ad957f7afd867b259cfe2d58d8907c2f127486fee42c4e6 WatchSource:0}: Error finding container b1ff42d05b25e0151ad957f7afd867b259cfe2d58d8907c2f127486fee42c4e6: Status 404 returned error can't find the container with id b1ff42d05b25e0151ad957f7afd867b259cfe2d58d8907c2f127486fee42c4e6 Jul 06 15:35:17.940030 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:17.940010 2563 scope.go:117] "RemoveContainer" containerID="480484ec82135bd625d8d09c98421660c0d1254a0c803ce88adeccc150df00ee" Jul 06 15:35:18.017397 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:18.017361 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-d668d48f7-cncvq"] Jul 06 15:35:18.437885 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:18.437859 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-l47bf_9ad7e968-ed2e-4cbe-9b4b-780bc9d01b88/console-operator/2.log" Jul 06 15:35:18.438302 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:18.437967 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-l47bf" event={"ID":"9ad7e968-ed2e-4cbe-9b4b-780bc9d01b88","Type":"ContainerStarted","Data":"d25830b235e691ed67f172c1d62fe73e515ad648ff2610dd3d51778d7366d969"} Jul 06 15:35:18.438302 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:18.438284 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-68b757fb67-l47bf" Jul 06 15:35:18.439305 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:18.439279 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-54dc4c49bb-mj926" event={"ID":"206bbf71-dc24-4a17-bae2-8d17e7488a47","Type":"ContainerStarted","Data":"b1ff42d05b25e0151ad957f7afd867b259cfe2d58d8907c2f127486fee42c4e6"} Jul 06 15:35:18.460249 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:18.460199 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-68b757fb67-l47bf" podStartSLOduration=59.637318776 podStartE2EDuration="1m7.460183921s" totalImagesPullingTime="7.822865145s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 15:34:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 15:35:18.459808904 +0000 UTC m=+94.969042080" watchObservedRunningTime="2026-07-06 15:35:18.460183921 +0000 UTC m=+94.969417094" Jul 06 15:35:18.656067 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:18.656017 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-8465bdc94b-6kxjv"] Jul 06 15:35:18.660754 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:18.660729 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-8465bdc94b-6kxjv" Jul 06 15:35:18.664032 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:18.663819 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-ed9c02dthk6ut\"" Jul 06 15:35:18.664032 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:18.663917 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\"" Jul 06 15:35:18.664217 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:18.664095 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-w6h7g\"" Jul 06 15:35:18.664646 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:18.664621 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\"" Jul 06 15:35:18.664787 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:18.664735 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jul 06 15:35:18.665242 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:18.665222 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\"" Jul 06 15:35:18.673818 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:18.673793 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-8465bdc94b-6kxjv"] Jul 06 15:35:18.727047 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:18.726947 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-czfgf\" (UniqueName: \"kubernetes.io/projected/82494c64-e789-4829-8017-c44bd404e188-kube-api-access-czfgf\") pod \"metrics-server-8465bdc94b-6kxjv\" (UID: \"82494c64-e789-4829-8017-c44bd404e188\") " pod="openshift-monitoring/metrics-server-8465bdc94b-6kxjv" Jul 06 15:35:18.727047 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:18.726996 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/82494c64-e789-4829-8017-c44bd404e188-metrics-server-audit-profiles\") pod \"metrics-server-8465bdc94b-6kxjv\" (UID: \"82494c64-e789-4829-8017-c44bd404e188\") " pod="openshift-monitoring/metrics-server-8465bdc94b-6kxjv" Jul 06 15:35:18.727047 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:18.727033 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/82494c64-e789-4829-8017-c44bd404e188-secret-metrics-server-client-certs\") pod \"metrics-server-8465bdc94b-6kxjv\" (UID: \"82494c64-e789-4829-8017-c44bd404e188\") " pod="openshift-monitoring/metrics-server-8465bdc94b-6kxjv" Jul 06 15:35:18.727399 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:18.727106 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/82494c64-e789-4829-8017-c44bd404e188-client-ca-bundle\") pod \"metrics-server-8465bdc94b-6kxjv\" (UID: \"82494c64-e789-4829-8017-c44bd404e188\") " pod="openshift-monitoring/metrics-server-8465bdc94b-6kxjv" Jul 06 15:35:18.727399 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:18.727134 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/82494c64-e789-4829-8017-c44bd404e188-secret-metrics-server-tls\") pod \"metrics-server-8465bdc94b-6kxjv\" (UID: \"82494c64-e789-4829-8017-c44bd404e188\") " pod="openshift-monitoring/metrics-server-8465bdc94b-6kxjv" Jul 06 15:35:18.727399 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:18.727165 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/82494c64-e789-4829-8017-c44bd404e188-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-8465bdc94b-6kxjv\" (UID: \"82494c64-e789-4829-8017-c44bd404e188\") " pod="openshift-monitoring/metrics-server-8465bdc94b-6kxjv" Jul 06 15:35:18.727399 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:18.727188 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/82494c64-e789-4829-8017-c44bd404e188-audit-log\") pod \"metrics-server-8465bdc94b-6kxjv\" (UID: \"82494c64-e789-4829-8017-c44bd404e188\") " pod="openshift-monitoring/metrics-server-8465bdc94b-6kxjv" Jul 06 15:35:18.828552 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:18.828513 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/82494c64-e789-4829-8017-c44bd404e188-client-ca-bundle\") pod \"metrics-server-8465bdc94b-6kxjv\" (UID: \"82494c64-e789-4829-8017-c44bd404e188\") " pod="openshift-monitoring/metrics-server-8465bdc94b-6kxjv" Jul 06 15:35:18.828552 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:18.828551 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/82494c64-e789-4829-8017-c44bd404e188-secret-metrics-server-tls\") pod \"metrics-server-8465bdc94b-6kxjv\" (UID: \"82494c64-e789-4829-8017-c44bd404e188\") " pod="openshift-monitoring/metrics-server-8465bdc94b-6kxjv" Jul 06 15:35:18.828795 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:18.828575 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/82494c64-e789-4829-8017-c44bd404e188-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-8465bdc94b-6kxjv\" (UID: \"82494c64-e789-4829-8017-c44bd404e188\") " pod="openshift-monitoring/metrics-server-8465bdc94b-6kxjv" Jul 06 15:35:18.828795 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:18.828592 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/82494c64-e789-4829-8017-c44bd404e188-audit-log\") pod \"metrics-server-8465bdc94b-6kxjv\" (UID: \"82494c64-e789-4829-8017-c44bd404e188\") " pod="openshift-monitoring/metrics-server-8465bdc94b-6kxjv" Jul 06 15:35:18.828795 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:18.828620 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-czfgf\" (UniqueName: \"kubernetes.io/projected/82494c64-e789-4829-8017-c44bd404e188-kube-api-access-czfgf\") pod \"metrics-server-8465bdc94b-6kxjv\" (UID: \"82494c64-e789-4829-8017-c44bd404e188\") " pod="openshift-monitoring/metrics-server-8465bdc94b-6kxjv" Jul 06 15:35:18.828795 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:18.828643 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/82494c64-e789-4829-8017-c44bd404e188-metrics-server-audit-profiles\") pod \"metrics-server-8465bdc94b-6kxjv\" (UID: \"82494c64-e789-4829-8017-c44bd404e188\") " pod="openshift-monitoring/metrics-server-8465bdc94b-6kxjv" Jul 06 15:35:18.828795 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:18.828676 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/82494c64-e789-4829-8017-c44bd404e188-secret-metrics-server-client-certs\") pod \"metrics-server-8465bdc94b-6kxjv\" (UID: \"82494c64-e789-4829-8017-c44bd404e188\") " pod="openshift-monitoring/metrics-server-8465bdc94b-6kxjv" Jul 06 15:35:18.829119 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:18.829094 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/82494c64-e789-4829-8017-c44bd404e188-audit-log\") pod \"metrics-server-8465bdc94b-6kxjv\" (UID: \"82494c64-e789-4829-8017-c44bd404e188\") " pod="openshift-monitoring/metrics-server-8465bdc94b-6kxjv" Jul 06 15:35:18.829400 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:18.829375 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/82494c64-e789-4829-8017-c44bd404e188-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-8465bdc94b-6kxjv\" (UID: \"82494c64-e789-4829-8017-c44bd404e188\") " pod="openshift-monitoring/metrics-server-8465bdc94b-6kxjv" Jul 06 15:35:18.829782 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:18.829757 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/82494c64-e789-4829-8017-c44bd404e188-metrics-server-audit-profiles\") pod \"metrics-server-8465bdc94b-6kxjv\" (UID: \"82494c64-e789-4829-8017-c44bd404e188\") " pod="openshift-monitoring/metrics-server-8465bdc94b-6kxjv" Jul 06 15:35:18.831384 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:18.831352 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/82494c64-e789-4829-8017-c44bd404e188-secret-metrics-server-client-certs\") pod \"metrics-server-8465bdc94b-6kxjv\" (UID: \"82494c64-e789-4829-8017-c44bd404e188\") " pod="openshift-monitoring/metrics-server-8465bdc94b-6kxjv" Jul 06 15:35:18.831384 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:18.831376 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/82494c64-e789-4829-8017-c44bd404e188-client-ca-bundle\") pod \"metrics-server-8465bdc94b-6kxjv\" (UID: \"82494c64-e789-4829-8017-c44bd404e188\") " pod="openshift-monitoring/metrics-server-8465bdc94b-6kxjv" Jul 06 15:35:18.831523 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:18.831500 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/82494c64-e789-4829-8017-c44bd404e188-secret-metrics-server-tls\") pod \"metrics-server-8465bdc94b-6kxjv\" (UID: \"82494c64-e789-4829-8017-c44bd404e188\") " pod="openshift-monitoring/metrics-server-8465bdc94b-6kxjv" Jul 06 15:35:18.837565 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:18.837545 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-czfgf\" (UniqueName: \"kubernetes.io/projected/82494c64-e789-4829-8017-c44bd404e188-kube-api-access-czfgf\") pod \"metrics-server-8465bdc94b-6kxjv\" (UID: \"82494c64-e789-4829-8017-c44bd404e188\") " pod="openshift-monitoring/metrics-server-8465bdc94b-6kxjv" Jul 06 15:35:18.939806 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:18.939543 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-68b757fb67-l47bf" Jul 06 15:35:18.974576 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:18.974538 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-8465bdc94b-6kxjv" Jul 06 15:35:19.173789 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:19.173737 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-8465bdc94b-6kxjv"] Jul 06 15:35:19.180535 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:35:19.180461 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod82494c64_e789_4829_8017_c44bd404e188.slice/crio-3ebf714d3768b4aaa07583314f1ee03b65afe3e471dba3d366dd94c2f61a9e88 WatchSource:0}: Error finding container 3ebf714d3768b4aaa07583314f1ee03b65afe3e471dba3d366dd94c2f61a9e88: Status 404 returned error can't find the container with id 3ebf714d3768b4aaa07583314f1ee03b65afe3e471dba3d366dd94c2f61a9e88 Jul 06 15:35:19.445880 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:19.445846 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-8465bdc94b-6kxjv" event={"ID":"82494c64-e789-4829-8017-c44bd404e188","Type":"ContainerStarted","Data":"3ebf714d3768b4aaa07583314f1ee03b65afe3e471dba3d366dd94c2f61a9e88"} Jul 06 15:35:20.452394 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:20.452362 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-54dc4c49bb-mj926" event={"ID":"206bbf71-dc24-4a17-bae2-8d17e7488a47","Type":"ContainerStarted","Data":"011068748a8640ae0663df074c10e4509a062ab7786276a79e96bd4b73c3720a"} Jul 06 15:35:20.452771 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:20.452443 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-54dc4c49bb-mj926" event={"ID":"206bbf71-dc24-4a17-bae2-8d17e7488a47","Type":"ContainerStarted","Data":"aa2162175f4d5fe968900f31e2365566134b00577b1217b7f3336d3dbc6836c5"} Jul 06 15:35:20.452771 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:20.452470 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-54dc4c49bb-mj926" event={"ID":"206bbf71-dc24-4a17-bae2-8d17e7488a47","Type":"ContainerStarted","Data":"15421c45dd7245249e39a3cb7379e15f9f0a216ddc0f15c2b6b5b803b2833d2d"} Jul 06 15:35:21.271161 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:21.271133 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-pdw4n" Jul 06 15:35:22.462881 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:22.462844 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-54dc4c49bb-mj926" event={"ID":"206bbf71-dc24-4a17-bae2-8d17e7488a47","Type":"ContainerStarted","Data":"db5b8c21d4fe97f463379f3772fcbeba75206a540a96d094f97e694012d19e04"} Jul 06 15:35:22.463360 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:22.462962 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-54dc4c49bb-mj926" Jul 06 15:35:22.463360 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:22.462995 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-54dc4c49bb-mj926" event={"ID":"206bbf71-dc24-4a17-bae2-8d17e7488a47","Type":"ContainerStarted","Data":"4dbc0c404fa14703417aac40144d3b27d9ef9e080c464b3d0b040b4182b56a17"} Jul 06 15:35:22.463360 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:22.463030 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-54dc4c49bb-mj926" event={"ID":"206bbf71-dc24-4a17-bae2-8d17e7488a47","Type":"ContainerStarted","Data":"8726ec84e0c328af52744a9416e22df8d9cf04bebab89d406f36846464275a47"} Jul 06 15:35:22.464309 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:22.464284 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-8465bdc94b-6kxjv" event={"ID":"82494c64-e789-4829-8017-c44bd404e188","Type":"ContainerStarted","Data":"216fedbcec27da69ad18aac05ddd5e42023d30f25a20840991a0a3f7bd1eb369"} Jul 06 15:35:22.491676 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:22.491628 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-54dc4c49bb-mj926" podStartSLOduration=2.085019677 podStartE2EDuration="5.491613478s" totalImagesPullingTime="3.406593801s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 15:35:17 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 15:35:22.489692754 +0000 UTC m=+98.998925977" watchObservedRunningTime="2026-07-06 15:35:22.491613478 +0000 UTC m=+99.000846651" Jul 06 15:35:22.511234 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:22.511191 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-8465bdc94b-6kxjv" podStartSLOduration=2.19278698 podStartE2EDuration="4.511173491s" totalImagesPullingTime="2.318386511s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 15:35:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 15:35:22.510037607 +0000 UTC m=+99.019270783" watchObservedRunningTime="2026-07-06 15:35:22.511173491 +0000 UTC m=+99.020406729" Jul 06 15:35:24.061059 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:24.061017 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-749448d4c7-6qfwt"] Jul 06 15:35:24.065613 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:24.065589 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-749448d4c7-6qfwt" Jul 06 15:35:24.068840 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:24.068818 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-8t88w\"" Jul 06 15:35:24.069227 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:24.069198 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 06 15:35:24.069907 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:24.069892 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 06 15:35:24.070000 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:24.069897 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 06 15:35:24.070000 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:24.069925 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 06 15:35:24.070256 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:24.070237 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 06 15:35:24.070320 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:24.070245 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 06 15:35:24.070471 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:24.070445 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 06 15:35:24.077769 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:24.077745 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-749448d4c7-6qfwt"] Jul 06 15:35:24.177527 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:24.177491 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/13924c76-bb59-4b1f-ad02-929e453aeeed-service-ca\") pod \"console-749448d4c7-6qfwt\" (UID: \"13924c76-bb59-4b1f-ad02-929e453aeeed\") " pod="openshift-console/console-749448d4c7-6qfwt" Jul 06 15:35:24.177683 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:24.177544 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/13924c76-bb59-4b1f-ad02-929e453aeeed-oauth-serving-cert\") pod \"console-749448d4c7-6qfwt\" (UID: \"13924c76-bb59-4b1f-ad02-929e453aeeed\") " pod="openshift-console/console-749448d4c7-6qfwt" Jul 06 15:35:24.177683 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:24.177611 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/13924c76-bb59-4b1f-ad02-929e453aeeed-console-config\") pod \"console-749448d4c7-6qfwt\" (UID: \"13924c76-bb59-4b1f-ad02-929e453aeeed\") " pod="openshift-console/console-749448d4c7-6qfwt" Jul 06 15:35:24.177683 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:24.177635 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jhc2j\" (UniqueName: \"kubernetes.io/projected/13924c76-bb59-4b1f-ad02-929e453aeeed-kube-api-access-jhc2j\") pod \"console-749448d4c7-6qfwt\" (UID: \"13924c76-bb59-4b1f-ad02-929e453aeeed\") " pod="openshift-console/console-749448d4c7-6qfwt" Jul 06 15:35:24.177683 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:24.177665 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/13924c76-bb59-4b1f-ad02-929e453aeeed-console-serving-cert\") pod \"console-749448d4c7-6qfwt\" (UID: \"13924c76-bb59-4b1f-ad02-929e453aeeed\") " pod="openshift-console/console-749448d4c7-6qfwt" Jul 06 15:35:24.177855 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:24.177707 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/13924c76-bb59-4b1f-ad02-929e453aeeed-console-oauth-config\") pod \"console-749448d4c7-6qfwt\" (UID: \"13924c76-bb59-4b1f-ad02-929e453aeeed\") " pod="openshift-console/console-749448d4c7-6qfwt" Jul 06 15:35:24.278952 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:24.278915 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/13924c76-bb59-4b1f-ad02-929e453aeeed-oauth-serving-cert\") pod \"console-749448d4c7-6qfwt\" (UID: \"13924c76-bb59-4b1f-ad02-929e453aeeed\") " pod="openshift-console/console-749448d4c7-6qfwt" Jul 06 15:35:24.279145 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:24.278965 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/13924c76-bb59-4b1f-ad02-929e453aeeed-console-config\") pod \"console-749448d4c7-6qfwt\" (UID: \"13924c76-bb59-4b1f-ad02-929e453aeeed\") " pod="openshift-console/console-749448d4c7-6qfwt" Jul 06 15:35:24.279145 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:24.278983 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jhc2j\" (UniqueName: \"kubernetes.io/projected/13924c76-bb59-4b1f-ad02-929e453aeeed-kube-api-access-jhc2j\") pod \"console-749448d4c7-6qfwt\" (UID: \"13924c76-bb59-4b1f-ad02-929e453aeeed\") " pod="openshift-console/console-749448d4c7-6qfwt" Jul 06 15:35:24.279145 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:24.279007 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/13924c76-bb59-4b1f-ad02-929e453aeeed-console-serving-cert\") pod \"console-749448d4c7-6qfwt\" (UID: \"13924c76-bb59-4b1f-ad02-929e453aeeed\") " pod="openshift-console/console-749448d4c7-6qfwt" Jul 06 15:35:24.279145 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:24.279043 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/13924c76-bb59-4b1f-ad02-929e453aeeed-console-oauth-config\") pod \"console-749448d4c7-6qfwt\" (UID: \"13924c76-bb59-4b1f-ad02-929e453aeeed\") " pod="openshift-console/console-749448d4c7-6qfwt" Jul 06 15:35:24.279145 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:24.279061 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/13924c76-bb59-4b1f-ad02-929e453aeeed-service-ca\") pod \"console-749448d4c7-6qfwt\" (UID: \"13924c76-bb59-4b1f-ad02-929e453aeeed\") " pod="openshift-console/console-749448d4c7-6qfwt" Jul 06 15:35:24.279712 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:24.279686 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/13924c76-bb59-4b1f-ad02-929e453aeeed-console-config\") pod \"console-749448d4c7-6qfwt\" (UID: \"13924c76-bb59-4b1f-ad02-929e453aeeed\") " pod="openshift-console/console-749448d4c7-6qfwt" Jul 06 15:35:24.279865 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:24.279799 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/13924c76-bb59-4b1f-ad02-929e453aeeed-oauth-serving-cert\") pod \"console-749448d4c7-6qfwt\" (UID: \"13924c76-bb59-4b1f-ad02-929e453aeeed\") " pod="openshift-console/console-749448d4c7-6qfwt" Jul 06 15:35:24.279972 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:24.279832 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/13924c76-bb59-4b1f-ad02-929e453aeeed-service-ca\") pod \"console-749448d4c7-6qfwt\" (UID: \"13924c76-bb59-4b1f-ad02-929e453aeeed\") " pod="openshift-console/console-749448d4c7-6qfwt" Jul 06 15:35:24.281489 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:24.281469 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/13924c76-bb59-4b1f-ad02-929e453aeeed-console-serving-cert\") pod \"console-749448d4c7-6qfwt\" (UID: \"13924c76-bb59-4b1f-ad02-929e453aeeed\") " pod="openshift-console/console-749448d4c7-6qfwt" Jul 06 15:35:24.281572 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:24.281556 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/13924c76-bb59-4b1f-ad02-929e453aeeed-console-oauth-config\") pod \"console-749448d4c7-6qfwt\" (UID: \"13924c76-bb59-4b1f-ad02-929e453aeeed\") " pod="openshift-console/console-749448d4c7-6qfwt" Jul 06 15:35:24.288867 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:24.288848 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jhc2j\" (UniqueName: \"kubernetes.io/projected/13924c76-bb59-4b1f-ad02-929e453aeeed-kube-api-access-jhc2j\") pod \"console-749448d4c7-6qfwt\" (UID: \"13924c76-bb59-4b1f-ad02-929e453aeeed\") " pod="openshift-console/console-749448d4c7-6qfwt" Jul 06 15:35:24.375011 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:24.374935 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-749448d4c7-6qfwt" Jul 06 15:35:24.511961 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:24.511937 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-749448d4c7-6qfwt"] Jul 06 15:35:24.513914 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:35:24.513884 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod13924c76_bb59_4b1f_ad02_929e453aeeed.slice/crio-5e9d634e251e2ed23b4ea236225967de84e9b960a9d1f6ef2b0145d6df025413 WatchSource:0}: Error finding container 5e9d634e251e2ed23b4ea236225967de84e9b960a9d1f6ef2b0145d6df025413: Status 404 returned error can't find the container with id 5e9d634e251e2ed23b4ea236225967de84e9b960a9d1f6ef2b0145d6df025413 Jul 06 15:35:25.479923 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:25.479885 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-749448d4c7-6qfwt" event={"ID":"13924c76-bb59-4b1f-ad02-929e453aeeed","Type":"ContainerStarted","Data":"5e9d634e251e2ed23b4ea236225967de84e9b960a9d1f6ef2b0145d6df025413"} Jul 06 15:35:28.476990 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:28.476964 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-54dc4c49bb-mj926" Jul 06 15:35:28.494541 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:28.494509 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-749448d4c7-6qfwt" event={"ID":"13924c76-bb59-4b1f-ad02-929e453aeeed","Type":"ContainerStarted","Data":"d30ff9f3c26a7c8f5a1543c4cc935f448a1a1d038e63c4693a7a6869c32b3ce7"} Jul 06 15:35:28.530902 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:28.530840 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-749448d4c7-6qfwt" podStartSLOduration=0.685197077 podStartE2EDuration="4.530816241s" totalImagesPullingTime="3.845619164s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 15:35:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 15:35:28.528852718 +0000 UTC m=+105.038085916" watchObservedRunningTime="2026-07-06 15:35:28.530816241 +0000 UTC m=+105.040049416" Jul 06 15:35:30.497362 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:30.497326 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-7578959f54-4tp2s"] Jul 06 15:35:30.516641 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:30.516609 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7578959f54-4tp2s" Jul 06 15:35:30.522301 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:30.522271 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7578959f54-4tp2s"] Jul 06 15:35:30.527593 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:30.527566 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 06 15:35:30.637840 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:30.637811 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c0d108a0-96a4-4660-9421-c3e5dca56c02-console-serving-cert\") pod \"console-7578959f54-4tp2s\" (UID: \"c0d108a0-96a4-4660-9421-c3e5dca56c02\") " pod="openshift-console/console-7578959f54-4tp2s" Jul 06 15:35:30.638011 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:30.637848 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c0d108a0-96a4-4660-9421-c3e5dca56c02-console-config\") pod \"console-7578959f54-4tp2s\" (UID: \"c0d108a0-96a4-4660-9421-c3e5dca56c02\") " pod="openshift-console/console-7578959f54-4tp2s" Jul 06 15:35:30.638011 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:30.637878 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c0d108a0-96a4-4660-9421-c3e5dca56c02-trusted-ca-bundle\") pod \"console-7578959f54-4tp2s\" (UID: \"c0d108a0-96a4-4660-9421-c3e5dca56c02\") " pod="openshift-console/console-7578959f54-4tp2s" Jul 06 15:35:30.638011 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:30.637979 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c0d108a0-96a4-4660-9421-c3e5dca56c02-service-ca\") pod \"console-7578959f54-4tp2s\" (UID: \"c0d108a0-96a4-4660-9421-c3e5dca56c02\") " pod="openshift-console/console-7578959f54-4tp2s" Jul 06 15:35:30.638156 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:30.638038 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c0d108a0-96a4-4660-9421-c3e5dca56c02-oauth-serving-cert\") pod \"console-7578959f54-4tp2s\" (UID: \"c0d108a0-96a4-4660-9421-c3e5dca56c02\") " pod="openshift-console/console-7578959f54-4tp2s" Jul 06 15:35:30.638156 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:30.638055 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5pnb4\" (UniqueName: \"kubernetes.io/projected/c0d108a0-96a4-4660-9421-c3e5dca56c02-kube-api-access-5pnb4\") pod \"console-7578959f54-4tp2s\" (UID: \"c0d108a0-96a4-4660-9421-c3e5dca56c02\") " pod="openshift-console/console-7578959f54-4tp2s" Jul 06 15:35:30.638156 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:30.638088 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c0d108a0-96a4-4660-9421-c3e5dca56c02-console-oauth-config\") pod \"console-7578959f54-4tp2s\" (UID: \"c0d108a0-96a4-4660-9421-c3e5dca56c02\") " pod="openshift-console/console-7578959f54-4tp2s" Jul 06 15:35:30.738692 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:30.738654 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c0d108a0-96a4-4660-9421-c3e5dca56c02-console-oauth-config\") pod \"console-7578959f54-4tp2s\" (UID: \"c0d108a0-96a4-4660-9421-c3e5dca56c02\") " pod="openshift-console/console-7578959f54-4tp2s" Jul 06 15:35:30.738855 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:30.738706 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c0d108a0-96a4-4660-9421-c3e5dca56c02-console-serving-cert\") pod \"console-7578959f54-4tp2s\" (UID: \"c0d108a0-96a4-4660-9421-c3e5dca56c02\") " pod="openshift-console/console-7578959f54-4tp2s" Jul 06 15:35:30.738855 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:30.738731 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c0d108a0-96a4-4660-9421-c3e5dca56c02-console-config\") pod \"console-7578959f54-4tp2s\" (UID: \"c0d108a0-96a4-4660-9421-c3e5dca56c02\") " pod="openshift-console/console-7578959f54-4tp2s" Jul 06 15:35:30.738855 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:30.738753 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c0d108a0-96a4-4660-9421-c3e5dca56c02-trusted-ca-bundle\") pod \"console-7578959f54-4tp2s\" (UID: \"c0d108a0-96a4-4660-9421-c3e5dca56c02\") " pod="openshift-console/console-7578959f54-4tp2s" Jul 06 15:35:30.738855 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:30.738774 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c0d108a0-96a4-4660-9421-c3e5dca56c02-service-ca\") pod \"console-7578959f54-4tp2s\" (UID: \"c0d108a0-96a4-4660-9421-c3e5dca56c02\") " pod="openshift-console/console-7578959f54-4tp2s" Jul 06 15:35:30.738855 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:30.738806 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c0d108a0-96a4-4660-9421-c3e5dca56c02-oauth-serving-cert\") pod \"console-7578959f54-4tp2s\" (UID: \"c0d108a0-96a4-4660-9421-c3e5dca56c02\") " pod="openshift-console/console-7578959f54-4tp2s" Jul 06 15:35:30.738855 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:30.738832 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5pnb4\" (UniqueName: \"kubernetes.io/projected/c0d108a0-96a4-4660-9421-c3e5dca56c02-kube-api-access-5pnb4\") pod \"console-7578959f54-4tp2s\" (UID: \"c0d108a0-96a4-4660-9421-c3e5dca56c02\") " pod="openshift-console/console-7578959f54-4tp2s" Jul 06 15:35:30.739577 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:30.739549 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c0d108a0-96a4-4660-9421-c3e5dca56c02-service-ca\") pod \"console-7578959f54-4tp2s\" (UID: \"c0d108a0-96a4-4660-9421-c3e5dca56c02\") " pod="openshift-console/console-7578959f54-4tp2s" Jul 06 15:35:30.739741 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:30.739718 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c0d108a0-96a4-4660-9421-c3e5dca56c02-console-config\") pod \"console-7578959f54-4tp2s\" (UID: \"c0d108a0-96a4-4660-9421-c3e5dca56c02\") " pod="openshift-console/console-7578959f54-4tp2s" Jul 06 15:35:30.739788 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:30.739742 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c0d108a0-96a4-4660-9421-c3e5dca56c02-trusted-ca-bundle\") pod \"console-7578959f54-4tp2s\" (UID: \"c0d108a0-96a4-4660-9421-c3e5dca56c02\") " pod="openshift-console/console-7578959f54-4tp2s" Jul 06 15:35:30.739952 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:30.739927 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c0d108a0-96a4-4660-9421-c3e5dca56c02-oauth-serving-cert\") pod \"console-7578959f54-4tp2s\" (UID: \"c0d108a0-96a4-4660-9421-c3e5dca56c02\") " pod="openshift-console/console-7578959f54-4tp2s" Jul 06 15:35:30.741183 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:30.741162 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c0d108a0-96a4-4660-9421-c3e5dca56c02-console-oauth-config\") pod \"console-7578959f54-4tp2s\" (UID: \"c0d108a0-96a4-4660-9421-c3e5dca56c02\") " pod="openshift-console/console-7578959f54-4tp2s" Jul 06 15:35:30.741343 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:30.741328 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c0d108a0-96a4-4660-9421-c3e5dca56c02-console-serving-cert\") pod \"console-7578959f54-4tp2s\" (UID: \"c0d108a0-96a4-4660-9421-c3e5dca56c02\") " pod="openshift-console/console-7578959f54-4tp2s" Jul 06 15:35:30.753146 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:30.753095 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5pnb4\" (UniqueName: \"kubernetes.io/projected/c0d108a0-96a4-4660-9421-c3e5dca56c02-kube-api-access-5pnb4\") pod \"console-7578959f54-4tp2s\" (UID: \"c0d108a0-96a4-4660-9421-c3e5dca56c02\") " pod="openshift-console/console-7578959f54-4tp2s" Jul 06 15:35:30.826994 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:30.826964 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7578959f54-4tp2s" Jul 06 15:35:31.000087 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:31.000060 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7578959f54-4tp2s"] Jul 06 15:35:31.002671 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:35:31.002644 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc0d108a0_96a4_4660_9421_c3e5dca56c02.slice/crio-6c562c049e2411a72ea82d22b1364ea29a83ceddf714e36d1968c47697c583b1 WatchSource:0}: Error finding container 6c562c049e2411a72ea82d22b1364ea29a83ceddf714e36d1968c47697c583b1: Status 404 returned error can't find the container with id 6c562c049e2411a72ea82d22b1364ea29a83ceddf714e36d1968c47697c583b1 Jul 06 15:35:31.509662 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:31.509626 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7578959f54-4tp2s" event={"ID":"c0d108a0-96a4-4660-9421-c3e5dca56c02","Type":"ContainerStarted","Data":"f07eab0be0266c22f5fc392b43bf7a7612e543f219990dc7cd572c75ebf0aa30"} Jul 06 15:35:31.510198 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:31.509680 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7578959f54-4tp2s" event={"ID":"c0d108a0-96a4-4660-9421-c3e5dca56c02","Type":"ContainerStarted","Data":"6c562c049e2411a72ea82d22b1364ea29a83ceddf714e36d1968c47697c583b1"} Jul 06 15:35:31.535668 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:31.535623 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-7578959f54-4tp2s" podStartSLOduration=1.5356080840000002 podStartE2EDuration="1.535608084s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 15:35:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 15:35:31.535145174 +0000 UTC m=+108.044378349" watchObservedRunningTime="2026-07-06 15:35:31.535608084 +0000 UTC m=+108.044841258" Jul 06 15:35:34.375982 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:34.375943 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-749448d4c7-6qfwt" Jul 06 15:35:34.376357 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:34.376014 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-749448d4c7-6qfwt" Jul 06 15:35:34.380813 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:34.380788 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-749448d4c7-6qfwt" Jul 06 15:35:34.525290 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:34.525261 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-749448d4c7-6qfwt" Jul 06 15:35:38.975032 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:38.974997 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-8465bdc94b-6kxjv" Jul 06 15:35:38.975450 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:38.975053 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-8465bdc94b-6kxjv" Jul 06 15:35:40.828088 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:40.828044 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7578959f54-4tp2s" Jul 06 15:35:40.828532 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:40.828121 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-7578959f54-4tp2s" Jul 06 15:35:40.833047 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:40.833023 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-7578959f54-4tp2s" Jul 06 15:35:41.554158 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:41.554128 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-7578959f54-4tp2s" Jul 06 15:35:41.607213 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:41.607180 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-749448d4c7-6qfwt"] Jul 06 15:35:42.555937 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:42.555901 2563 generic.go:356] "Generic (PLEG): container finished" podID="0e2e4871-594a-42f5-abb7-867d61453254" containerID="8b8d5c03f37cc0a4bda97b51dcbc0fafa10cf71a83f229be05696af5ddb9b797" exitCode=0 Jul 06 15:35:42.556348 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:42.555979 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5c847dcfc4-nvj9g" event={"ID":"0e2e4871-594a-42f5-abb7-867d61453254","Type":"ContainerDied","Data":"8b8d5c03f37cc0a4bda97b51dcbc0fafa10cf71a83f229be05696af5ddb9b797"} Jul 06 15:35:42.556441 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:42.556394 2563 scope.go:117] "RemoveContainer" containerID="8b8d5c03f37cc0a4bda97b51dcbc0fafa10cf71a83f229be05696af5ddb9b797" Jul 06 15:35:43.046397 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:43.046306 2563 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-image-registry/image-registry-d668d48f7-cncvq" podUID="1911fa1c-48ce-44ab-8b64-7ae70ce80a0d" containerName="registry" containerID="cri-o://5578602aacd91466dd03f773d33f7690d2421d91505213f790978053b14a929c" gracePeriod=30 Jul 06 15:35:43.343727 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:43.343699 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-d668d48f7-cncvq" Jul 06 15:35:43.453124 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:43.453092 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/1911fa1c-48ce-44ab-8b64-7ae70ce80a0d-trusted-ca\") pod \"1911fa1c-48ce-44ab-8b64-7ae70ce80a0d\" (UID: \"1911fa1c-48ce-44ab-8b64-7ae70ce80a0d\") " Jul 06 15:35:43.453280 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:43.453158 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cc9vl\" (UniqueName: \"kubernetes.io/projected/1911fa1c-48ce-44ab-8b64-7ae70ce80a0d-kube-api-access-cc9vl\") pod \"1911fa1c-48ce-44ab-8b64-7ae70ce80a0d\" (UID: \"1911fa1c-48ce-44ab-8b64-7ae70ce80a0d\") " Jul 06 15:35:43.453280 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:43.453183 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/1911fa1c-48ce-44ab-8b64-7ae70ce80a0d-image-registry-private-configuration\") pod \"1911fa1c-48ce-44ab-8b64-7ae70ce80a0d\" (UID: \"1911fa1c-48ce-44ab-8b64-7ae70ce80a0d\") " Jul 06 15:35:43.453280 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:43.453216 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/1911fa1c-48ce-44ab-8b64-7ae70ce80a0d-registry-certificates\") pod \"1911fa1c-48ce-44ab-8b64-7ae70ce80a0d\" (UID: \"1911fa1c-48ce-44ab-8b64-7ae70ce80a0d\") " Jul 06 15:35:43.453280 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:43.453235 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1911fa1c-48ce-44ab-8b64-7ae70ce80a0d-registry-tls\") pod \"1911fa1c-48ce-44ab-8b64-7ae70ce80a0d\" (UID: \"1911fa1c-48ce-44ab-8b64-7ae70ce80a0d\") " Jul 06 15:35:43.453280 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:43.453264 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/1911fa1c-48ce-44ab-8b64-7ae70ce80a0d-bound-sa-token\") pod \"1911fa1c-48ce-44ab-8b64-7ae70ce80a0d\" (UID: \"1911fa1c-48ce-44ab-8b64-7ae70ce80a0d\") " Jul 06 15:35:43.453536 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:43.453299 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/1911fa1c-48ce-44ab-8b64-7ae70ce80a0d-installation-pull-secrets\") pod \"1911fa1c-48ce-44ab-8b64-7ae70ce80a0d\" (UID: \"1911fa1c-48ce-44ab-8b64-7ae70ce80a0d\") " Jul 06 15:35:43.453536 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:43.453336 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/1911fa1c-48ce-44ab-8b64-7ae70ce80a0d-ca-trust-extracted\") pod \"1911fa1c-48ce-44ab-8b64-7ae70ce80a0d\" (UID: \"1911fa1c-48ce-44ab-8b64-7ae70ce80a0d\") " Jul 06 15:35:43.453837 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:43.453635 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1911fa1c-48ce-44ab-8b64-7ae70ce80a0d-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "1911fa1c-48ce-44ab-8b64-7ae70ce80a0d" (UID: "1911fa1c-48ce-44ab-8b64-7ae70ce80a0d"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 15:35:43.453837 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:43.453765 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1911fa1c-48ce-44ab-8b64-7ae70ce80a0d-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "1911fa1c-48ce-44ab-8b64-7ae70ce80a0d" (UID: "1911fa1c-48ce-44ab-8b64-7ae70ce80a0d"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 15:35:43.456019 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:43.455985 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1911fa1c-48ce-44ab-8b64-7ae70ce80a0d-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "1911fa1c-48ce-44ab-8b64-7ae70ce80a0d" (UID: "1911fa1c-48ce-44ab-8b64-7ae70ce80a0d"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 15:35:43.456019 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:43.455996 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1911fa1c-48ce-44ab-8b64-7ae70ce80a0d-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "1911fa1c-48ce-44ab-8b64-7ae70ce80a0d" (UID: "1911fa1c-48ce-44ab-8b64-7ae70ce80a0d"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 15:35:43.456182 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:43.456096 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1911fa1c-48ce-44ab-8b64-7ae70ce80a0d-kube-api-access-cc9vl" (OuterVolumeSpecName: "kube-api-access-cc9vl") pod "1911fa1c-48ce-44ab-8b64-7ae70ce80a0d" (UID: "1911fa1c-48ce-44ab-8b64-7ae70ce80a0d"). InnerVolumeSpecName "kube-api-access-cc9vl". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 15:35:43.456182 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:43.456117 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1911fa1c-48ce-44ab-8b64-7ae70ce80a0d-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "1911fa1c-48ce-44ab-8b64-7ae70ce80a0d" (UID: "1911fa1c-48ce-44ab-8b64-7ae70ce80a0d"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 15:35:43.456335 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:43.456312 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1911fa1c-48ce-44ab-8b64-7ae70ce80a0d-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "1911fa1c-48ce-44ab-8b64-7ae70ce80a0d" (UID: "1911fa1c-48ce-44ab-8b64-7ae70ce80a0d"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 15:35:43.462148 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:43.462126 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1911fa1c-48ce-44ab-8b64-7ae70ce80a0d-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "1911fa1c-48ce-44ab-8b64-7ae70ce80a0d" (UID: "1911fa1c-48ce-44ab-8b64-7ae70ce80a0d"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 15:35:43.554762 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:43.554727 2563 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/1911fa1c-48ce-44ab-8b64-7ae70ce80a0d-registry-certificates\") on node \"ip-10-0-129-154.ec2.internal\" DevicePath \"\"" Jul 06 15:35:43.554762 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:43.554758 2563 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1911fa1c-48ce-44ab-8b64-7ae70ce80a0d-registry-tls\") on node \"ip-10-0-129-154.ec2.internal\" DevicePath \"\"" Jul 06 15:35:43.554762 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:43.554768 2563 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/1911fa1c-48ce-44ab-8b64-7ae70ce80a0d-bound-sa-token\") on node \"ip-10-0-129-154.ec2.internal\" DevicePath \"\"" Jul 06 15:35:43.554972 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:43.554776 2563 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/1911fa1c-48ce-44ab-8b64-7ae70ce80a0d-installation-pull-secrets\") on node \"ip-10-0-129-154.ec2.internal\" DevicePath \"\"" Jul 06 15:35:43.554972 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:43.554784 2563 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/1911fa1c-48ce-44ab-8b64-7ae70ce80a0d-ca-trust-extracted\") on node \"ip-10-0-129-154.ec2.internal\" DevicePath \"\"" Jul 06 15:35:43.554972 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:43.554794 2563 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/1911fa1c-48ce-44ab-8b64-7ae70ce80a0d-trusted-ca\") on node \"ip-10-0-129-154.ec2.internal\" DevicePath \"\"" Jul 06 15:35:43.554972 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:43.554803 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cc9vl\" (UniqueName: \"kubernetes.io/projected/1911fa1c-48ce-44ab-8b64-7ae70ce80a0d-kube-api-access-cc9vl\") on node \"ip-10-0-129-154.ec2.internal\" DevicePath \"\"" Jul 06 15:35:43.554972 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:43.554811 2563 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/1911fa1c-48ce-44ab-8b64-7ae70ce80a0d-image-registry-private-configuration\") on node \"ip-10-0-129-154.ec2.internal\" DevicePath \"\"" Jul 06 15:35:43.561514 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:43.561443 2563 generic.go:356] "Generic (PLEG): container finished" podID="1911fa1c-48ce-44ab-8b64-7ae70ce80a0d" containerID="5578602aacd91466dd03f773d33f7690d2421d91505213f790978053b14a929c" exitCode=0 Jul 06 15:35:43.561857 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:43.561522 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-d668d48f7-cncvq" event={"ID":"1911fa1c-48ce-44ab-8b64-7ae70ce80a0d","Type":"ContainerDied","Data":"5578602aacd91466dd03f773d33f7690d2421d91505213f790978053b14a929c"} Jul 06 15:35:43.561857 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:43.561563 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-d668d48f7-cncvq" Jul 06 15:35:43.561857 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:43.561579 2563 scope.go:117] "RemoveContainer" containerID="5578602aacd91466dd03f773d33f7690d2421d91505213f790978053b14a929c" Jul 06 15:35:43.561857 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:43.561568 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-d668d48f7-cncvq" event={"ID":"1911fa1c-48ce-44ab-8b64-7ae70ce80a0d","Type":"ContainerDied","Data":"1fa93229d8c438423dcde9915280961afb79833fbe88fd71b69ded7d7c453d07"} Jul 06 15:35:43.563729 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:43.563704 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5c847dcfc4-nvj9g" event={"ID":"0e2e4871-594a-42f5-abb7-867d61453254","Type":"ContainerStarted","Data":"a573fec90cdc40ccd45da44bd964560facb52f66edc8c76e90eeddad500791e0"} Jul 06 15:35:43.571554 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:43.571530 2563 scope.go:117] "RemoveContainer" containerID="5578602aacd91466dd03f773d33f7690d2421d91505213f790978053b14a929c" Jul 06 15:35:43.571826 ip-10-0-129-154 kubenswrapper[2563]: E0706 15:35:43.571804 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5578602aacd91466dd03f773d33f7690d2421d91505213f790978053b14a929c\": container with ID starting with 5578602aacd91466dd03f773d33f7690d2421d91505213f790978053b14a929c not found: ID does not exist" containerID="5578602aacd91466dd03f773d33f7690d2421d91505213f790978053b14a929c" Jul 06 15:35:43.571878 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:43.571835 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5578602aacd91466dd03f773d33f7690d2421d91505213f790978053b14a929c"} err="failed to get container status \"5578602aacd91466dd03f773d33f7690d2421d91505213f790978053b14a929c\": rpc error: code = NotFound desc = could not find container \"5578602aacd91466dd03f773d33f7690d2421d91505213f790978053b14a929c\": container with ID starting with 5578602aacd91466dd03f773d33f7690d2421d91505213f790978053b14a929c not found: ID does not exist" Jul 06 15:35:43.624922 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:43.624884 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-d668d48f7-cncvq"] Jul 06 15:35:43.633252 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:43.633224 2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-d668d48f7-cncvq"] Jul 06 15:35:43.944722 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:43.944691 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1911fa1c-48ce-44ab-8b64-7ae70ce80a0d" path="/var/lib/kubelet/pods/1911fa1c-48ce-44ab-8b64-7ae70ce80a0d/volumes" Jul 06 15:35:44.410926 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:44.410900 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-64786d9cbb-trjfs_bf5c30e2-bf88-468b-adf2-2e8f73f136b4/cluster-monitoring-operator/0.log" Jul 06 15:35:44.631752 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:44.631720 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-64786d9cbb-trjfs_bf5c30e2-bf88-468b-adf2-2e8f73f136b4/cluster-monitoring-operator/0.log" Jul 06 15:35:44.809552 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:44.809479 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-64786d9cbb-trjfs_bf5c30e2-bf88-468b-adf2-2e8f73f136b4/cluster-monitoring-operator/1.log" Jul 06 15:35:45.608813 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:45.608783 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-8465bdc94b-6kxjv_82494c64-e789-4829-8017-c44bd404e188/metrics-server/0.log" Jul 06 15:35:47.210785 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:47.210755 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-sjxxv_c55d32ae-8d34-4ca2-8d07-a3a374ecf12d/init-textfile/0.log" Jul 06 15:35:47.409136 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:47.409108 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-sjxxv_c55d32ae-8d34-4ca2-8d07-a3a374ecf12d/node-exporter/0.log" Jul 06 15:35:47.580986 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:47.580890 2563 generic.go:356] "Generic (PLEG): container finished" podID="929783be-42bc-444c-9c66-a3a17fd9ff04" containerID="8bda6142bb256e89cb2b5d85c534bc5e29680e3fecf442b0e33d10e13955b2ef" exitCode=0 Jul 06 15:35:47.580986 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:47.580953 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-28kt7" event={"ID":"929783be-42bc-444c-9c66-a3a17fd9ff04","Type":"ContainerDied","Data":"8bda6142bb256e89cb2b5d85c534bc5e29680e3fecf442b0e33d10e13955b2ef"} Jul 06 15:35:47.581337 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:47.581319 2563 scope.go:117] "RemoveContainer" containerID="8bda6142bb256e89cb2b5d85c534bc5e29680e3fecf442b0e33d10e13955b2ef" Jul 06 15:35:47.611033 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:47.611005 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-sjxxv_c55d32ae-8d34-4ca2-8d07-a3a374ecf12d/kube-rbac-proxy/0.log" Jul 06 15:35:48.588778 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:48.588738 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-28kt7" event={"ID":"929783be-42bc-444c-9c66-a3a17fd9ff04","Type":"ContainerStarted","Data":"9cea0d98e06ab98d80d22fdd52859a4fd5ebd9831e8242b97c6599394d1a1798"} Jul 06 15:35:49.808900 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:49.808872 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-567f7b8b7c-b7s4w_eeb3e868-088a-4b4d-80a1-143a15ad91c8/prometheus-operator/0.log" Jul 06 15:35:50.008228 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:50.008175 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-567f7b8b7c-b7s4w_eeb3e868-088a-4b4d-80a1-143a15ad91c8/kube-rbac-proxy/0.log" Jul 06 15:35:50.207787 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:50.207760 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-d559d66d5-2bbvr_e5f102a2-585e-4fc2-9579-2c221f542f83/prometheus-operator-admission-webhook/0.log" Jul 06 15:35:50.407691 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:50.407657 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-54dc4c49bb-mj926_206bbf71-dc24-4a17-bae2-8d17e7488a47/thanos-query/0.log" Jul 06 15:35:50.610492 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:50.610384 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-54dc4c49bb-mj926_206bbf71-dc24-4a17-bae2-8d17e7488a47/kube-rbac-proxy-web/0.log" Jul 06 15:35:50.807849 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:50.807819 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-54dc4c49bb-mj926_206bbf71-dc24-4a17-bae2-8d17e7488a47/kube-rbac-proxy/0.log" Jul 06 15:35:51.009040 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:51.009012 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-54dc4c49bb-mj926_206bbf71-dc24-4a17-bae2-8d17e7488a47/prom-label-proxy/0.log" Jul 06 15:35:51.207983 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:51.207957 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-54dc4c49bb-mj926_206bbf71-dc24-4a17-bae2-8d17e7488a47/kube-rbac-proxy-rules/0.log" Jul 06 15:35:51.411277 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:51.411259 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-54dc4c49bb-mj926_206bbf71-dc24-4a17-bae2-8d17e7488a47/kube-rbac-proxy-metrics/0.log" Jul 06 15:35:51.607581 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:51.607558 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-6bffdf9778-fgktw_f955a985-08b6-49d9-b159-8ec3916d8b20/networking-console-plugin/0.log" Jul 06 15:35:51.817360 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:51.817283 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-l47bf_9ad7e968-ed2e-4cbe-9b4b-780bc9d01b88/console-operator/2.log" Jul 06 15:35:52.013358 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:52.013328 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-l47bf_9ad7e968-ed2e-4cbe-9b4b-780bc9d01b88/console-operator/3.log" Jul 06 15:35:52.208358 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:52.208329 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-749448d4c7-6qfwt_13924c76-bb59-4b1f-ad02-929e453aeeed/console/0.log" Jul 06 15:35:52.411194 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:52.411168 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-7578959f54-4tp2s_c0d108a0-96a4-4660-9421-c3e5dca56c02/console/0.log" Jul 06 15:35:56.625284 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:56.625253 2563 generic.go:356] "Generic (PLEG): container finished" podID="966f6ac8-fde8-46e3-b9fe-816b6f334efa" containerID="5c23b23a2eb925f2b22b3d2363322bbe1ba87661591daa2c13ea7c2a2163cf78" exitCode=0 Jul 06 15:35:56.625726 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:56.625340 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-662qp" event={"ID":"966f6ac8-fde8-46e3-b9fe-816b6f334efa","Type":"ContainerDied","Data":"5c23b23a2eb925f2b22b3d2363322bbe1ba87661591daa2c13ea7c2a2163cf78"} Jul 06 15:35:56.625772 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:56.625729 2563 scope.go:117] "RemoveContainer" containerID="5c23b23a2eb925f2b22b3d2363322bbe1ba87661591daa2c13ea7c2a2163cf78" Jul 06 15:35:57.631060 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:57.631025 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-662qp" event={"ID":"966f6ac8-fde8-46e3-b9fe-816b6f334efa","Type":"ContainerStarted","Data":"869b48607287352957be6c7d28b4e7267bf89d0d43879f8d7e6dfe89b2f7f55b"} Jul 06 15:35:58.981224 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:58.981193 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-8465bdc94b-6kxjv" Jul 06 15:35:58.985003 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:35:58.984980 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-8465bdc94b-6kxjv" Jul 06 15:36:06.633091 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:36:06.633047 2563 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-749448d4c7-6qfwt" podUID="13924c76-bb59-4b1f-ad02-929e453aeeed" containerName="console" containerID="cri-o://d30ff9f3c26a7c8f5a1543c4cc935f448a1a1d038e63c4693a7a6869c32b3ce7" gracePeriod=15 Jul 06 15:36:06.886818 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:36:06.886764 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-749448d4c7-6qfwt" Jul 06 15:36:06.969290 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:36:06.969263 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jhc2j\" (UniqueName: \"kubernetes.io/projected/13924c76-bb59-4b1f-ad02-929e453aeeed-kube-api-access-jhc2j\") pod \"13924c76-bb59-4b1f-ad02-929e453aeeed\" (UID: \"13924c76-bb59-4b1f-ad02-929e453aeeed\") " Jul 06 15:36:06.969492 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:36:06.969311 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/13924c76-bb59-4b1f-ad02-929e453aeeed-service-ca\") pod \"13924c76-bb59-4b1f-ad02-929e453aeeed\" (UID: \"13924c76-bb59-4b1f-ad02-929e453aeeed\") " Jul 06 15:36:06.969492 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:36:06.969329 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/13924c76-bb59-4b1f-ad02-929e453aeeed-oauth-serving-cert\") pod \"13924c76-bb59-4b1f-ad02-929e453aeeed\" (UID: \"13924c76-bb59-4b1f-ad02-929e453aeeed\") " Jul 06 15:36:06.969492 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:36:06.969371 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/13924c76-bb59-4b1f-ad02-929e453aeeed-console-config\") pod \"13924c76-bb59-4b1f-ad02-929e453aeeed\" (UID: \"13924c76-bb59-4b1f-ad02-929e453aeeed\") " Jul 06 15:36:06.969492 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:36:06.969451 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/13924c76-bb59-4b1f-ad02-929e453aeeed-console-serving-cert\") pod \"13924c76-bb59-4b1f-ad02-929e453aeeed\" (UID: \"13924c76-bb59-4b1f-ad02-929e453aeeed\") " Jul 06 15:36:06.969492 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:36:06.969480 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/13924c76-bb59-4b1f-ad02-929e453aeeed-console-oauth-config\") pod \"13924c76-bb59-4b1f-ad02-929e453aeeed\" (UID: \"13924c76-bb59-4b1f-ad02-929e453aeeed\") " Jul 06 15:36:06.969885 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:36:06.969830 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/13924c76-bb59-4b1f-ad02-929e453aeeed-service-ca" (OuterVolumeSpecName: "service-ca") pod "13924c76-bb59-4b1f-ad02-929e453aeeed" (UID: "13924c76-bb59-4b1f-ad02-929e453aeeed"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 15:36:06.969885 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:36:06.969850 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/13924c76-bb59-4b1f-ad02-929e453aeeed-console-config" (OuterVolumeSpecName: "console-config") pod "13924c76-bb59-4b1f-ad02-929e453aeeed" (UID: "13924c76-bb59-4b1f-ad02-929e453aeeed"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 15:36:06.970042 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:36:06.969912 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/13924c76-bb59-4b1f-ad02-929e453aeeed-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "13924c76-bb59-4b1f-ad02-929e453aeeed" (UID: "13924c76-bb59-4b1f-ad02-929e453aeeed"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 15:36:06.971833 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:36:06.971807 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/13924c76-bb59-4b1f-ad02-929e453aeeed-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "13924c76-bb59-4b1f-ad02-929e453aeeed" (UID: "13924c76-bb59-4b1f-ad02-929e453aeeed"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 15:36:06.971926 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:36:06.971899 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/13924c76-bb59-4b1f-ad02-929e453aeeed-kube-api-access-jhc2j" (OuterVolumeSpecName: "kube-api-access-jhc2j") pod "13924c76-bb59-4b1f-ad02-929e453aeeed" (UID: "13924c76-bb59-4b1f-ad02-929e453aeeed"). InnerVolumeSpecName "kube-api-access-jhc2j". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 15:36:06.971926 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:36:06.971902 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/13924c76-bb59-4b1f-ad02-929e453aeeed-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "13924c76-bb59-4b1f-ad02-929e453aeeed" (UID: "13924c76-bb59-4b1f-ad02-929e453aeeed"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 15:36:07.070354 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:36:07.070302 2563 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/13924c76-bb59-4b1f-ad02-929e453aeeed-console-serving-cert\") on node \"ip-10-0-129-154.ec2.internal\" DevicePath \"\"" Jul 06 15:36:07.070354 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:36:07.070351 2563 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/13924c76-bb59-4b1f-ad02-929e453aeeed-console-oauth-config\") on node \"ip-10-0-129-154.ec2.internal\" DevicePath \"\"" Jul 06 15:36:07.070354 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:36:07.070362 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jhc2j\" (UniqueName: \"kubernetes.io/projected/13924c76-bb59-4b1f-ad02-929e453aeeed-kube-api-access-jhc2j\") on node \"ip-10-0-129-154.ec2.internal\" DevicePath \"\"" Jul 06 15:36:07.070354 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:36:07.070371 2563 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/13924c76-bb59-4b1f-ad02-929e453aeeed-service-ca\") on node \"ip-10-0-129-154.ec2.internal\" DevicePath \"\"" Jul 06 15:36:07.070636 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:36:07.070379 2563 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/13924c76-bb59-4b1f-ad02-929e453aeeed-oauth-serving-cert\") on node \"ip-10-0-129-154.ec2.internal\" DevicePath \"\"" Jul 06 15:36:07.070636 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:36:07.070388 2563 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/13924c76-bb59-4b1f-ad02-929e453aeeed-console-config\") on node \"ip-10-0-129-154.ec2.internal\" DevicePath \"\"" Jul 06 15:36:07.673021 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:36:07.672985 2563 generic.go:356] "Generic (PLEG): container finished" podID="13924c76-bb59-4b1f-ad02-929e453aeeed" containerID="d30ff9f3c26a7c8f5a1543c4cc935f448a1a1d038e63c4693a7a6869c32b3ce7" exitCode=0 Jul 06 15:36:07.673402 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:36:07.673030 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-749448d4c7-6qfwt" Jul 06 15:36:07.673402 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:36:07.673078 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-749448d4c7-6qfwt" event={"ID":"13924c76-bb59-4b1f-ad02-929e453aeeed","Type":"ContainerDied","Data":"d30ff9f3c26a7c8f5a1543c4cc935f448a1a1d038e63c4693a7a6869c32b3ce7"} Jul 06 15:36:07.673402 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:36:07.673131 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-749448d4c7-6qfwt" event={"ID":"13924c76-bb59-4b1f-ad02-929e453aeeed","Type":"ContainerDied","Data":"5e9d634e251e2ed23b4ea236225967de84e9b960a9d1f6ef2b0145d6df025413"} Jul 06 15:36:07.673402 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:36:07.673153 2563 scope.go:117] "RemoveContainer" containerID="d30ff9f3c26a7c8f5a1543c4cc935f448a1a1d038e63c4693a7a6869c32b3ce7" Jul 06 15:36:07.682219 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:36:07.682200 2563 scope.go:117] "RemoveContainer" containerID="d30ff9f3c26a7c8f5a1543c4cc935f448a1a1d038e63c4693a7a6869c32b3ce7" Jul 06 15:36:07.682488 ip-10-0-129-154 kubenswrapper[2563]: E0706 15:36:07.682468 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d30ff9f3c26a7c8f5a1543c4cc935f448a1a1d038e63c4693a7a6869c32b3ce7\": container with ID starting with d30ff9f3c26a7c8f5a1543c4cc935f448a1a1d038e63c4693a7a6869c32b3ce7 not found: ID does not exist" containerID="d30ff9f3c26a7c8f5a1543c4cc935f448a1a1d038e63c4693a7a6869c32b3ce7" Jul 06 15:36:07.682566 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:36:07.682496 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d30ff9f3c26a7c8f5a1543c4cc935f448a1a1d038e63c4693a7a6869c32b3ce7"} err="failed to get container status \"d30ff9f3c26a7c8f5a1543c4cc935f448a1a1d038e63c4693a7a6869c32b3ce7\": rpc error: code = NotFound desc = could not find container \"d30ff9f3c26a7c8f5a1543c4cc935f448a1a1d038e63c4693a7a6869c32b3ce7\": container with ID starting with d30ff9f3c26a7c8f5a1543c4cc935f448a1a1d038e63c4693a7a6869c32b3ce7 not found: ID does not exist" Jul 06 15:36:07.694636 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:36:07.694612 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-749448d4c7-6qfwt"] Jul 06 15:36:07.698546 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:36:07.698525 2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-749448d4c7-6qfwt"] Jul 06 15:36:07.945855 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:36:07.945755 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="13924c76-bb59-4b1f-ad02-929e453aeeed" path="/var/lib/kubelet/pods/13924c76-bb59-4b1f-ad02-929e453aeeed/volumes" Jul 06 15:36:43.015249 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:36:43.015166 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-58b4579674-7h6d5"] Jul 06 15:36:43.016263 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:36:43.016240 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1911fa1c-48ce-44ab-8b64-7ae70ce80a0d" containerName="registry" Jul 06 15:36:43.016382 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:36:43.016371 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="1911fa1c-48ce-44ab-8b64-7ae70ce80a0d" containerName="registry" Jul 06 15:36:43.016536 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:36:43.016524 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="13924c76-bb59-4b1f-ad02-929e453aeeed" containerName="console" Jul 06 15:36:43.016617 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:36:43.016584 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="13924c76-bb59-4b1f-ad02-929e453aeeed" containerName="console" Jul 06 15:36:43.016688 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:36:43.016678 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="1911fa1c-48ce-44ab-8b64-7ae70ce80a0d" containerName="registry" Jul 06 15:36:43.016739 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:36:43.016696 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="13924c76-bb59-4b1f-ad02-929e453aeeed" containerName="console" Jul 06 15:36:43.019877 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:36:43.019856 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-58b4579674-7h6d5" Jul 06 15:36:43.049869 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:36:43.049832 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-58b4579674-7h6d5"] Jul 06 15:36:43.189868 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:36:43.189815 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/006dd985-e1c3-4e9f-a858-c5d77b5173f6-oauth-serving-cert\") pod \"console-58b4579674-7h6d5\" (UID: \"006dd985-e1c3-4e9f-a858-c5d77b5173f6\") " pod="openshift-console/console-58b4579674-7h6d5" Jul 06 15:36:43.190055 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:36:43.189893 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/006dd985-e1c3-4e9f-a858-c5d77b5173f6-console-serving-cert\") pod \"console-58b4579674-7h6d5\" (UID: \"006dd985-e1c3-4e9f-a858-c5d77b5173f6\") " pod="openshift-console/console-58b4579674-7h6d5" Jul 06 15:36:43.190055 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:36:43.189932 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rhw48\" (UniqueName: \"kubernetes.io/projected/006dd985-e1c3-4e9f-a858-c5d77b5173f6-kube-api-access-rhw48\") pod \"console-58b4579674-7h6d5\" (UID: \"006dd985-e1c3-4e9f-a858-c5d77b5173f6\") " pod="openshift-console/console-58b4579674-7h6d5" Jul 06 15:36:43.190055 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:36:43.189997 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/006dd985-e1c3-4e9f-a858-c5d77b5173f6-console-oauth-config\") pod \"console-58b4579674-7h6d5\" (UID: \"006dd985-e1c3-4e9f-a858-c5d77b5173f6\") " pod="openshift-console/console-58b4579674-7h6d5" Jul 06 15:36:43.190055 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:36:43.190020 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/006dd985-e1c3-4e9f-a858-c5d77b5173f6-trusted-ca-bundle\") pod \"console-58b4579674-7h6d5\" (UID: \"006dd985-e1c3-4e9f-a858-c5d77b5173f6\") " pod="openshift-console/console-58b4579674-7h6d5" Jul 06 15:36:43.190220 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:36:43.190135 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/006dd985-e1c3-4e9f-a858-c5d77b5173f6-service-ca\") pod \"console-58b4579674-7h6d5\" (UID: \"006dd985-e1c3-4e9f-a858-c5d77b5173f6\") " pod="openshift-console/console-58b4579674-7h6d5" Jul 06 15:36:43.190220 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:36:43.190184 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/006dd985-e1c3-4e9f-a858-c5d77b5173f6-console-config\") pod \"console-58b4579674-7h6d5\" (UID: \"006dd985-e1c3-4e9f-a858-c5d77b5173f6\") " pod="openshift-console/console-58b4579674-7h6d5" Jul 06 15:36:43.291398 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:36:43.291311 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rhw48\" (UniqueName: \"kubernetes.io/projected/006dd985-e1c3-4e9f-a858-c5d77b5173f6-kube-api-access-rhw48\") pod \"console-58b4579674-7h6d5\" (UID: \"006dd985-e1c3-4e9f-a858-c5d77b5173f6\") " pod="openshift-console/console-58b4579674-7h6d5" Jul 06 15:36:43.291398 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:36:43.291356 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/006dd985-e1c3-4e9f-a858-c5d77b5173f6-console-oauth-config\") pod \"console-58b4579674-7h6d5\" (UID: \"006dd985-e1c3-4e9f-a858-c5d77b5173f6\") " pod="openshift-console/console-58b4579674-7h6d5" Jul 06 15:36:43.291398 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:36:43.291372 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/006dd985-e1c3-4e9f-a858-c5d77b5173f6-trusted-ca-bundle\") pod \"console-58b4579674-7h6d5\" (UID: \"006dd985-e1c3-4e9f-a858-c5d77b5173f6\") " pod="openshift-console/console-58b4579674-7h6d5" Jul 06 15:36:43.291705 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:36:43.291406 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/006dd985-e1c3-4e9f-a858-c5d77b5173f6-service-ca\") pod \"console-58b4579674-7h6d5\" (UID: \"006dd985-e1c3-4e9f-a858-c5d77b5173f6\") " pod="openshift-console/console-58b4579674-7h6d5" Jul 06 15:36:43.291705 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:36:43.291610 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/006dd985-e1c3-4e9f-a858-c5d77b5173f6-console-config\") pod \"console-58b4579674-7h6d5\" (UID: \"006dd985-e1c3-4e9f-a858-c5d77b5173f6\") " pod="openshift-console/console-58b4579674-7h6d5" Jul 06 15:36:43.291705 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:36:43.291686 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/006dd985-e1c3-4e9f-a858-c5d77b5173f6-oauth-serving-cert\") pod \"console-58b4579674-7h6d5\" (UID: \"006dd985-e1c3-4e9f-a858-c5d77b5173f6\") " pod="openshift-console/console-58b4579674-7h6d5" Jul 06 15:36:43.291857 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:36:43.291733 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/006dd985-e1c3-4e9f-a858-c5d77b5173f6-console-serving-cert\") pod \"console-58b4579674-7h6d5\" (UID: \"006dd985-e1c3-4e9f-a858-c5d77b5173f6\") " pod="openshift-console/console-58b4579674-7h6d5" Jul 06 15:36:43.292250 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:36:43.292223 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/006dd985-e1c3-4e9f-a858-c5d77b5173f6-service-ca\") pod \"console-58b4579674-7h6d5\" (UID: \"006dd985-e1c3-4e9f-a858-c5d77b5173f6\") " pod="openshift-console/console-58b4579674-7h6d5" Jul 06 15:36:43.292617 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:36:43.292327 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/006dd985-e1c3-4e9f-a858-c5d77b5173f6-console-config\") pod \"console-58b4579674-7h6d5\" (UID: \"006dd985-e1c3-4e9f-a858-c5d77b5173f6\") " pod="openshift-console/console-58b4579674-7h6d5" Jul 06 15:36:43.292617 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:36:43.292390 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/006dd985-e1c3-4e9f-a858-c5d77b5173f6-oauth-serving-cert\") pod \"console-58b4579674-7h6d5\" (UID: \"006dd985-e1c3-4e9f-a858-c5d77b5173f6\") " pod="openshift-console/console-58b4579674-7h6d5" Jul 06 15:36:43.292735 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:36:43.292401 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/006dd985-e1c3-4e9f-a858-c5d77b5173f6-trusted-ca-bundle\") pod \"console-58b4579674-7h6d5\" (UID: \"006dd985-e1c3-4e9f-a858-c5d77b5173f6\") " pod="openshift-console/console-58b4579674-7h6d5" Jul 06 15:36:43.293932 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:36:43.293911 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/006dd985-e1c3-4e9f-a858-c5d77b5173f6-console-oauth-config\") pod \"console-58b4579674-7h6d5\" (UID: \"006dd985-e1c3-4e9f-a858-c5d77b5173f6\") " pod="openshift-console/console-58b4579674-7h6d5" Jul 06 15:36:43.294210 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:36:43.294193 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/006dd985-e1c3-4e9f-a858-c5d77b5173f6-console-serving-cert\") pod \"console-58b4579674-7h6d5\" (UID: \"006dd985-e1c3-4e9f-a858-c5d77b5173f6\") " pod="openshift-console/console-58b4579674-7h6d5" Jul 06 15:36:43.300245 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:36:43.300221 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rhw48\" (UniqueName: \"kubernetes.io/projected/006dd985-e1c3-4e9f-a858-c5d77b5173f6-kube-api-access-rhw48\") pod \"console-58b4579674-7h6d5\" (UID: \"006dd985-e1c3-4e9f-a858-c5d77b5173f6\") " pod="openshift-console/console-58b4579674-7h6d5" Jul 06 15:36:43.331234 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:36:43.331197 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-58b4579674-7h6d5" Jul 06 15:36:43.487806 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:36:43.487775 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-58b4579674-7h6d5"] Jul 06 15:36:43.491488 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:36:43.491460 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod006dd985_e1c3_4e9f_a858_c5d77b5173f6.slice/crio-144543a83be3cfeb3098ec98c6d7127c6f9ce9336ec31f026ee6049781932af7 WatchSource:0}: Error finding container 144543a83be3cfeb3098ec98c6d7127c6f9ce9336ec31f026ee6049781932af7: Status 404 returned error can't find the container with id 144543a83be3cfeb3098ec98c6d7127c6f9ce9336ec31f026ee6049781932af7 Jul 06 15:36:43.820973 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:36:43.820898 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-58b4579674-7h6d5" event={"ID":"006dd985-e1c3-4e9f-a858-c5d77b5173f6","Type":"ContainerStarted","Data":"7d48116fcb15fbce533159e23f1dbff40177dab28a0cd8890fa3ca435ba5bf15"} Jul 06 15:36:43.820973 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:36:43.820961 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-58b4579674-7h6d5" event={"ID":"006dd985-e1c3-4e9f-a858-c5d77b5173f6","Type":"ContainerStarted","Data":"144543a83be3cfeb3098ec98c6d7127c6f9ce9336ec31f026ee6049781932af7"} Jul 06 15:36:43.846999 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:36:43.846955 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-58b4579674-7h6d5" podStartSLOduration=1.846940074 podStartE2EDuration="1.846940074s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 15:36:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 15:36:43.84547182 +0000 UTC m=+180.354704995" watchObservedRunningTime="2026-07-06 15:36:43.846940074 +0000 UTC m=+180.356173239" Jul 06 15:36:53.331503 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:36:53.331463 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-58b4579674-7h6d5" Jul 06 15:36:53.331936 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:36:53.331526 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-58b4579674-7h6d5" Jul 06 15:36:53.337244 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:36:53.337220 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-58b4579674-7h6d5" Jul 06 15:36:53.866263 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:36:53.866235 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-58b4579674-7h6d5" Jul 06 15:36:53.918720 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:36:53.918694 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7578959f54-4tp2s"] Jul 06 15:37:18.942373 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:37:18.942316 2563 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-7578959f54-4tp2s" podUID="c0d108a0-96a4-4660-9421-c3e5dca56c02" containerName="console" containerID="cri-o://f07eab0be0266c22f5fc392b43bf7a7612e543f219990dc7cd572c75ebf0aa30" gracePeriod=15 Jul 06 15:37:19.196022 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:37:19.195964 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7578959f54-4tp2s" Jul 06 15:37:19.282968 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:37:19.282937 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c0d108a0-96a4-4660-9421-c3e5dca56c02-service-ca\") pod \"c0d108a0-96a4-4660-9421-c3e5dca56c02\" (UID: \"c0d108a0-96a4-4660-9421-c3e5dca56c02\") " Jul 06 15:37:19.283124 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:37:19.283026 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c0d108a0-96a4-4660-9421-c3e5dca56c02-trusted-ca-bundle\") pod \"c0d108a0-96a4-4660-9421-c3e5dca56c02\" (UID: \"c0d108a0-96a4-4660-9421-c3e5dca56c02\") " Jul 06 15:37:19.283124 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:37:19.283052 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c0d108a0-96a4-4660-9421-c3e5dca56c02-console-oauth-config\") pod \"c0d108a0-96a4-4660-9421-c3e5dca56c02\" (UID: \"c0d108a0-96a4-4660-9421-c3e5dca56c02\") " Jul 06 15:37:19.283124 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:37:19.283087 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c0d108a0-96a4-4660-9421-c3e5dca56c02-console-serving-cert\") pod \"c0d108a0-96a4-4660-9421-c3e5dca56c02\" (UID: \"c0d108a0-96a4-4660-9421-c3e5dca56c02\") " Jul 06 15:37:19.283124 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:37:19.283107 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c0d108a0-96a4-4660-9421-c3e5dca56c02-console-config\") pod \"c0d108a0-96a4-4660-9421-c3e5dca56c02\" (UID: \"c0d108a0-96a4-4660-9421-c3e5dca56c02\") " Jul 06 15:37:19.283324 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:37:19.283127 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c0d108a0-96a4-4660-9421-c3e5dca56c02-oauth-serving-cert\") pod \"c0d108a0-96a4-4660-9421-c3e5dca56c02\" (UID: \"c0d108a0-96a4-4660-9421-c3e5dca56c02\") " Jul 06 15:37:19.283324 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:37:19.283143 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5pnb4\" (UniqueName: \"kubernetes.io/projected/c0d108a0-96a4-4660-9421-c3e5dca56c02-kube-api-access-5pnb4\") pod \"c0d108a0-96a4-4660-9421-c3e5dca56c02\" (UID: \"c0d108a0-96a4-4660-9421-c3e5dca56c02\") " Jul 06 15:37:19.283454 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:37:19.283351 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c0d108a0-96a4-4660-9421-c3e5dca56c02-service-ca" (OuterVolumeSpecName: "service-ca") pod "c0d108a0-96a4-4660-9421-c3e5dca56c02" (UID: "c0d108a0-96a4-4660-9421-c3e5dca56c02"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 15:37:19.283616 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:37:19.283591 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c0d108a0-96a4-4660-9421-c3e5dca56c02-console-config" (OuterVolumeSpecName: "console-config") pod "c0d108a0-96a4-4660-9421-c3e5dca56c02" (UID: "c0d108a0-96a4-4660-9421-c3e5dca56c02"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 15:37:19.283616 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:37:19.283599 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c0d108a0-96a4-4660-9421-c3e5dca56c02-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "c0d108a0-96a4-4660-9421-c3e5dca56c02" (UID: "c0d108a0-96a4-4660-9421-c3e5dca56c02"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 15:37:19.283714 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:37:19.283610 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c0d108a0-96a4-4660-9421-c3e5dca56c02-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "c0d108a0-96a4-4660-9421-c3e5dca56c02" (UID: "c0d108a0-96a4-4660-9421-c3e5dca56c02"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 15:37:19.285273 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:37:19.285254 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c0d108a0-96a4-4660-9421-c3e5dca56c02-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "c0d108a0-96a4-4660-9421-c3e5dca56c02" (UID: "c0d108a0-96a4-4660-9421-c3e5dca56c02"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 15:37:19.285375 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:37:19.285350 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c0d108a0-96a4-4660-9421-c3e5dca56c02-kube-api-access-5pnb4" (OuterVolumeSpecName: "kube-api-access-5pnb4") pod "c0d108a0-96a4-4660-9421-c3e5dca56c02" (UID: "c0d108a0-96a4-4660-9421-c3e5dca56c02"). InnerVolumeSpecName "kube-api-access-5pnb4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 15:37:19.285491 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:37:19.285470 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c0d108a0-96a4-4660-9421-c3e5dca56c02-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "c0d108a0-96a4-4660-9421-c3e5dca56c02" (UID: "c0d108a0-96a4-4660-9421-c3e5dca56c02"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 15:37:19.384400 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:37:19.384359 2563 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c0d108a0-96a4-4660-9421-c3e5dca56c02-trusted-ca-bundle\") on node \"ip-10-0-129-154.ec2.internal\" DevicePath \"\"" Jul 06 15:37:19.384400 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:37:19.384391 2563 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c0d108a0-96a4-4660-9421-c3e5dca56c02-console-oauth-config\") on node \"ip-10-0-129-154.ec2.internal\" DevicePath \"\"" Jul 06 15:37:19.384400 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:37:19.384403 2563 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c0d108a0-96a4-4660-9421-c3e5dca56c02-console-serving-cert\") on node \"ip-10-0-129-154.ec2.internal\" DevicePath \"\"" Jul 06 15:37:19.384724 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:37:19.384446 2563 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c0d108a0-96a4-4660-9421-c3e5dca56c02-console-config\") on node \"ip-10-0-129-154.ec2.internal\" DevicePath \"\"" Jul 06 15:37:19.384724 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:37:19.384462 2563 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c0d108a0-96a4-4660-9421-c3e5dca56c02-oauth-serving-cert\") on node \"ip-10-0-129-154.ec2.internal\" DevicePath \"\"" Jul 06 15:37:19.384724 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:37:19.384473 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5pnb4\" (UniqueName: \"kubernetes.io/projected/c0d108a0-96a4-4660-9421-c3e5dca56c02-kube-api-access-5pnb4\") on node \"ip-10-0-129-154.ec2.internal\" DevicePath \"\"" Jul 06 15:37:19.384724 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:37:19.384486 2563 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c0d108a0-96a4-4660-9421-c3e5dca56c02-service-ca\") on node \"ip-10-0-129-154.ec2.internal\" DevicePath \"\"" Jul 06 15:37:19.964983 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:37:19.964950 2563 generic.go:356] "Generic (PLEG): container finished" podID="c0d108a0-96a4-4660-9421-c3e5dca56c02" containerID="f07eab0be0266c22f5fc392b43bf7a7612e543f219990dc7cd572c75ebf0aa30" exitCode=0 Jul 06 15:37:19.965282 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:37:19.964995 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7578959f54-4tp2s" Jul 06 15:37:19.965282 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:37:19.965033 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7578959f54-4tp2s" event={"ID":"c0d108a0-96a4-4660-9421-c3e5dca56c02","Type":"ContainerDied","Data":"f07eab0be0266c22f5fc392b43bf7a7612e543f219990dc7cd572c75ebf0aa30"} Jul 06 15:37:19.965282 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:37:19.965083 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7578959f54-4tp2s" event={"ID":"c0d108a0-96a4-4660-9421-c3e5dca56c02","Type":"ContainerDied","Data":"6c562c049e2411a72ea82d22b1364ea29a83ceddf714e36d1968c47697c583b1"} Jul 06 15:37:19.965282 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:37:19.965103 2563 scope.go:117] "RemoveContainer" containerID="f07eab0be0266c22f5fc392b43bf7a7612e543f219990dc7cd572c75ebf0aa30" Jul 06 15:37:19.973345 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:37:19.973321 2563 scope.go:117] "RemoveContainer" containerID="f07eab0be0266c22f5fc392b43bf7a7612e543f219990dc7cd572c75ebf0aa30" Jul 06 15:37:19.973640 ip-10-0-129-154 kubenswrapper[2563]: E0706 15:37:19.973621 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f07eab0be0266c22f5fc392b43bf7a7612e543f219990dc7cd572c75ebf0aa30\": container with ID starting with f07eab0be0266c22f5fc392b43bf7a7612e543f219990dc7cd572c75ebf0aa30 not found: ID does not exist" containerID="f07eab0be0266c22f5fc392b43bf7a7612e543f219990dc7cd572c75ebf0aa30" Jul 06 15:37:19.973732 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:37:19.973651 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f07eab0be0266c22f5fc392b43bf7a7612e543f219990dc7cd572c75ebf0aa30"} err="failed to get container status \"f07eab0be0266c22f5fc392b43bf7a7612e543f219990dc7cd572c75ebf0aa30\": rpc error: code = NotFound desc = could not find container \"f07eab0be0266c22f5fc392b43bf7a7612e543f219990dc7cd572c75ebf0aa30\": container with ID starting with f07eab0be0266c22f5fc392b43bf7a7612e543f219990dc7cd572c75ebf0aa30 not found: ID does not exist" Jul 06 15:37:19.983091 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:37:19.983067 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7578959f54-4tp2s"] Jul 06 15:37:19.984894 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:37:19.984872 2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-7578959f54-4tp2s"] Jul 06 15:37:21.944203 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:37:21.944171 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c0d108a0-96a4-4660-9421-c3e5dca56c02" path="/var/lib/kubelet/pods/c0d108a0-96a4-4660-9421-c3e5dca56c02/volumes" Jul 06 15:37:58.683024 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:37:58.682955 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-d9b4d8c44-zxfhk"] Jul 06 15:37:58.683525 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:37:58.683476 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c0d108a0-96a4-4660-9421-c3e5dca56c02" containerName="console" Jul 06 15:37:58.683525 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:37:58.683497 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="c0d108a0-96a4-4660-9421-c3e5dca56c02" containerName="console" Jul 06 15:37:58.683605 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:37:58.683586 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="c0d108a0-96a4-4660-9421-c3e5dca56c02" containerName="console" Jul 06 15:37:58.687970 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:37:58.687946 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-d9b4d8c44-zxfhk" Jul 06 15:37:58.697898 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:37:58.697871 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-d9b4d8c44-zxfhk"] Jul 06 15:37:58.712363 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:37:58.712337 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3cf66f47-ed5b-46c0-8728-9ac64f2ccdad-service-ca\") pod \"console-d9b4d8c44-zxfhk\" (UID: \"3cf66f47-ed5b-46c0-8728-9ac64f2ccdad\") " pod="openshift-console/console-d9b4d8c44-zxfhk" Jul 06 15:37:58.712524 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:37:58.712387 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3cf66f47-ed5b-46c0-8728-9ac64f2ccdad-console-serving-cert\") pod \"console-d9b4d8c44-zxfhk\" (UID: \"3cf66f47-ed5b-46c0-8728-9ac64f2ccdad\") " pod="openshift-console/console-d9b4d8c44-zxfhk" Jul 06 15:37:58.712597 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:37:58.712525 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7fjmr\" (UniqueName: \"kubernetes.io/projected/3cf66f47-ed5b-46c0-8728-9ac64f2ccdad-kube-api-access-7fjmr\") pod \"console-d9b4d8c44-zxfhk\" (UID: \"3cf66f47-ed5b-46c0-8728-9ac64f2ccdad\") " pod="openshift-console/console-d9b4d8c44-zxfhk" Jul 06 15:37:58.712657 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:37:58.712591 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3cf66f47-ed5b-46c0-8728-9ac64f2ccdad-console-oauth-config\") pod \"console-d9b4d8c44-zxfhk\" (UID: \"3cf66f47-ed5b-46c0-8728-9ac64f2ccdad\") " pod="openshift-console/console-d9b4d8c44-zxfhk" Jul 06 15:37:58.712657 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:37:58.712643 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3cf66f47-ed5b-46c0-8728-9ac64f2ccdad-oauth-serving-cert\") pod \"console-d9b4d8c44-zxfhk\" (UID: \"3cf66f47-ed5b-46c0-8728-9ac64f2ccdad\") " pod="openshift-console/console-d9b4d8c44-zxfhk" Jul 06 15:37:58.712810 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:37:58.712677 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3cf66f47-ed5b-46c0-8728-9ac64f2ccdad-console-config\") pod \"console-d9b4d8c44-zxfhk\" (UID: \"3cf66f47-ed5b-46c0-8728-9ac64f2ccdad\") " pod="openshift-console/console-d9b4d8c44-zxfhk" Jul 06 15:37:58.712810 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:37:58.712704 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3cf66f47-ed5b-46c0-8728-9ac64f2ccdad-trusted-ca-bundle\") pod \"console-d9b4d8c44-zxfhk\" (UID: \"3cf66f47-ed5b-46c0-8728-9ac64f2ccdad\") " pod="openshift-console/console-d9b4d8c44-zxfhk" Jul 06 15:37:58.813246 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:37:58.813211 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3cf66f47-ed5b-46c0-8728-9ac64f2ccdad-service-ca\") pod \"console-d9b4d8c44-zxfhk\" (UID: \"3cf66f47-ed5b-46c0-8728-9ac64f2ccdad\") " pod="openshift-console/console-d9b4d8c44-zxfhk" Jul 06 15:37:58.813246 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:37:58.813255 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3cf66f47-ed5b-46c0-8728-9ac64f2ccdad-console-serving-cert\") pod \"console-d9b4d8c44-zxfhk\" (UID: \"3cf66f47-ed5b-46c0-8728-9ac64f2ccdad\") " pod="openshift-console/console-d9b4d8c44-zxfhk" Jul 06 15:37:58.813554 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:37:58.813292 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7fjmr\" (UniqueName: \"kubernetes.io/projected/3cf66f47-ed5b-46c0-8728-9ac64f2ccdad-kube-api-access-7fjmr\") pod \"console-d9b4d8c44-zxfhk\" (UID: \"3cf66f47-ed5b-46c0-8728-9ac64f2ccdad\") " pod="openshift-console/console-d9b4d8c44-zxfhk" Jul 06 15:37:58.813554 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:37:58.813319 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3cf66f47-ed5b-46c0-8728-9ac64f2ccdad-console-oauth-config\") pod \"console-d9b4d8c44-zxfhk\" (UID: \"3cf66f47-ed5b-46c0-8728-9ac64f2ccdad\") " pod="openshift-console/console-d9b4d8c44-zxfhk" Jul 06 15:37:58.813554 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:37:58.813351 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3cf66f47-ed5b-46c0-8728-9ac64f2ccdad-oauth-serving-cert\") pod \"console-d9b4d8c44-zxfhk\" (UID: \"3cf66f47-ed5b-46c0-8728-9ac64f2ccdad\") " pod="openshift-console/console-d9b4d8c44-zxfhk" Jul 06 15:37:58.813554 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:37:58.813381 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3cf66f47-ed5b-46c0-8728-9ac64f2ccdad-console-config\") pod \"console-d9b4d8c44-zxfhk\" (UID: \"3cf66f47-ed5b-46c0-8728-9ac64f2ccdad\") " pod="openshift-console/console-d9b4d8c44-zxfhk" Jul 06 15:37:58.813554 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:37:58.813406 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3cf66f47-ed5b-46c0-8728-9ac64f2ccdad-trusted-ca-bundle\") pod \"console-d9b4d8c44-zxfhk\" (UID: \"3cf66f47-ed5b-46c0-8728-9ac64f2ccdad\") " pod="openshift-console/console-d9b4d8c44-zxfhk" Jul 06 15:37:58.814103 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:37:58.814076 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3cf66f47-ed5b-46c0-8728-9ac64f2ccdad-service-ca\") pod \"console-d9b4d8c44-zxfhk\" (UID: \"3cf66f47-ed5b-46c0-8728-9ac64f2ccdad\") " pod="openshift-console/console-d9b4d8c44-zxfhk" Jul 06 15:37:58.814227 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:37:58.814175 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3cf66f47-ed5b-46c0-8728-9ac64f2ccdad-oauth-serving-cert\") pod \"console-d9b4d8c44-zxfhk\" (UID: \"3cf66f47-ed5b-46c0-8728-9ac64f2ccdad\") " pod="openshift-console/console-d9b4d8c44-zxfhk" Jul 06 15:37:58.814227 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:37:58.814184 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3cf66f47-ed5b-46c0-8728-9ac64f2ccdad-console-config\") pod \"console-d9b4d8c44-zxfhk\" (UID: \"3cf66f47-ed5b-46c0-8728-9ac64f2ccdad\") " pod="openshift-console/console-d9b4d8c44-zxfhk" Jul 06 15:37:58.814325 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:37:58.814307 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3cf66f47-ed5b-46c0-8728-9ac64f2ccdad-trusted-ca-bundle\") pod \"console-d9b4d8c44-zxfhk\" (UID: \"3cf66f47-ed5b-46c0-8728-9ac64f2ccdad\") " pod="openshift-console/console-d9b4d8c44-zxfhk" Jul 06 15:37:58.816621 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:37:58.816593 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3cf66f47-ed5b-46c0-8728-9ac64f2ccdad-console-oauth-config\") pod \"console-d9b4d8c44-zxfhk\" (UID: \"3cf66f47-ed5b-46c0-8728-9ac64f2ccdad\") " pod="openshift-console/console-d9b4d8c44-zxfhk" Jul 06 15:37:58.816708 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:37:58.816600 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3cf66f47-ed5b-46c0-8728-9ac64f2ccdad-console-serving-cert\") pod \"console-d9b4d8c44-zxfhk\" (UID: \"3cf66f47-ed5b-46c0-8728-9ac64f2ccdad\") " pod="openshift-console/console-d9b4d8c44-zxfhk" Jul 06 15:37:58.822469 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:37:58.822447 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7fjmr\" (UniqueName: \"kubernetes.io/projected/3cf66f47-ed5b-46c0-8728-9ac64f2ccdad-kube-api-access-7fjmr\") pod \"console-d9b4d8c44-zxfhk\" (UID: \"3cf66f47-ed5b-46c0-8728-9ac64f2ccdad\") " pod="openshift-console/console-d9b4d8c44-zxfhk" Jul 06 15:37:58.997955 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:37:58.997883 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-d9b4d8c44-zxfhk" Jul 06 15:37:59.126727 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:37:59.126704 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-d9b4d8c44-zxfhk"] Jul 06 15:37:59.128603 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:37:59.128576 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3cf66f47_ed5b_46c0_8728_9ac64f2ccdad.slice/crio-f2bca1bcac8f6e8d7395e302f61a9cb292e0627222cd489010797d4d82ebd8e3 WatchSource:0}: Error finding container f2bca1bcac8f6e8d7395e302f61a9cb292e0627222cd489010797d4d82ebd8e3: Status 404 returned error can't find the container with id f2bca1bcac8f6e8d7395e302f61a9cb292e0627222cd489010797d4d82ebd8e3 Jul 06 15:38:00.123499 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:38:00.123462 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-d9b4d8c44-zxfhk" event={"ID":"3cf66f47-ed5b-46c0-8728-9ac64f2ccdad","Type":"ContainerStarted","Data":"939c033e6503b64516b03814fdaef9b360f75104e352046dccc7b3c3524ca8c7"} Jul 06 15:38:00.123910 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:38:00.123511 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-d9b4d8c44-zxfhk" event={"ID":"3cf66f47-ed5b-46c0-8728-9ac64f2ccdad","Type":"ContainerStarted","Data":"f2bca1bcac8f6e8d7395e302f61a9cb292e0627222cd489010797d4d82ebd8e3"} Jul 06 15:38:00.145023 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:38:00.144973 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-d9b4d8c44-zxfhk" podStartSLOduration=2.144958179 podStartE2EDuration="2.144958179s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 15:37:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 15:38:00.143565975 +0000 UTC m=+256.652799152" watchObservedRunningTime="2026-07-06 15:38:00.144958179 +0000 UTC m=+256.654191352" Jul 06 15:38:08.998107 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:38:08.998073 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-d9b4d8c44-zxfhk" Jul 06 15:38:08.998545 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:38:08.998142 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-d9b4d8c44-zxfhk" Jul 06 15:38:09.003068 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:38:09.003040 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-d9b4d8c44-zxfhk" Jul 06 15:38:09.165336 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:38:09.165310 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-d9b4d8c44-zxfhk" Jul 06 15:38:09.224792 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:38:09.224754 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-58b4579674-7h6d5"] Jul 06 15:38:34.247301 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:38:34.247237 2563 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-58b4579674-7h6d5" podUID="006dd985-e1c3-4e9f-a858-c5d77b5173f6" containerName="console" containerID="cri-o://7d48116fcb15fbce533159e23f1dbff40177dab28a0cd8890fa3ca435ba5bf15" gracePeriod=15 Jul 06 15:38:34.499082 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:38:34.499026 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-58b4579674-7h6d5" Jul 06 15:38:34.615931 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:38:34.615897 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/006dd985-e1c3-4e9f-a858-c5d77b5173f6-service-ca\") pod \"006dd985-e1c3-4e9f-a858-c5d77b5173f6\" (UID: \"006dd985-e1c3-4e9f-a858-c5d77b5173f6\") " Jul 06 15:38:34.615931 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:38:34.615938 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/006dd985-e1c3-4e9f-a858-c5d77b5173f6-console-config\") pod \"006dd985-e1c3-4e9f-a858-c5d77b5173f6\" (UID: \"006dd985-e1c3-4e9f-a858-c5d77b5173f6\") " Jul 06 15:38:34.616169 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:38:34.615968 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/006dd985-e1c3-4e9f-a858-c5d77b5173f6-trusted-ca-bundle\") pod \"006dd985-e1c3-4e9f-a858-c5d77b5173f6\" (UID: \"006dd985-e1c3-4e9f-a858-c5d77b5173f6\") " Jul 06 15:38:34.616169 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:38:34.615997 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/006dd985-e1c3-4e9f-a858-c5d77b5173f6-console-serving-cert\") pod \"006dd985-e1c3-4e9f-a858-c5d77b5173f6\" (UID: \"006dd985-e1c3-4e9f-a858-c5d77b5173f6\") " Jul 06 15:38:34.616169 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:38:34.616109 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/006dd985-e1c3-4e9f-a858-c5d77b5173f6-console-oauth-config\") pod \"006dd985-e1c3-4e9f-a858-c5d77b5173f6\" (UID: \"006dd985-e1c3-4e9f-a858-c5d77b5173f6\") " Jul 06 15:38:34.616169 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:38:34.616167 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rhw48\" (UniqueName: \"kubernetes.io/projected/006dd985-e1c3-4e9f-a858-c5d77b5173f6-kube-api-access-rhw48\") pod \"006dd985-e1c3-4e9f-a858-c5d77b5173f6\" (UID: \"006dd985-e1c3-4e9f-a858-c5d77b5173f6\") " Jul 06 15:38:34.616369 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:38:34.616197 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/006dd985-e1c3-4e9f-a858-c5d77b5173f6-oauth-serving-cert\") pod \"006dd985-e1c3-4e9f-a858-c5d77b5173f6\" (UID: \"006dd985-e1c3-4e9f-a858-c5d77b5173f6\") " Jul 06 15:38:34.616456 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:38:34.616390 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/006dd985-e1c3-4e9f-a858-c5d77b5173f6-console-config" (OuterVolumeSpecName: "console-config") pod "006dd985-e1c3-4e9f-a858-c5d77b5173f6" (UID: "006dd985-e1c3-4e9f-a858-c5d77b5173f6"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 15:38:34.616456 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:38:34.616403 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/006dd985-e1c3-4e9f-a858-c5d77b5173f6-service-ca" (OuterVolumeSpecName: "service-ca") pod "006dd985-e1c3-4e9f-a858-c5d77b5173f6" (UID: "006dd985-e1c3-4e9f-a858-c5d77b5173f6"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 15:38:34.616572 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:38:34.616489 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/006dd985-e1c3-4e9f-a858-c5d77b5173f6-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "006dd985-e1c3-4e9f-a858-c5d77b5173f6" (UID: "006dd985-e1c3-4e9f-a858-c5d77b5173f6"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 15:38:34.616572 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:38:34.616540 2563 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/006dd985-e1c3-4e9f-a858-c5d77b5173f6-service-ca\") on node \"ip-10-0-129-154.ec2.internal\" DevicePath \"\"" Jul 06 15:38:34.616572 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:38:34.616563 2563 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/006dd985-e1c3-4e9f-a858-c5d77b5173f6-console-config\") on node \"ip-10-0-129-154.ec2.internal\" DevicePath \"\"" Jul 06 15:38:34.616785 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:38:34.616734 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/006dd985-e1c3-4e9f-a858-c5d77b5173f6-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "006dd985-e1c3-4e9f-a858-c5d77b5173f6" (UID: "006dd985-e1c3-4e9f-a858-c5d77b5173f6"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 15:38:34.618320 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:38:34.618295 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/006dd985-e1c3-4e9f-a858-c5d77b5173f6-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "006dd985-e1c3-4e9f-a858-c5d77b5173f6" (UID: "006dd985-e1c3-4e9f-a858-c5d77b5173f6"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 15:38:34.618320 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:38:34.618310 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/006dd985-e1c3-4e9f-a858-c5d77b5173f6-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "006dd985-e1c3-4e9f-a858-c5d77b5173f6" (UID: "006dd985-e1c3-4e9f-a858-c5d77b5173f6"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 15:38:34.618464 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:38:34.618397 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/006dd985-e1c3-4e9f-a858-c5d77b5173f6-kube-api-access-rhw48" (OuterVolumeSpecName: "kube-api-access-rhw48") pod "006dd985-e1c3-4e9f-a858-c5d77b5173f6" (UID: "006dd985-e1c3-4e9f-a858-c5d77b5173f6"). InnerVolumeSpecName "kube-api-access-rhw48". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 15:38:34.717870 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:38:34.717836 2563 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/006dd985-e1c3-4e9f-a858-c5d77b5173f6-trusted-ca-bundle\") on node \"ip-10-0-129-154.ec2.internal\" DevicePath \"\"" Jul 06 15:38:34.717870 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:38:34.717867 2563 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/006dd985-e1c3-4e9f-a858-c5d77b5173f6-console-serving-cert\") on node \"ip-10-0-129-154.ec2.internal\" DevicePath \"\"" Jul 06 15:38:34.717870 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:38:34.717876 2563 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/006dd985-e1c3-4e9f-a858-c5d77b5173f6-console-oauth-config\") on node \"ip-10-0-129-154.ec2.internal\" DevicePath \"\"" Jul 06 15:38:34.718101 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:38:34.717885 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rhw48\" (UniqueName: \"kubernetes.io/projected/006dd985-e1c3-4e9f-a858-c5d77b5173f6-kube-api-access-rhw48\") on node \"ip-10-0-129-154.ec2.internal\" DevicePath \"\"" Jul 06 15:38:34.718101 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:38:34.717894 2563 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/006dd985-e1c3-4e9f-a858-c5d77b5173f6-oauth-serving-cert\") on node \"ip-10-0-129-154.ec2.internal\" DevicePath \"\"" Jul 06 15:38:35.268120 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:38:35.268083 2563 generic.go:356] "Generic (PLEG): container finished" podID="006dd985-e1c3-4e9f-a858-c5d77b5173f6" containerID="7d48116fcb15fbce533159e23f1dbff40177dab28a0cd8890fa3ca435ba5bf15" exitCode=0 Jul 06 15:38:35.268520 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:38:35.268130 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-58b4579674-7h6d5" Jul 06 15:38:35.268520 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:38:35.268165 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-58b4579674-7h6d5" event={"ID":"006dd985-e1c3-4e9f-a858-c5d77b5173f6","Type":"ContainerDied","Data":"7d48116fcb15fbce533159e23f1dbff40177dab28a0cd8890fa3ca435ba5bf15"} Jul 06 15:38:35.268520 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:38:35.268210 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-58b4579674-7h6d5" event={"ID":"006dd985-e1c3-4e9f-a858-c5d77b5173f6","Type":"ContainerDied","Data":"144543a83be3cfeb3098ec98c6d7127c6f9ce9336ec31f026ee6049781932af7"} Jul 06 15:38:35.268520 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:38:35.268232 2563 scope.go:117] "RemoveContainer" containerID="7d48116fcb15fbce533159e23f1dbff40177dab28a0cd8890fa3ca435ba5bf15" Jul 06 15:38:35.276913 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:38:35.276898 2563 scope.go:117] "RemoveContainer" containerID="7d48116fcb15fbce533159e23f1dbff40177dab28a0cd8890fa3ca435ba5bf15" Jul 06 15:38:35.277139 ip-10-0-129-154 kubenswrapper[2563]: E0706 15:38:35.277121 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7d48116fcb15fbce533159e23f1dbff40177dab28a0cd8890fa3ca435ba5bf15\": container with ID starting with 7d48116fcb15fbce533159e23f1dbff40177dab28a0cd8890fa3ca435ba5bf15 not found: ID does not exist" containerID="7d48116fcb15fbce533159e23f1dbff40177dab28a0cd8890fa3ca435ba5bf15" Jul 06 15:38:35.277184 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:38:35.277147 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7d48116fcb15fbce533159e23f1dbff40177dab28a0cd8890fa3ca435ba5bf15"} err="failed to get container status \"7d48116fcb15fbce533159e23f1dbff40177dab28a0cd8890fa3ca435ba5bf15\": rpc error: code = NotFound desc = could not find container \"7d48116fcb15fbce533159e23f1dbff40177dab28a0cd8890fa3ca435ba5bf15\": container with ID starting with 7d48116fcb15fbce533159e23f1dbff40177dab28a0cd8890fa3ca435ba5bf15 not found: ID does not exist" Jul 06 15:38:35.289863 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:38:35.289841 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-58b4579674-7h6d5"] Jul 06 15:38:35.294327 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:38:35.294305 2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-58b4579674-7h6d5"] Jul 06 15:38:35.944949 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:38:35.944915 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="006dd985-e1c3-4e9f-a858-c5d77b5173f6" path="/var/lib/kubelet/pods/006dd985-e1c3-4e9f-a858-c5d77b5173f6/volumes" Jul 06 15:38:43.815382 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:38:43.815342 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-64786d9cbb-trjfs_bf5c30e2-bf88-468b-adf2-2e8f73f136b4/cluster-monitoring-operator/0.log" Jul 06 15:38:43.818147 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:38:43.818126 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-64786d9cbb-trjfs_bf5c30e2-bf88-468b-adf2-2e8f73f136b4/cluster-monitoring-operator/0.log" Jul 06 15:38:43.818974 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:38:43.818952 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-l47bf_9ad7e968-ed2e-4cbe-9b4b-780bc9d01b88/console-operator/2.log" Jul 06 15:38:43.821729 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:38:43.821707 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-l47bf_9ad7e968-ed2e-4cbe-9b4b-780bc9d01b88/console-operator/2.log" Jul 06 15:38:43.829863 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:38:43.829846 2563 kubelet.go:1648] "Image garbage collection succeeded" Jul 06 15:40:06.177054 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:40:06.176961 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675406sqg6"] Jul 06 15:40:06.177655 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:40:06.177415 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="006dd985-e1c3-4e9f-a858-c5d77b5173f6" containerName="console" Jul 06 15:40:06.177655 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:40:06.177452 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="006dd985-e1c3-4e9f-a858-c5d77b5173f6" containerName="console" Jul 06 15:40:06.177655 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:40:06.177540 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="006dd985-e1c3-4e9f-a858-c5d77b5173f6" containerName="console" Jul 06 15:40:06.180608 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:40:06.180589 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675406sqg6" Jul 06 15:40:06.183487 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:40:06.183464 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 06 15:40:06.183626 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:40:06.183596 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-kxcwp\"" Jul 06 15:40:06.184726 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:40:06.184711 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 06 15:40:06.211785 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:40:06.206914 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675406sqg6"] Jul 06 15:40:06.265026 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:40:06.264979 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8c631f7a-9375-494f-a985-6379ce1fb375-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675406sqg6\" (UID: \"8c631f7a-9375-494f-a985-6379ce1fb375\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675406sqg6" Jul 06 15:40:06.265189 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:40:06.265079 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hfhvn\" (UniqueName: \"kubernetes.io/projected/8c631f7a-9375-494f-a985-6379ce1fb375-kube-api-access-hfhvn\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675406sqg6\" (UID: \"8c631f7a-9375-494f-a985-6379ce1fb375\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675406sqg6" Jul 06 15:40:06.265189 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:40:06.265101 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8c631f7a-9375-494f-a985-6379ce1fb375-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675406sqg6\" (UID: \"8c631f7a-9375-494f-a985-6379ce1fb375\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675406sqg6" Jul 06 15:40:06.366483 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:40:06.366400 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8c631f7a-9375-494f-a985-6379ce1fb375-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675406sqg6\" (UID: \"8c631f7a-9375-494f-a985-6379ce1fb375\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675406sqg6" Jul 06 15:40:06.366654 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:40:06.366574 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hfhvn\" (UniqueName: \"kubernetes.io/projected/8c631f7a-9375-494f-a985-6379ce1fb375-kube-api-access-hfhvn\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675406sqg6\" (UID: \"8c631f7a-9375-494f-a985-6379ce1fb375\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675406sqg6" Jul 06 15:40:06.366654 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:40:06.366613 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8c631f7a-9375-494f-a985-6379ce1fb375-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675406sqg6\" (UID: \"8c631f7a-9375-494f-a985-6379ce1fb375\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675406sqg6" Jul 06 15:40:06.366814 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:40:06.366794 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8c631f7a-9375-494f-a985-6379ce1fb375-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675406sqg6\" (UID: \"8c631f7a-9375-494f-a985-6379ce1fb375\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675406sqg6" Jul 06 15:40:06.366884 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:40:06.366875 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8c631f7a-9375-494f-a985-6379ce1fb375-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675406sqg6\" (UID: \"8c631f7a-9375-494f-a985-6379ce1fb375\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675406sqg6" Jul 06 15:40:06.376349 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:40:06.376326 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hfhvn\" (UniqueName: \"kubernetes.io/projected/8c631f7a-9375-494f-a985-6379ce1fb375-kube-api-access-hfhvn\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675406sqg6\" (UID: \"8c631f7a-9375-494f-a985-6379ce1fb375\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675406sqg6" Jul 06 15:40:06.489738 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:40:06.489642 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675406sqg6" Jul 06 15:40:06.627605 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:40:06.627583 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675406sqg6"] Jul 06 15:40:06.629160 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:40:06.629124 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8c631f7a_9375_494f_a985_6379ce1fb375.slice/crio-0c691fbe9d4a6353eb34390237a3d84490bb517a6d839c2a5585118310df93cf WatchSource:0}: Error finding container 0c691fbe9d4a6353eb34390237a3d84490bb517a6d839c2a5585118310df93cf: Status 404 returned error can't find the container with id 0c691fbe9d4a6353eb34390237a3d84490bb517a6d839c2a5585118310df93cf Jul 06 15:40:06.631406 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:40:06.631389 2563 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 06 15:40:07.639196 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:40:07.639163 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675406sqg6" event={"ID":"8c631f7a-9375-494f-a985-6379ce1fb375","Type":"ContainerStarted","Data":"0c691fbe9d4a6353eb34390237a3d84490bb517a6d839c2a5585118310df93cf"} Jul 06 15:40:14.672171 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:40:14.672131 2563 generic.go:356] "Generic (PLEG): container finished" podID="8c631f7a-9375-494f-a985-6379ce1fb375" containerID="860720118e7564d4a29f65bd3bc7c2aa82a117895489db1f3510a0dc5e82a2e5" exitCode=0 Jul 06 15:40:14.672572 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:40:14.672224 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675406sqg6" event={"ID":"8c631f7a-9375-494f-a985-6379ce1fb375","Type":"ContainerDied","Data":"860720118e7564d4a29f65bd3bc7c2aa82a117895489db1f3510a0dc5e82a2e5"} Jul 06 15:40:19.695329 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:40:19.695295 2563 generic.go:356] "Generic (PLEG): container finished" podID="8c631f7a-9375-494f-a985-6379ce1fb375" containerID="1f395aba8bc5137444ca0b1ca31d971080a9d4ae17b5be86247cb0fd82df7ba1" exitCode=0 Jul 06 15:40:19.695713 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:40:19.695366 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675406sqg6" event={"ID":"8c631f7a-9375-494f-a985-6379ce1fb375","Type":"ContainerDied","Data":"1f395aba8bc5137444ca0b1ca31d971080a9d4ae17b5be86247cb0fd82df7ba1"} Jul 06 15:40:27.738207 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:40:27.738174 2563 generic.go:356] "Generic (PLEG): container finished" podID="8c631f7a-9375-494f-a985-6379ce1fb375" containerID="f84a8f15f0511fe70ede7b59d0eae773efeb1f73f0ed8164656b5c0a79e82460" exitCode=0 Jul 06 15:40:27.738672 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:40:27.738275 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675406sqg6" event={"ID":"8c631f7a-9375-494f-a985-6379ce1fb375","Type":"ContainerDied","Data":"f84a8f15f0511fe70ede7b59d0eae773efeb1f73f0ed8164656b5c0a79e82460"} Jul 06 15:40:28.927045 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:40:28.927013 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675406sqg6" Jul 06 15:40:29.067918 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:40:29.067842 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8c631f7a-9375-494f-a985-6379ce1fb375-bundle\") pod \"8c631f7a-9375-494f-a985-6379ce1fb375\" (UID: \"8c631f7a-9375-494f-a985-6379ce1fb375\") " Jul 06 15:40:29.067918 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:40:29.067873 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8c631f7a-9375-494f-a985-6379ce1fb375-util\") pod \"8c631f7a-9375-494f-a985-6379ce1fb375\" (UID: \"8c631f7a-9375-494f-a985-6379ce1fb375\") " Jul 06 15:40:29.068100 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:40:29.067971 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hfhvn\" (UniqueName: \"kubernetes.io/projected/8c631f7a-9375-494f-a985-6379ce1fb375-kube-api-access-hfhvn\") pod \"8c631f7a-9375-494f-a985-6379ce1fb375\" (UID: \"8c631f7a-9375-494f-a985-6379ce1fb375\") " Jul 06 15:40:29.068544 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:40:29.068504 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8c631f7a-9375-494f-a985-6379ce1fb375-bundle" (OuterVolumeSpecName: "bundle") pod "8c631f7a-9375-494f-a985-6379ce1fb375" (UID: "8c631f7a-9375-494f-a985-6379ce1fb375"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 15:40:29.070267 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:40:29.070243 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8c631f7a-9375-494f-a985-6379ce1fb375-kube-api-access-hfhvn" (OuterVolumeSpecName: "kube-api-access-hfhvn") pod "8c631f7a-9375-494f-a985-6379ce1fb375" (UID: "8c631f7a-9375-494f-a985-6379ce1fb375"). InnerVolumeSpecName "kube-api-access-hfhvn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 15:40:29.071845 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:40:29.071821 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8c631f7a-9375-494f-a985-6379ce1fb375-util" (OuterVolumeSpecName: "util") pod "8c631f7a-9375-494f-a985-6379ce1fb375" (UID: "8c631f7a-9375-494f-a985-6379ce1fb375"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 15:40:29.168819 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:40:29.168788 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hfhvn\" (UniqueName: \"kubernetes.io/projected/8c631f7a-9375-494f-a985-6379ce1fb375-kube-api-access-hfhvn\") on node \"ip-10-0-129-154.ec2.internal\" DevicePath \"\"" Jul 06 15:40:29.168819 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:40:29.168814 2563 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8c631f7a-9375-494f-a985-6379ce1fb375-bundle\") on node \"ip-10-0-129-154.ec2.internal\" DevicePath \"\"" Jul 06 15:40:29.168819 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:40:29.168823 2563 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8c631f7a-9375-494f-a985-6379ce1fb375-util\") on node \"ip-10-0-129-154.ec2.internal\" DevicePath \"\"" Jul 06 15:40:29.748493 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:40:29.748453 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675406sqg6" event={"ID":"8c631f7a-9375-494f-a985-6379ce1fb375","Type":"ContainerDied","Data":"0c691fbe9d4a6353eb34390237a3d84490bb517a6d839c2a5585118310df93cf"} Jul 06 15:40:29.748493 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:40:29.748498 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675406sqg6" Jul 06 15:40:29.748752 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:40:29.748504 2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0c691fbe9d4a6353eb34390237a3d84490bb517a6d839c2a5585118310df93cf" Jul 06 15:40:35.002350 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:40:35.002322 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-8fpvb"] Jul 06 15:40:35.002758 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:40:35.002631 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8c631f7a-9375-494f-a985-6379ce1fb375" containerName="extract" Jul 06 15:40:35.002758 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:40:35.002643 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="8c631f7a-9375-494f-a985-6379ce1fb375" containerName="extract" Jul 06 15:40:35.002758 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:40:35.002653 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8c631f7a-9375-494f-a985-6379ce1fb375" containerName="pull" Jul 06 15:40:35.002758 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:40:35.002658 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="8c631f7a-9375-494f-a985-6379ce1fb375" containerName="pull" Jul 06 15:40:35.002758 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:40:35.002680 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8c631f7a-9375-494f-a985-6379ce1fb375" containerName="util" Jul 06 15:40:35.002758 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:40:35.002684 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="8c631f7a-9375-494f-a985-6379ce1fb375" containerName="util" Jul 06 15:40:35.002758 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:40:35.002734 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="8c631f7a-9375-494f-a985-6379ce1fb375" containerName="extract" Jul 06 15:40:35.005284 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:40:35.005269 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-8fpvb" Jul 06 15:40:35.008035 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:40:35.008015 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\"" Jul 06 15:40:35.008369 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:40:35.008351 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-zzqxv\"" Jul 06 15:40:35.008471 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:40:35.008446 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\"" Jul 06 15:40:35.030694 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:40:35.030669 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-8fpvb"] Jul 06 15:40:35.119676 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:40:35.119645 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-prqw2\" (UniqueName: \"kubernetes.io/projected/66cf8261-1821-46d7-ae9b-d36ecaf8dcdc-kube-api-access-prqw2\") pod \"cert-manager-operator-controller-manager-55c444bf86-8fpvb\" (UID: \"66cf8261-1821-46d7-ae9b-d36ecaf8dcdc\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-8fpvb" Jul 06 15:40:35.119836 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:40:35.119691 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/66cf8261-1821-46d7-ae9b-d36ecaf8dcdc-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-8fpvb\" (UID: \"66cf8261-1821-46d7-ae9b-d36ecaf8dcdc\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-8fpvb" Jul 06 15:40:35.220817 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:40:35.220782 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-prqw2\" (UniqueName: \"kubernetes.io/projected/66cf8261-1821-46d7-ae9b-d36ecaf8dcdc-kube-api-access-prqw2\") pod \"cert-manager-operator-controller-manager-55c444bf86-8fpvb\" (UID: \"66cf8261-1821-46d7-ae9b-d36ecaf8dcdc\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-8fpvb" Jul 06 15:40:35.220983 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:40:35.220834 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/66cf8261-1821-46d7-ae9b-d36ecaf8dcdc-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-8fpvb\" (UID: \"66cf8261-1821-46d7-ae9b-d36ecaf8dcdc\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-8fpvb" Jul 06 15:40:35.221242 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:40:35.221225 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/66cf8261-1821-46d7-ae9b-d36ecaf8dcdc-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-8fpvb\" (UID: \"66cf8261-1821-46d7-ae9b-d36ecaf8dcdc\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-8fpvb" Jul 06 15:40:35.230766 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:40:35.230741 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-prqw2\" (UniqueName: \"kubernetes.io/projected/66cf8261-1821-46d7-ae9b-d36ecaf8dcdc-kube-api-access-prqw2\") pod \"cert-manager-operator-controller-manager-55c444bf86-8fpvb\" (UID: \"66cf8261-1821-46d7-ae9b-d36ecaf8dcdc\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-8fpvb" Jul 06 15:40:35.314904 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:40:35.314828 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-8fpvb" Jul 06 15:40:35.455019 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:40:35.454988 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-8fpvb"] Jul 06 15:40:35.457636 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:40:35.457606 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod66cf8261_1821_46d7_ae9b_d36ecaf8dcdc.slice/crio-7e6347467aa9b1111a86e3f44564f79543d4c98f3ebdbc0a6845e4c7d9ead75a WatchSource:0}: Error finding container 7e6347467aa9b1111a86e3f44564f79543d4c98f3ebdbc0a6845e4c7d9ead75a: Status 404 returned error can't find the container with id 7e6347467aa9b1111a86e3f44564f79543d4c98f3ebdbc0a6845e4c7d9ead75a Jul 06 15:40:35.772663 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:40:35.772621 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-8fpvb" event={"ID":"66cf8261-1821-46d7-ae9b-d36ecaf8dcdc","Type":"ContainerStarted","Data":"7e6347467aa9b1111a86e3f44564f79543d4c98f3ebdbc0a6845e4c7d9ead75a"} Jul 06 15:40:39.797590 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:40:39.797556 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-8fpvb" event={"ID":"66cf8261-1821-46d7-ae9b-d36ecaf8dcdc","Type":"ContainerStarted","Data":"ace202300af1297712c8ae26af67a3860620b77c4af553787fde9e7d3bab2ceb"} Jul 06 15:40:39.822593 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:40:39.822535 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-8fpvb" podStartSLOduration=1.918442183 podStartE2EDuration="5.822514886s" totalImagesPullingTime="3.904072703s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 15:40:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 15:40:39.820149075 +0000 UTC m=+416.329382264" watchObservedRunningTime="2026-07-06 15:40:39.822514886 +0000 UTC m=+416.331748067" Jul 06 15:40:40.723736 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:40:40.723701 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-f5kmd"] Jul 06 15:40:40.727094 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:40:40.727077 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-f5kmd" Jul 06 15:40:40.730174 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:40:40.730152 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-ddtmx\"" Jul 06 15:40:40.731408 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:40:40.731384 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Jul 06 15:40:40.731486 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:40:40.731443 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Jul 06 15:40:40.738234 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:40:40.738211 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-f5kmd"] Jul 06 15:40:40.764536 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:40:40.764501 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cr6l2\" (UniqueName: \"kubernetes.io/projected/e2cb8b06-30e7-4286-a529-05da18e6f1e6-kube-api-access-cr6l2\") pod \"cert-manager-webhook-6486ffd997-f5kmd\" (UID: \"e2cb8b06-30e7-4286-a529-05da18e6f1e6\") " pod="cert-manager/cert-manager-webhook-6486ffd997-f5kmd" Jul 06 15:40:40.764691 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:40:40.764593 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e2cb8b06-30e7-4286-a529-05da18e6f1e6-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-f5kmd\" (UID: \"e2cb8b06-30e7-4286-a529-05da18e6f1e6\") " pod="cert-manager/cert-manager-webhook-6486ffd997-f5kmd" Jul 06 15:40:40.865077 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:40:40.865037 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e2cb8b06-30e7-4286-a529-05da18e6f1e6-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-f5kmd\" (UID: \"e2cb8b06-30e7-4286-a529-05da18e6f1e6\") " pod="cert-manager/cert-manager-webhook-6486ffd997-f5kmd" Jul 06 15:40:40.865519 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:40:40.865116 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cr6l2\" (UniqueName: \"kubernetes.io/projected/e2cb8b06-30e7-4286-a529-05da18e6f1e6-kube-api-access-cr6l2\") pod \"cert-manager-webhook-6486ffd997-f5kmd\" (UID: \"e2cb8b06-30e7-4286-a529-05da18e6f1e6\") " pod="cert-manager/cert-manager-webhook-6486ffd997-f5kmd" Jul 06 15:40:40.874496 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:40:40.874469 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cr6l2\" (UniqueName: \"kubernetes.io/projected/e2cb8b06-30e7-4286-a529-05da18e6f1e6-kube-api-access-cr6l2\") pod \"cert-manager-webhook-6486ffd997-f5kmd\" (UID: \"e2cb8b06-30e7-4286-a529-05da18e6f1e6\") " pod="cert-manager/cert-manager-webhook-6486ffd997-f5kmd" Jul 06 15:40:40.874631 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:40:40.874560 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e2cb8b06-30e7-4286-a529-05da18e6f1e6-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-f5kmd\" (UID: \"e2cb8b06-30e7-4286-a529-05da18e6f1e6\") " pod="cert-manager/cert-manager-webhook-6486ffd997-f5kmd" Jul 06 15:40:41.048649 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:40:41.048559 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-f5kmd" Jul 06 15:40:41.185939 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:40:41.185908 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-f5kmd"] Jul 06 15:40:41.188567 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:40:41.188538 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode2cb8b06_30e7_4286_a529_05da18e6f1e6.slice/crio-f13f3e41210f7beaa5315e9a1b010ec842298e1e11e586f38b91ed69a45fb070 WatchSource:0}: Error finding container f13f3e41210f7beaa5315e9a1b010ec842298e1e11e586f38b91ed69a45fb070: Status 404 returned error can't find the container with id f13f3e41210f7beaa5315e9a1b010ec842298e1e11e586f38b91ed69a45fb070 Jul 06 15:40:41.192118 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:40:41.192101 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmhrlk"] Jul 06 15:40:41.196706 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:40:41.196689 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmhrlk" Jul 06 15:40:41.200846 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:40:41.200823 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-kxcwp\"" Jul 06 15:40:41.201689 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:40:41.201670 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 06 15:40:41.201776 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:40:41.201682 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 06 15:40:41.207460 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:40:41.207439 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmhrlk"] Jul 06 15:40:41.269095 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:40:41.269058 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-82ptp\" (UniqueName: \"kubernetes.io/projected/1ddd5564-3f04-40e1-9c56-deecc8906b6f-kube-api-access-82ptp\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmhrlk\" (UID: \"1ddd5564-3f04-40e1-9c56-deecc8906b6f\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmhrlk" Jul 06 15:40:41.269316 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:40:41.269139 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1ddd5564-3f04-40e1-9c56-deecc8906b6f-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmhrlk\" (UID: \"1ddd5564-3f04-40e1-9c56-deecc8906b6f\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmhrlk" Jul 06 15:40:41.269316 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:40:41.269178 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1ddd5564-3f04-40e1-9c56-deecc8906b6f-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmhrlk\" (UID: \"1ddd5564-3f04-40e1-9c56-deecc8906b6f\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmhrlk" Jul 06 15:40:41.370019 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:40:41.369936 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-82ptp\" (UniqueName: \"kubernetes.io/projected/1ddd5564-3f04-40e1-9c56-deecc8906b6f-kube-api-access-82ptp\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmhrlk\" (UID: \"1ddd5564-3f04-40e1-9c56-deecc8906b6f\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmhrlk" Jul 06 15:40:41.370019 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:40:41.370011 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1ddd5564-3f04-40e1-9c56-deecc8906b6f-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmhrlk\" (UID: \"1ddd5564-3f04-40e1-9c56-deecc8906b6f\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmhrlk" Jul 06 15:40:41.370197 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:40:41.370039 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1ddd5564-3f04-40e1-9c56-deecc8906b6f-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmhrlk\" (UID: \"1ddd5564-3f04-40e1-9c56-deecc8906b6f\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmhrlk" Jul 06 15:40:41.370406 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:40:41.370389 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1ddd5564-3f04-40e1-9c56-deecc8906b6f-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmhrlk\" (UID: \"1ddd5564-3f04-40e1-9c56-deecc8906b6f\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmhrlk" Jul 06 15:40:41.370472 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:40:41.370407 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1ddd5564-3f04-40e1-9c56-deecc8906b6f-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmhrlk\" (UID: \"1ddd5564-3f04-40e1-9c56-deecc8906b6f\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmhrlk" Jul 06 15:40:41.378809 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:40:41.378787 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-82ptp\" (UniqueName: \"kubernetes.io/projected/1ddd5564-3f04-40e1-9c56-deecc8906b6f-kube-api-access-82ptp\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmhrlk\" (UID: \"1ddd5564-3f04-40e1-9c56-deecc8906b6f\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmhrlk" Jul 06 15:40:41.506532 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:40:41.506499 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmhrlk" Jul 06 15:40:41.643259 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:40:41.643077 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmhrlk"] Jul 06 15:40:41.645827 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:40:41.645802 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1ddd5564_3f04_40e1_9c56_deecc8906b6f.slice/crio-db31c7a8482f17639ba976d8963ec693ef1db80ee9a73c7a603f4891b0beaf19 WatchSource:0}: Error finding container db31c7a8482f17639ba976d8963ec693ef1db80ee9a73c7a603f4891b0beaf19: Status 404 returned error can't find the container with id db31c7a8482f17639ba976d8963ec693ef1db80ee9a73c7a603f4891b0beaf19 Jul 06 15:40:41.809701 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:40:41.809667 2563 generic.go:356] "Generic (PLEG): container finished" podID="1ddd5564-3f04-40e1-9c56-deecc8906b6f" containerID="e252bf328a94e42ddc96ec75142d57ff23c11d604639a773e6e02033a88f5290" exitCode=0 Jul 06 15:40:41.809883 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:40:41.809735 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmhrlk" event={"ID":"1ddd5564-3f04-40e1-9c56-deecc8906b6f","Type":"ContainerDied","Data":"e252bf328a94e42ddc96ec75142d57ff23c11d604639a773e6e02033a88f5290"} Jul 06 15:40:41.809883 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:40:41.809773 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmhrlk" event={"ID":"1ddd5564-3f04-40e1-9c56-deecc8906b6f","Type":"ContainerStarted","Data":"db31c7a8482f17639ba976d8963ec693ef1db80ee9a73c7a603f4891b0beaf19"} Jul 06 15:40:41.811004 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:40:41.810978 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-f5kmd" event={"ID":"e2cb8b06-30e7-4286-a529-05da18e6f1e6","Type":"ContainerStarted","Data":"f13f3e41210f7beaa5315e9a1b010ec842298e1e11e586f38b91ed69a45fb070"} Jul 06 15:40:45.831977 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:40:45.831944 2563 generic.go:356] "Generic (PLEG): container finished" podID="1ddd5564-3f04-40e1-9c56-deecc8906b6f" containerID="5985da5ec2b076ff5b6f86312ed604f32ad90363e3941d88ab5047b2bdf5dd97" exitCode=0 Jul 06 15:40:45.832410 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:40:45.832018 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmhrlk" event={"ID":"1ddd5564-3f04-40e1-9c56-deecc8906b6f","Type":"ContainerDied","Data":"5985da5ec2b076ff5b6f86312ed604f32ad90363e3941d88ab5047b2bdf5dd97"} Jul 06 15:40:45.833595 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:40:45.833554 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-f5kmd" event={"ID":"e2cb8b06-30e7-4286-a529-05da18e6f1e6","Type":"ContainerStarted","Data":"2ba8d1d5c209250d8bdfe10c534b989b7d6aaad1e16ad8ef60769dc0f0c97236"} Jul 06 15:40:45.833700 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:40:45.833684 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-f5kmd" Jul 06 15:40:45.874975 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:40:45.874930 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-f5kmd" podStartSLOduration=2.179855321 podStartE2EDuration="5.874912046s" totalImagesPullingTime="3.695056725s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 15:40:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 15:40:45.873252922 +0000 UTC m=+422.382486098" watchObservedRunningTime="2026-07-06 15:40:45.874912046 +0000 UTC m=+422.384145219" Jul 06 15:40:46.840719 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:40:46.840685 2563 generic.go:356] "Generic (PLEG): container finished" podID="1ddd5564-3f04-40e1-9c56-deecc8906b6f" containerID="8cc4e8a87b578d3f62df256b05d7d3c3ae7ea10974e37bd919e1270e137902bb" exitCode=0 Jul 06 15:40:46.841109 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:40:46.840772 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmhrlk" event={"ID":"1ddd5564-3f04-40e1-9c56-deecc8906b6f","Type":"ContainerDied","Data":"8cc4e8a87b578d3f62df256b05d7d3c3ae7ea10974e37bd919e1270e137902bb"} Jul 06 15:40:47.974026 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:40:47.974000 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmhrlk" Jul 06 15:40:48.027375 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:40:48.027340 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-82ptp\" (UniqueName: \"kubernetes.io/projected/1ddd5564-3f04-40e1-9c56-deecc8906b6f-kube-api-access-82ptp\") pod \"1ddd5564-3f04-40e1-9c56-deecc8906b6f\" (UID: \"1ddd5564-3f04-40e1-9c56-deecc8906b6f\") " Jul 06 15:40:48.027576 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:40:48.027389 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1ddd5564-3f04-40e1-9c56-deecc8906b6f-bundle\") pod \"1ddd5564-3f04-40e1-9c56-deecc8906b6f\" (UID: \"1ddd5564-3f04-40e1-9c56-deecc8906b6f\") " Jul 06 15:40:48.027576 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:40:48.027515 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1ddd5564-3f04-40e1-9c56-deecc8906b6f-util\") pod \"1ddd5564-3f04-40e1-9c56-deecc8906b6f\" (UID: \"1ddd5564-3f04-40e1-9c56-deecc8906b6f\") " Jul 06 15:40:48.027914 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:40:48.027887 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1ddd5564-3f04-40e1-9c56-deecc8906b6f-bundle" (OuterVolumeSpecName: "bundle") pod "1ddd5564-3f04-40e1-9c56-deecc8906b6f" (UID: "1ddd5564-3f04-40e1-9c56-deecc8906b6f"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 15:40:48.029526 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:40:48.029503 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1ddd5564-3f04-40e1-9c56-deecc8906b6f-kube-api-access-82ptp" (OuterVolumeSpecName: "kube-api-access-82ptp") pod "1ddd5564-3f04-40e1-9c56-deecc8906b6f" (UID: "1ddd5564-3f04-40e1-9c56-deecc8906b6f"). InnerVolumeSpecName "kube-api-access-82ptp". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 15:40:48.031914 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:40:48.031888 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1ddd5564-3f04-40e1-9c56-deecc8906b6f-util" (OuterVolumeSpecName: "util") pod "1ddd5564-3f04-40e1-9c56-deecc8906b6f" (UID: "1ddd5564-3f04-40e1-9c56-deecc8906b6f"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 15:40:48.128208 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:40:48.128129 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-82ptp\" (UniqueName: \"kubernetes.io/projected/1ddd5564-3f04-40e1-9c56-deecc8906b6f-kube-api-access-82ptp\") on node \"ip-10-0-129-154.ec2.internal\" DevicePath \"\"" Jul 06 15:40:48.128208 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:40:48.128157 2563 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1ddd5564-3f04-40e1-9c56-deecc8906b6f-bundle\") on node \"ip-10-0-129-154.ec2.internal\" DevicePath \"\"" Jul 06 15:40:48.128208 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:40:48.128166 2563 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1ddd5564-3f04-40e1-9c56-deecc8906b6f-util\") on node \"ip-10-0-129-154.ec2.internal\" DevicePath \"\"" Jul 06 15:40:48.851648 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:40:48.851612 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmhrlk" event={"ID":"1ddd5564-3f04-40e1-9c56-deecc8906b6f","Type":"ContainerDied","Data":"db31c7a8482f17639ba976d8963ec693ef1db80ee9a73c7a603f4891b0beaf19"} Jul 06 15:40:48.851648 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:40:48.851637 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmhrlk" Jul 06 15:40:48.851648 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:40:48.851657 2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="db31c7a8482f17639ba976d8963ec693ef1db80ee9a73c7a603f4891b0beaf19" Jul 06 15:40:51.623372 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:40:51.623337 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-glrdj"] Jul 06 15:40:51.623774 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:40:51.623672 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1ddd5564-3f04-40e1-9c56-deecc8906b6f" containerName="util" Jul 06 15:40:51.623774 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:40:51.623685 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="1ddd5564-3f04-40e1-9c56-deecc8906b6f" containerName="util" Jul 06 15:40:51.623774 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:40:51.623693 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1ddd5564-3f04-40e1-9c56-deecc8906b6f" containerName="pull" Jul 06 15:40:51.623774 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:40:51.623698 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="1ddd5564-3f04-40e1-9c56-deecc8906b6f" containerName="pull" Jul 06 15:40:51.623774 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:40:51.623708 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1ddd5564-3f04-40e1-9c56-deecc8906b6f" containerName="extract" Jul 06 15:40:51.623774 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:40:51.623714 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="1ddd5564-3f04-40e1-9c56-deecc8906b6f" containerName="extract" Jul 06 15:40:51.623774 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:40:51.623768 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="1ddd5564-3f04-40e1-9c56-deecc8906b6f" containerName="extract" Jul 06 15:40:51.627691 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:40:51.627672 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-glrdj" Jul 06 15:40:51.630104 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:40:51.630084 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-pct6n\"" Jul 06 15:40:51.637544 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:40:51.637514 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-glrdj"] Jul 06 15:40:51.659914 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:40:51.659885 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/19d0a49b-465b-4d02-a007-7be6fb808534-bound-sa-token\") pod \"cert-manager-55b4d954d5-glrdj\" (UID: \"19d0a49b-465b-4d02-a007-7be6fb808534\") " pod="cert-manager/cert-manager-55b4d954d5-glrdj" Jul 06 15:40:51.660019 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:40:51.659936 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6ztp7\" (UniqueName: \"kubernetes.io/projected/19d0a49b-465b-4d02-a007-7be6fb808534-kube-api-access-6ztp7\") pod \"cert-manager-55b4d954d5-glrdj\" (UID: \"19d0a49b-465b-4d02-a007-7be6fb808534\") " pod="cert-manager/cert-manager-55b4d954d5-glrdj" Jul 06 15:40:51.761124 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:40:51.761093 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/19d0a49b-465b-4d02-a007-7be6fb808534-bound-sa-token\") pod \"cert-manager-55b4d954d5-glrdj\" (UID: \"19d0a49b-465b-4d02-a007-7be6fb808534\") " pod="cert-manager/cert-manager-55b4d954d5-glrdj" Jul 06 15:40:51.761291 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:40:51.761144 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6ztp7\" (UniqueName: \"kubernetes.io/projected/19d0a49b-465b-4d02-a007-7be6fb808534-kube-api-access-6ztp7\") pod \"cert-manager-55b4d954d5-glrdj\" (UID: \"19d0a49b-465b-4d02-a007-7be6fb808534\") " pod="cert-manager/cert-manager-55b4d954d5-glrdj" Jul 06 15:40:51.770208 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:40:51.770178 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/19d0a49b-465b-4d02-a007-7be6fb808534-bound-sa-token\") pod \"cert-manager-55b4d954d5-glrdj\" (UID: \"19d0a49b-465b-4d02-a007-7be6fb808534\") " pod="cert-manager/cert-manager-55b4d954d5-glrdj" Jul 06 15:40:51.770546 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:40:51.770530 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6ztp7\" (UniqueName: \"kubernetes.io/projected/19d0a49b-465b-4d02-a007-7be6fb808534-kube-api-access-6ztp7\") pod \"cert-manager-55b4d954d5-glrdj\" (UID: \"19d0a49b-465b-4d02-a007-7be6fb808534\") " pod="cert-manager/cert-manager-55b4d954d5-glrdj" Jul 06 15:40:51.844021 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:40:51.843992 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-f5kmd" Jul 06 15:40:51.938011 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:40:51.937986 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-glrdj" Jul 06 15:40:52.076556 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:40:52.076533 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-glrdj"] Jul 06 15:40:52.078963 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:40:52.078937 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod19d0a49b_465b_4d02_a007_7be6fb808534.slice/crio-3c7e45989f3cd239ea9c1fd3a4af49779dc490ae952875ed57249000c10480e5 WatchSource:0}: Error finding container 3c7e45989f3cd239ea9c1fd3a4af49779dc490ae952875ed57249000c10480e5: Status 404 returned error can't find the container with id 3c7e45989f3cd239ea9c1fd3a4af49779dc490ae952875ed57249000c10480e5 Jul 06 15:40:52.871635 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:40:52.871598 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-glrdj" event={"ID":"19d0a49b-465b-4d02-a007-7be6fb808534","Type":"ContainerStarted","Data":"d4b186fe70bc11403fe1e7c877de8d0e9bea4ddd4587a2167ab2e09964e7ffcd"} Jul 06 15:40:52.872011 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:40:52.871650 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-glrdj" event={"ID":"19d0a49b-465b-4d02-a007-7be6fb808534","Type":"ContainerStarted","Data":"3c7e45989f3cd239ea9c1fd3a4af49779dc490ae952875ed57249000c10480e5"} Jul 06 15:40:52.894169 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:40:52.894116 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-glrdj" podStartSLOduration=1.8940949059999999 podStartE2EDuration="1.894094906s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 15:40:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 15:40:52.894035244 +0000 UTC m=+429.403268421" watchObservedRunningTime="2026-07-06 15:40:52.894094906 +0000 UTC m=+429.403328081" Jul 06 15:40:54.135285 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:40:54.135246 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-qwbnk"] Jul 06 15:40:54.139279 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:40:54.139260 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-qwbnk" Jul 06 15:40:54.141996 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:40:54.141977 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 06 15:40:54.143175 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:40:54.143157 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 06 15:40:54.143273 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:40:54.143180 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-f458h\"" Jul 06 15:40:54.150591 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:40:54.150571 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-qwbnk"] Jul 06 15:40:54.183837 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:40:54.183807 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/728a2166-19f5-42c2-8ced-023e8720b64a-tmp\") pod \"openshift-lws-operator-fd8ccff4c-qwbnk\" (UID: \"728a2166-19f5-42c2-8ced-023e8720b64a\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-qwbnk" Jul 06 15:40:54.183961 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:40:54.183863 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nnp2w\" (UniqueName: \"kubernetes.io/projected/728a2166-19f5-42c2-8ced-023e8720b64a-kube-api-access-nnp2w\") pod \"openshift-lws-operator-fd8ccff4c-qwbnk\" (UID: \"728a2166-19f5-42c2-8ced-023e8720b64a\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-qwbnk" Jul 06 15:40:54.284435 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:40:54.284402 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nnp2w\" (UniqueName: \"kubernetes.io/projected/728a2166-19f5-42c2-8ced-023e8720b64a-kube-api-access-nnp2w\") pod \"openshift-lws-operator-fd8ccff4c-qwbnk\" (UID: \"728a2166-19f5-42c2-8ced-023e8720b64a\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-qwbnk" Jul 06 15:40:54.284579 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:40:54.284495 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/728a2166-19f5-42c2-8ced-023e8720b64a-tmp\") pod \"openshift-lws-operator-fd8ccff4c-qwbnk\" (UID: \"728a2166-19f5-42c2-8ced-023e8720b64a\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-qwbnk" Jul 06 15:40:54.284799 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:40:54.284785 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/728a2166-19f5-42c2-8ced-023e8720b64a-tmp\") pod \"openshift-lws-operator-fd8ccff4c-qwbnk\" (UID: \"728a2166-19f5-42c2-8ced-023e8720b64a\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-qwbnk" Jul 06 15:40:54.293810 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:40:54.293783 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nnp2w\" (UniqueName: \"kubernetes.io/projected/728a2166-19f5-42c2-8ced-023e8720b64a-kube-api-access-nnp2w\") pod \"openshift-lws-operator-fd8ccff4c-qwbnk\" (UID: \"728a2166-19f5-42c2-8ced-023e8720b64a\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-qwbnk" Jul 06 15:40:54.449002 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:40:54.448972 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-qwbnk" Jul 06 15:40:54.587008 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:40:54.586970 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-qwbnk"] Jul 06 15:40:54.590716 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:40:54.590688 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod728a2166_19f5_42c2_8ced_023e8720b64a.slice/crio-ab26ef9e3daf7512f4d287c65449f4011c79e46c6d0fd355780fd2c56a856afd WatchSource:0}: Error finding container ab26ef9e3daf7512f4d287c65449f4011c79e46c6d0fd355780fd2c56a856afd: Status 404 returned error can't find the container with id ab26ef9e3daf7512f4d287c65449f4011c79e46c6d0fd355780fd2c56a856afd Jul 06 15:40:54.887262 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:40:54.887181 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-qwbnk" event={"ID":"728a2166-19f5-42c2-8ced-023e8720b64a","Type":"ContainerStarted","Data":"ab26ef9e3daf7512f4d287c65449f4011c79e46c6d0fd355780fd2c56a856afd"} Jul 06 15:40:58.909848 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:40:58.909820 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-qwbnk" event={"ID":"728a2166-19f5-42c2-8ced-023e8720b64a","Type":"ContainerStarted","Data":"f9c685cbb42539f06f8061b24e478a6de2a46c1ab33126ce0147d63acd9fe1f8"} Jul 06 15:40:58.931731 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:40:58.931681 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-qwbnk" podStartSLOduration=0.943919046 podStartE2EDuration="4.931663687s" totalImagesPullingTime="3.987744641s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 15:40:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 15:40:58.930945272 +0000 UTC m=+435.440178459" watchObservedRunningTime="2026-07-06 15:40:58.931663687 +0000 UTC m=+435.440896859" Jul 06 15:41:01.504836 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:41:01.504798 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgkvpn"] Jul 06 15:41:01.528151 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:41:01.528121 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgkvpn"] Jul 06 15:41:01.528306 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:41:01.528249 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgkvpn" Jul 06 15:41:01.531063 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:41:01.531032 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 06 15:41:01.531213 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:41:01.531139 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-kxcwp\"" Jul 06 15:41:01.532278 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:41:01.532258 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 06 15:41:01.651849 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:41:01.651811 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nbrpz\" (UniqueName: \"kubernetes.io/projected/c5fa17a8-48c6-4baf-ab23-428f6ee2eeff-kube-api-access-nbrpz\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgkvpn\" (UID: \"c5fa17a8-48c6-4baf-ab23-428f6ee2eeff\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgkvpn" Jul 06 15:41:01.651849 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:41:01.651851 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c5fa17a8-48c6-4baf-ab23-428f6ee2eeff-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgkvpn\" (UID: \"c5fa17a8-48c6-4baf-ab23-428f6ee2eeff\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgkvpn" Jul 06 15:41:01.652067 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:41:01.651885 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c5fa17a8-48c6-4baf-ab23-428f6ee2eeff-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgkvpn\" (UID: \"c5fa17a8-48c6-4baf-ab23-428f6ee2eeff\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgkvpn" Jul 06 15:41:01.753290 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:41:01.753252 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nbrpz\" (UniqueName: \"kubernetes.io/projected/c5fa17a8-48c6-4baf-ab23-428f6ee2eeff-kube-api-access-nbrpz\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgkvpn\" (UID: \"c5fa17a8-48c6-4baf-ab23-428f6ee2eeff\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgkvpn" Jul 06 15:41:01.753290 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:41:01.753291 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c5fa17a8-48c6-4baf-ab23-428f6ee2eeff-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgkvpn\" (UID: \"c5fa17a8-48c6-4baf-ab23-428f6ee2eeff\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgkvpn" Jul 06 15:41:01.753594 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:41:01.753317 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c5fa17a8-48c6-4baf-ab23-428f6ee2eeff-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgkvpn\" (UID: \"c5fa17a8-48c6-4baf-ab23-428f6ee2eeff\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgkvpn" Jul 06 15:41:01.753757 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:41:01.753736 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c5fa17a8-48c6-4baf-ab23-428f6ee2eeff-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgkvpn\" (UID: \"c5fa17a8-48c6-4baf-ab23-428f6ee2eeff\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgkvpn" Jul 06 15:41:01.753819 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:41:01.753744 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c5fa17a8-48c6-4baf-ab23-428f6ee2eeff-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgkvpn\" (UID: \"c5fa17a8-48c6-4baf-ab23-428f6ee2eeff\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgkvpn" Jul 06 15:41:01.762628 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:41:01.762572 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nbrpz\" (UniqueName: \"kubernetes.io/projected/c5fa17a8-48c6-4baf-ab23-428f6ee2eeff-kube-api-access-nbrpz\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgkvpn\" (UID: \"c5fa17a8-48c6-4baf-ab23-428f6ee2eeff\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgkvpn" Jul 06 15:41:01.839073 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:41:01.839032 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgkvpn" Jul 06 15:41:02.004015 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:41:02.003936 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgkvpn"] Jul 06 15:41:02.006250 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:41:02.006224 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc5fa17a8_48c6_4baf_ab23_428f6ee2eeff.slice/crio-6f643aa459848ea3b65f86067404e257a9942598c036a5175168be743cdf7d93 WatchSource:0}: Error finding container 6f643aa459848ea3b65f86067404e257a9942598c036a5175168be743cdf7d93: Status 404 returned error can't find the container with id 6f643aa459848ea3b65f86067404e257a9942598c036a5175168be743cdf7d93 Jul 06 15:41:02.929868 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:41:02.929834 2563 generic.go:356] "Generic (PLEG): container finished" podID="c5fa17a8-48c6-4baf-ab23-428f6ee2eeff" containerID="93598f587d6c2b0bb53aefbbca4c4369d34a49a79d82d8777ad7a000d005999d" exitCode=0 Jul 06 15:41:02.930245 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:41:02.929896 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgkvpn" event={"ID":"c5fa17a8-48c6-4baf-ab23-428f6ee2eeff","Type":"ContainerDied","Data":"93598f587d6c2b0bb53aefbbca4c4369d34a49a79d82d8777ad7a000d005999d"} Jul 06 15:41:02.930245 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:41:02.929933 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgkvpn" event={"ID":"c5fa17a8-48c6-4baf-ab23-428f6ee2eeff","Type":"ContainerStarted","Data":"6f643aa459848ea3b65f86067404e257a9942598c036a5175168be743cdf7d93"} Jul 06 15:41:03.939308 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:41:03.939281 2563 generic.go:356] "Generic (PLEG): container finished" podID="c5fa17a8-48c6-4baf-ab23-428f6ee2eeff" containerID="ecdc88ef32c89d92ebb031835d74326e9b2f9e5d625670b48fbcd2435171655e" exitCode=0 Jul 06 15:41:03.944955 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:41:03.944932 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgkvpn" event={"ID":"c5fa17a8-48c6-4baf-ab23-428f6ee2eeff","Type":"ContainerDied","Data":"ecdc88ef32c89d92ebb031835d74326e9b2f9e5d625670b48fbcd2435171655e"} Jul 06 15:41:04.946040 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:41:04.945964 2563 generic.go:356] "Generic (PLEG): container finished" podID="c5fa17a8-48c6-4baf-ab23-428f6ee2eeff" containerID="bbf801f1d8d37205a518815605cfc4d94d8ce98f9111fb3fdc6e5593c2d96cee" exitCode=0 Jul 06 15:41:04.946040 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:41:04.946004 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgkvpn" event={"ID":"c5fa17a8-48c6-4baf-ab23-428f6ee2eeff","Type":"ContainerDied","Data":"bbf801f1d8d37205a518815605cfc4d94d8ce98f9111fb3fdc6e5593c2d96cee"} Jul 06 15:41:06.083717 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:41:06.083692 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgkvpn" Jul 06 15:41:06.194765 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:41:06.194718 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nbrpz\" (UniqueName: \"kubernetes.io/projected/c5fa17a8-48c6-4baf-ab23-428f6ee2eeff-kube-api-access-nbrpz\") pod \"c5fa17a8-48c6-4baf-ab23-428f6ee2eeff\" (UID: \"c5fa17a8-48c6-4baf-ab23-428f6ee2eeff\") " Jul 06 15:41:06.194985 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:41:06.194820 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c5fa17a8-48c6-4baf-ab23-428f6ee2eeff-bundle\") pod \"c5fa17a8-48c6-4baf-ab23-428f6ee2eeff\" (UID: \"c5fa17a8-48c6-4baf-ab23-428f6ee2eeff\") " Jul 06 15:41:06.194985 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:41:06.194868 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c5fa17a8-48c6-4baf-ab23-428f6ee2eeff-util\") pod \"c5fa17a8-48c6-4baf-ab23-428f6ee2eeff\" (UID: \"c5fa17a8-48c6-4baf-ab23-428f6ee2eeff\") " Jul 06 15:41:06.195620 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:41:06.195593 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c5fa17a8-48c6-4baf-ab23-428f6ee2eeff-bundle" (OuterVolumeSpecName: "bundle") pod "c5fa17a8-48c6-4baf-ab23-428f6ee2eeff" (UID: "c5fa17a8-48c6-4baf-ab23-428f6ee2eeff"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 15:41:06.196931 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:41:06.196903 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c5fa17a8-48c6-4baf-ab23-428f6ee2eeff-kube-api-access-nbrpz" (OuterVolumeSpecName: "kube-api-access-nbrpz") pod "c5fa17a8-48c6-4baf-ab23-428f6ee2eeff" (UID: "c5fa17a8-48c6-4baf-ab23-428f6ee2eeff"). InnerVolumeSpecName "kube-api-access-nbrpz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 15:41:06.200133 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:41:06.200110 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c5fa17a8-48c6-4baf-ab23-428f6ee2eeff-util" (OuterVolumeSpecName: "util") pod "c5fa17a8-48c6-4baf-ab23-428f6ee2eeff" (UID: "c5fa17a8-48c6-4baf-ab23-428f6ee2eeff"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 15:41:06.295807 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:41:06.295773 2563 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c5fa17a8-48c6-4baf-ab23-428f6ee2eeff-bundle\") on node \"ip-10-0-129-154.ec2.internal\" DevicePath \"\"" Jul 06 15:41:06.295807 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:41:06.295799 2563 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c5fa17a8-48c6-4baf-ab23-428f6ee2eeff-util\") on node \"ip-10-0-129-154.ec2.internal\" DevicePath \"\"" Jul 06 15:41:06.295807 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:41:06.295807 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nbrpz\" (UniqueName: \"kubernetes.io/projected/c5fa17a8-48c6-4baf-ab23-428f6ee2eeff-kube-api-access-nbrpz\") on node \"ip-10-0-129-154.ec2.internal\" DevicePath \"\"" Jul 06 15:41:06.956895 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:41:06.956858 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgkvpn" event={"ID":"c5fa17a8-48c6-4baf-ab23-428f6ee2eeff","Type":"ContainerDied","Data":"6f643aa459848ea3b65f86067404e257a9942598c036a5175168be743cdf7d93"} Jul 06 15:41:06.957102 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:41:06.956914 2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6f643aa459848ea3b65f86067404e257a9942598c036a5175168be743cdf7d93" Jul 06 15:41:06.957102 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:41:06.956868 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgkvpn" Jul 06 15:41:18.894365 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:41:18.894329 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-749df7788b-fdjpk"] Jul 06 15:41:18.894771 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:41:18.894681 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c5fa17a8-48c6-4baf-ab23-428f6ee2eeff" containerName="util" Jul 06 15:41:18.894771 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:41:18.894693 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="c5fa17a8-48c6-4baf-ab23-428f6ee2eeff" containerName="util" Jul 06 15:41:18.894771 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:41:18.894703 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c5fa17a8-48c6-4baf-ab23-428f6ee2eeff" containerName="extract" Jul 06 15:41:18.894771 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:41:18.894709 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="c5fa17a8-48c6-4baf-ab23-428f6ee2eeff" containerName="extract" Jul 06 15:41:18.894771 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:41:18.894726 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c5fa17a8-48c6-4baf-ab23-428f6ee2eeff" containerName="pull" Jul 06 15:41:18.894771 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:41:18.894731 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="c5fa17a8-48c6-4baf-ab23-428f6ee2eeff" containerName="pull" Jul 06 15:41:18.894951 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:41:18.894791 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="c5fa17a8-48c6-4baf-ab23-428f6ee2eeff" containerName="extract" Jul 06 15:41:18.898094 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:41:18.898079 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-749df7788b-fdjpk" Jul 06 15:41:18.900582 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:41:18.900560 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 06 15:41:18.900706 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:41:18.900568 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-dj8mj\"" Jul 06 15:41:18.900706 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:41:18.900662 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 06 15:41:18.901015 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:41:18.900998 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 06 15:41:18.901486 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:41:18.901467 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 06 15:41:18.921382 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:41:18.921359 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-749df7788b-fdjpk"] Jul 06 15:41:19.005842 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:41:19.005809 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d44bfd0f-f64b-4f20-b0bd-d3ac355633aa-tmp\") pod \"opendatahub-operator-controller-manager-749df7788b-fdjpk\" (UID: \"d44bfd0f-f64b-4f20-b0bd-d3ac355633aa\") " pod="opendatahub/opendatahub-operator-controller-manager-749df7788b-fdjpk" Jul 06 15:41:19.005842 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:41:19.005848 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wm2np\" (UniqueName: \"kubernetes.io/projected/d44bfd0f-f64b-4f20-b0bd-d3ac355633aa-kube-api-access-wm2np\") pod \"opendatahub-operator-controller-manager-749df7788b-fdjpk\" (UID: \"d44bfd0f-f64b-4f20-b0bd-d3ac355633aa\") " pod="opendatahub/opendatahub-operator-controller-manager-749df7788b-fdjpk" Jul 06 15:41:19.006043 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:41:19.005903 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/d44bfd0f-f64b-4f20-b0bd-d3ac355633aa-manifests\") pod \"opendatahub-operator-controller-manager-749df7788b-fdjpk\" (UID: \"d44bfd0f-f64b-4f20-b0bd-d3ac355633aa\") " pod="opendatahub/opendatahub-operator-controller-manager-749df7788b-fdjpk" Jul 06 15:41:19.006043 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:41:19.005931 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/d44bfd0f-f64b-4f20-b0bd-d3ac355633aa-apiservice-cert\") pod \"opendatahub-operator-controller-manager-749df7788b-fdjpk\" (UID: \"d44bfd0f-f64b-4f20-b0bd-d3ac355633aa\") " pod="opendatahub/opendatahub-operator-controller-manager-749df7788b-fdjpk" Jul 06 15:41:19.006043 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:41:19.005975 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d44bfd0f-f64b-4f20-b0bd-d3ac355633aa-webhook-cert\") pod \"opendatahub-operator-controller-manager-749df7788b-fdjpk\" (UID: \"d44bfd0f-f64b-4f20-b0bd-d3ac355633aa\") " pod="opendatahub/opendatahub-operator-controller-manager-749df7788b-fdjpk" Jul 06 15:41:19.106824 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:41:19.106790 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/d44bfd0f-f64b-4f20-b0bd-d3ac355633aa-manifests\") pod \"opendatahub-operator-controller-manager-749df7788b-fdjpk\" (UID: \"d44bfd0f-f64b-4f20-b0bd-d3ac355633aa\") " pod="opendatahub/opendatahub-operator-controller-manager-749df7788b-fdjpk" Jul 06 15:41:19.107001 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:41:19.106832 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/d44bfd0f-f64b-4f20-b0bd-d3ac355633aa-apiservice-cert\") pod \"opendatahub-operator-controller-manager-749df7788b-fdjpk\" (UID: \"d44bfd0f-f64b-4f20-b0bd-d3ac355633aa\") " pod="opendatahub/opendatahub-operator-controller-manager-749df7788b-fdjpk" Jul 06 15:41:19.107001 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:41:19.106867 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d44bfd0f-f64b-4f20-b0bd-d3ac355633aa-webhook-cert\") pod \"opendatahub-operator-controller-manager-749df7788b-fdjpk\" (UID: \"d44bfd0f-f64b-4f20-b0bd-d3ac355633aa\") " pod="opendatahub/opendatahub-operator-controller-manager-749df7788b-fdjpk" Jul 06 15:41:19.107001 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:41:19.106896 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d44bfd0f-f64b-4f20-b0bd-d3ac355633aa-tmp\") pod \"opendatahub-operator-controller-manager-749df7788b-fdjpk\" (UID: \"d44bfd0f-f64b-4f20-b0bd-d3ac355633aa\") " pod="opendatahub/opendatahub-operator-controller-manager-749df7788b-fdjpk" Jul 06 15:41:19.107001 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:41:19.106935 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wm2np\" (UniqueName: \"kubernetes.io/projected/d44bfd0f-f64b-4f20-b0bd-d3ac355633aa-kube-api-access-wm2np\") pod \"opendatahub-operator-controller-manager-749df7788b-fdjpk\" (UID: \"d44bfd0f-f64b-4f20-b0bd-d3ac355633aa\") " pod="opendatahub/opendatahub-operator-controller-manager-749df7788b-fdjpk" Jul 06 15:41:19.107253 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:41:19.107227 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/d44bfd0f-f64b-4f20-b0bd-d3ac355633aa-manifests\") pod \"opendatahub-operator-controller-manager-749df7788b-fdjpk\" (UID: \"d44bfd0f-f64b-4f20-b0bd-d3ac355633aa\") " pod="opendatahub/opendatahub-operator-controller-manager-749df7788b-fdjpk" Jul 06 15:41:19.109307 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:41:19.109273 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d44bfd0f-f64b-4f20-b0bd-d3ac355633aa-tmp\") pod \"opendatahub-operator-controller-manager-749df7788b-fdjpk\" (UID: \"d44bfd0f-f64b-4f20-b0bd-d3ac355633aa\") " pod="opendatahub/opendatahub-operator-controller-manager-749df7788b-fdjpk" Jul 06 15:41:19.109307 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:41:19.109307 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/d44bfd0f-f64b-4f20-b0bd-d3ac355633aa-apiservice-cert\") pod \"opendatahub-operator-controller-manager-749df7788b-fdjpk\" (UID: \"d44bfd0f-f64b-4f20-b0bd-d3ac355633aa\") " pod="opendatahub/opendatahub-operator-controller-manager-749df7788b-fdjpk" Jul 06 15:41:19.109494 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:41:19.109349 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d44bfd0f-f64b-4f20-b0bd-d3ac355633aa-webhook-cert\") pod \"opendatahub-operator-controller-manager-749df7788b-fdjpk\" (UID: \"d44bfd0f-f64b-4f20-b0bd-d3ac355633aa\") " pod="opendatahub/opendatahub-operator-controller-manager-749df7788b-fdjpk" Jul 06 15:41:19.124844 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:41:19.124824 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wm2np\" (UniqueName: \"kubernetes.io/projected/d44bfd0f-f64b-4f20-b0bd-d3ac355633aa-kube-api-access-wm2np\") pod \"opendatahub-operator-controller-manager-749df7788b-fdjpk\" (UID: \"d44bfd0f-f64b-4f20-b0bd-d3ac355633aa\") " pod="opendatahub/opendatahub-operator-controller-manager-749df7788b-fdjpk" Jul 06 15:41:19.207901 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:41:19.207867 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-749df7788b-fdjpk" Jul 06 15:41:19.377911 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:41:19.377874 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-749df7788b-fdjpk"] Jul 06 15:41:19.382013 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:41:19.381983 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd44bfd0f_f64b_4f20_b0bd_d3ac355633aa.slice/crio-48ff232852fda6fb4857ad0f6d1e560f9e6ad26539170fae6a13334d30292894 WatchSource:0}: Error finding container 48ff232852fda6fb4857ad0f6d1e560f9e6ad26539170fae6a13334d30292894: Status 404 returned error can't find the container with id 48ff232852fda6fb4857ad0f6d1e560f9e6ad26539170fae6a13334d30292894 Jul 06 15:41:20.024157 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:41:20.024124 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-749df7788b-fdjpk" event={"ID":"d44bfd0f-f64b-4f20-b0bd-d3ac355633aa","Type":"ContainerStarted","Data":"48ff232852fda6fb4857ad0f6d1e560f9e6ad26539170fae6a13334d30292894"} Jul 06 15:41:23.042946 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:41:23.042897 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-749df7788b-fdjpk" event={"ID":"d44bfd0f-f64b-4f20-b0bd-d3ac355633aa","Type":"ContainerStarted","Data":"6b1ff5369d10783f867cf8b4d3f509a141940eb6f4f3832c7123ec8b21a255f5"} Jul 06 15:41:24.048959 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:41:24.048922 2563 generic.go:356] "Generic (PLEG): container finished" podID="d44bfd0f-f64b-4f20-b0bd-d3ac355633aa" containerID="6b1ff5369d10783f867cf8b4d3f509a141940eb6f4f3832c7123ec8b21a255f5" exitCode=0 Jul 06 15:41:24.049335 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:41:24.049009 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-749df7788b-fdjpk" event={"ID":"d44bfd0f-f64b-4f20-b0bd-d3ac355633aa","Type":"ContainerDied","Data":"6b1ff5369d10783f867cf8b4d3f509a141940eb6f4f3832c7123ec8b21a255f5"} Jul 06 15:41:25.055584 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:41:25.055549 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-749df7788b-fdjpk" event={"ID":"d44bfd0f-f64b-4f20-b0bd-d3ac355633aa","Type":"ContainerStarted","Data":"e936cc40f3ebc660446d37bf7dcf934873f3119925943252ae1b379d09c2fcc8"} Jul 06 15:41:25.055972 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:41:25.055743 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-749df7788b-fdjpk" Jul 06 15:41:25.088203 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:41:25.088151 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-749df7788b-fdjpk" podStartSLOduration=2.643405929 podStartE2EDuration="7.088134328s" totalImagesPullingTime="3.343673694s" totalInitContainerRuntime="1.101054705s" isStatefulPod=false podCreationTimestamp="2026-07-06 15:41:18 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 15:41:25.08579205 +0000 UTC m=+461.595025224" watchObservedRunningTime="2026-07-06 15:41:25.088134328 +0000 UTC m=+461.597367569" Jul 06 15:41:36.063220 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:41:36.063185 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-749df7788b-fdjpk" Jul 06 15:41:39.728219 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:41:39.728183 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-745c454698-6f9d2"] Jul 06 15:41:39.732745 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:41:39.732728 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-745c454698-6f9d2" Jul 06 15:41:39.739199 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:41:39.739174 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Jul 06 15:41:39.739502 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:41:39.739476 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-bqv6c\"" Jul 06 15:41:39.742292 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:41:39.742271 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Jul 06 15:41:39.783284 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:41:39.783246 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a801e02a-0014-4850-bbb6-44986a875432-tls-certs\") pod \"kube-auth-proxy-745c454698-6f9d2\" (UID: \"a801e02a-0014-4850-bbb6-44986a875432\") " pod="openshift-ingress/kube-auth-proxy-745c454698-6f9d2" Jul 06 15:41:39.783464 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:41:39.783399 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a801e02a-0014-4850-bbb6-44986a875432-tmp\") pod \"kube-auth-proxy-745c454698-6f9d2\" (UID: \"a801e02a-0014-4850-bbb6-44986a875432\") " pod="openshift-ingress/kube-auth-proxy-745c454698-6f9d2" Jul 06 15:41:39.783516 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:41:39.783459 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ztljm\" (UniqueName: \"kubernetes.io/projected/a801e02a-0014-4850-bbb6-44986a875432-kube-api-access-ztljm\") pod \"kube-auth-proxy-745c454698-6f9d2\" (UID: \"a801e02a-0014-4850-bbb6-44986a875432\") " pod="openshift-ingress/kube-auth-proxy-745c454698-6f9d2" Jul 06 15:41:39.786090 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:41:39.786066 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-745c454698-6f9d2"] Jul 06 15:41:39.884180 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:41:39.884142 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a801e02a-0014-4850-bbb6-44986a875432-tmp\") pod \"kube-auth-proxy-745c454698-6f9d2\" (UID: \"a801e02a-0014-4850-bbb6-44986a875432\") " pod="openshift-ingress/kube-auth-proxy-745c454698-6f9d2" Jul 06 15:41:39.884180 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:41:39.884183 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ztljm\" (UniqueName: \"kubernetes.io/projected/a801e02a-0014-4850-bbb6-44986a875432-kube-api-access-ztljm\") pod \"kube-auth-proxy-745c454698-6f9d2\" (UID: \"a801e02a-0014-4850-bbb6-44986a875432\") " pod="openshift-ingress/kube-auth-proxy-745c454698-6f9d2" Jul 06 15:41:39.884470 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:41:39.884257 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a801e02a-0014-4850-bbb6-44986a875432-tls-certs\") pod \"kube-auth-proxy-745c454698-6f9d2\" (UID: \"a801e02a-0014-4850-bbb6-44986a875432\") " pod="openshift-ingress/kube-auth-proxy-745c454698-6f9d2" Jul 06 15:41:39.886739 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:41:39.886703 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a801e02a-0014-4850-bbb6-44986a875432-tmp\") pod \"kube-auth-proxy-745c454698-6f9d2\" (UID: \"a801e02a-0014-4850-bbb6-44986a875432\") " pod="openshift-ingress/kube-auth-proxy-745c454698-6f9d2" Jul 06 15:41:39.886867 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:41:39.886818 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a801e02a-0014-4850-bbb6-44986a875432-tls-certs\") pod \"kube-auth-proxy-745c454698-6f9d2\" (UID: \"a801e02a-0014-4850-bbb6-44986a875432\") " pod="openshift-ingress/kube-auth-proxy-745c454698-6f9d2" Jul 06 15:41:39.954845 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:41:39.954815 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ztljm\" (UniqueName: \"kubernetes.io/projected/a801e02a-0014-4850-bbb6-44986a875432-kube-api-access-ztljm\") pod \"kube-auth-proxy-745c454698-6f9d2\" (UID: \"a801e02a-0014-4850-bbb6-44986a875432\") " pod="openshift-ingress/kube-auth-proxy-745c454698-6f9d2" Jul 06 15:41:40.041785 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:41:40.041690 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-745c454698-6f9d2" Jul 06 15:41:40.192802 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:41:40.192774 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-745c454698-6f9d2"] Jul 06 15:41:40.194943 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:41:40.194914 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda801e02a_0014_4850_bbb6_44986a875432.slice/crio-2d980d9d890bb25bdd30b3b587d28fbdb7a84390d4594574d17d209bf02f79be WatchSource:0}: Error finding container 2d980d9d890bb25bdd30b3b587d28fbdb7a84390d4594574d17d209bf02f79be: Status 404 returned error can't find the container with id 2d980d9d890bb25bdd30b3b587d28fbdb7a84390d4594574d17d209bf02f79be Jul 06 15:41:40.979403 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:41:40.979356 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-s2j2x"] Jul 06 15:41:40.982712 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:41:40.982694 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-s2j2x" Jul 06 15:41:40.988236 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:41:40.988212 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\"" Jul 06 15:41:41.005199 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:41:41.005169 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-s2j2x"] Jul 06 15:41:41.094848 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:41:41.094811 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6frw8\" (UniqueName: \"kubernetes.io/projected/cd0760e4-311a-4634-9392-f904d4ca5499-kube-api-access-6frw8\") pod \"model-serving-api-54d89bd5fc-s2j2x\" (UID: \"cd0760e4-311a-4634-9392-f904d4ca5499\") " pod="opendatahub/model-serving-api-54d89bd5fc-s2j2x" Jul 06 15:41:41.095040 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:41:41.094873 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cd0760e4-311a-4634-9392-f904d4ca5499-tls-certs\") pod \"model-serving-api-54d89bd5fc-s2j2x\" (UID: \"cd0760e4-311a-4634-9392-f904d4ca5499\") " pod="opendatahub/model-serving-api-54d89bd5fc-s2j2x" Jul 06 15:41:41.152509 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:41:41.152473 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-745c454698-6f9d2" event={"ID":"a801e02a-0014-4850-bbb6-44986a875432","Type":"ContainerStarted","Data":"2d980d9d890bb25bdd30b3b587d28fbdb7a84390d4594574d17d209bf02f79be"} Jul 06 15:41:41.196149 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:41:41.196110 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6frw8\" (UniqueName: \"kubernetes.io/projected/cd0760e4-311a-4634-9392-f904d4ca5499-kube-api-access-6frw8\") pod \"model-serving-api-54d89bd5fc-s2j2x\" (UID: \"cd0760e4-311a-4634-9392-f904d4ca5499\") " pod="opendatahub/model-serving-api-54d89bd5fc-s2j2x" Jul 06 15:41:41.196350 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:41:41.196173 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cd0760e4-311a-4634-9392-f904d4ca5499-tls-certs\") pod \"model-serving-api-54d89bd5fc-s2j2x\" (UID: \"cd0760e4-311a-4634-9392-f904d4ca5499\") " pod="opendatahub/model-serving-api-54d89bd5fc-s2j2x" Jul 06 15:41:41.199603 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:41:41.199571 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cd0760e4-311a-4634-9392-f904d4ca5499-tls-certs\") pod \"model-serving-api-54d89bd5fc-s2j2x\" (UID: \"cd0760e4-311a-4634-9392-f904d4ca5499\") " pod="opendatahub/model-serving-api-54d89bd5fc-s2j2x" Jul 06 15:41:41.210942 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:41:41.210911 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6frw8\" (UniqueName: \"kubernetes.io/projected/cd0760e4-311a-4634-9392-f904d4ca5499-kube-api-access-6frw8\") pod \"model-serving-api-54d89bd5fc-s2j2x\" (UID: \"cd0760e4-311a-4634-9392-f904d4ca5499\") " pod="opendatahub/model-serving-api-54d89bd5fc-s2j2x" Jul 06 15:41:41.293992 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:41:41.293896 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-s2j2x" Jul 06 15:41:41.469912 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:41:41.469871 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-s2j2x"] Jul 06 15:41:41.471452 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:41:41.471404 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcd0760e4_311a_4634_9392_f904d4ca5499.slice/crio-09c3d59233e534c1e84899493bd2f127998bb7292baa2506eec4505f1bc5693c WatchSource:0}: Error finding container 09c3d59233e534c1e84899493bd2f127998bb7292baa2506eec4505f1bc5693c: Status 404 returned error can't find the container with id 09c3d59233e534c1e84899493bd2f127998bb7292baa2506eec4505f1bc5693c Jul 06 15:41:42.159222 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:41:42.159155 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-s2j2x" event={"ID":"cd0760e4-311a-4634-9392-f904d4ca5499","Type":"ContainerStarted","Data":"09c3d59233e534c1e84899493bd2f127998bb7292baa2506eec4505f1bc5693c"} Jul 06 15:41:44.158148 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:41:44.158116 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Jul 06 15:41:44.171323 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:41:44.171294 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-s2j2x" event={"ID":"cd0760e4-311a-4634-9392-f904d4ca5499","Type":"ContainerStarted","Data":"6d1efd04670af4591835f150f7833ca6509746334fbbe5057eea418ba9b39dba"} Jul 06 15:41:44.171493 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:41:44.171463 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-s2j2x" Jul 06 15:41:44.205900 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:41:44.205854 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-s2j2x" podStartSLOduration=2.378889237 podStartE2EDuration="4.205838359s" totalImagesPullingTime="1.826949122s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 15:41:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 15:41:44.204204011 +0000 UTC m=+480.713437187" watchObservedRunningTime="2026-07-06 15:41:44.205838359 +0000 UTC m=+480.715071533" Jul 06 15:41:45.178147 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:41:45.178113 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-745c454698-6f9d2" event={"ID":"a801e02a-0014-4850-bbb6-44986a875432","Type":"ContainerStarted","Data":"53166acd1d519bb4d37cd8004ea008dba9cd3d25a05346b9d534689fb2681f7c"} Jul 06 15:41:46.148185 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:41:46.148133 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-745c454698-6f9d2" podStartSLOduration=3.206876983 podStartE2EDuration="7.148115163s" totalImagesPullingTime="3.94123818s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 15:41:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 15:41:45.195808437 +0000 UTC m=+481.705041611" watchObservedRunningTime="2026-07-06 15:41:46.148115163 +0000 UTC m=+482.657348337" Jul 06 15:41:46.148800 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:41:46.148784 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-544dccfdb4-rl2tq"] Jul 06 15:41:46.152056 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:41:46.152041 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-544dccfdb4-rl2tq" Jul 06 15:41:46.160182 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:41:46.160161 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Jul 06 15:41:46.160303 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:41:46.160215 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-lz2bv\"" Jul 06 15:41:46.160303 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:41:46.160245 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Jul 06 15:41:46.160303 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:41:46.160218 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Jul 06 15:41:46.173429 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:41:46.173404 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-544dccfdb4-rl2tq"] Jul 06 15:41:46.240879 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:41:46.240838 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zclst\" (UniqueName: \"kubernetes.io/projected/c2d0ef2f-f717-4854-8a6b-1f09b3e0868e-kube-api-access-zclst\") pod \"lws-controller-manager-544dccfdb4-rl2tq\" (UID: \"c2d0ef2f-f717-4854-8a6b-1f09b3e0868e\") " pod="openshift-lws-operator/lws-controller-manager-544dccfdb4-rl2tq" Jul 06 15:41:46.241350 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:41:46.240947 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/c2d0ef2f-f717-4854-8a6b-1f09b3e0868e-metrics-cert\") pod \"lws-controller-manager-544dccfdb4-rl2tq\" (UID: \"c2d0ef2f-f717-4854-8a6b-1f09b3e0868e\") " pod="openshift-lws-operator/lws-controller-manager-544dccfdb4-rl2tq" Jul 06 15:41:46.241350 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:41:46.241092 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/c2d0ef2f-f717-4854-8a6b-1f09b3e0868e-manager-config\") pod \"lws-controller-manager-544dccfdb4-rl2tq\" (UID: \"c2d0ef2f-f717-4854-8a6b-1f09b3e0868e\") " pod="openshift-lws-operator/lws-controller-manager-544dccfdb4-rl2tq" Jul 06 15:41:46.241350 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:41:46.241142 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c2d0ef2f-f717-4854-8a6b-1f09b3e0868e-cert\") pod \"lws-controller-manager-544dccfdb4-rl2tq\" (UID: \"c2d0ef2f-f717-4854-8a6b-1f09b3e0868e\") " pod="openshift-lws-operator/lws-controller-manager-544dccfdb4-rl2tq" Jul 06 15:41:46.342042 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:41:46.342002 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/c2d0ef2f-f717-4854-8a6b-1f09b3e0868e-metrics-cert\") pod \"lws-controller-manager-544dccfdb4-rl2tq\" (UID: \"c2d0ef2f-f717-4854-8a6b-1f09b3e0868e\") " pod="openshift-lws-operator/lws-controller-manager-544dccfdb4-rl2tq" Jul 06 15:41:46.342205 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:41:46.342064 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/c2d0ef2f-f717-4854-8a6b-1f09b3e0868e-manager-config\") pod \"lws-controller-manager-544dccfdb4-rl2tq\" (UID: \"c2d0ef2f-f717-4854-8a6b-1f09b3e0868e\") " pod="openshift-lws-operator/lws-controller-manager-544dccfdb4-rl2tq" Jul 06 15:41:46.342205 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:41:46.342118 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c2d0ef2f-f717-4854-8a6b-1f09b3e0868e-cert\") pod \"lws-controller-manager-544dccfdb4-rl2tq\" (UID: \"c2d0ef2f-f717-4854-8a6b-1f09b3e0868e\") " pod="openshift-lws-operator/lws-controller-manager-544dccfdb4-rl2tq" Jul 06 15:41:46.342205 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:41:46.342166 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zclst\" (UniqueName: \"kubernetes.io/projected/c2d0ef2f-f717-4854-8a6b-1f09b3e0868e-kube-api-access-zclst\") pod \"lws-controller-manager-544dccfdb4-rl2tq\" (UID: \"c2d0ef2f-f717-4854-8a6b-1f09b3e0868e\") " pod="openshift-lws-operator/lws-controller-manager-544dccfdb4-rl2tq" Jul 06 15:41:46.342910 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:41:46.342841 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/c2d0ef2f-f717-4854-8a6b-1f09b3e0868e-manager-config\") pod \"lws-controller-manager-544dccfdb4-rl2tq\" (UID: \"c2d0ef2f-f717-4854-8a6b-1f09b3e0868e\") " pod="openshift-lws-operator/lws-controller-manager-544dccfdb4-rl2tq" Jul 06 15:41:46.344842 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:41:46.344818 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c2d0ef2f-f717-4854-8a6b-1f09b3e0868e-cert\") pod \"lws-controller-manager-544dccfdb4-rl2tq\" (UID: \"c2d0ef2f-f717-4854-8a6b-1f09b3e0868e\") " pod="openshift-lws-operator/lws-controller-manager-544dccfdb4-rl2tq" Jul 06 15:41:46.344922 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:41:46.344842 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/c2d0ef2f-f717-4854-8a6b-1f09b3e0868e-metrics-cert\") pod \"lws-controller-manager-544dccfdb4-rl2tq\" (UID: \"c2d0ef2f-f717-4854-8a6b-1f09b3e0868e\") " pod="openshift-lws-operator/lws-controller-manager-544dccfdb4-rl2tq" Jul 06 15:41:46.352593 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:41:46.352558 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zclst\" (UniqueName: \"kubernetes.io/projected/c2d0ef2f-f717-4854-8a6b-1f09b3e0868e-kube-api-access-zclst\") pod \"lws-controller-manager-544dccfdb4-rl2tq\" (UID: \"c2d0ef2f-f717-4854-8a6b-1f09b3e0868e\") " pod="openshift-lws-operator/lws-controller-manager-544dccfdb4-rl2tq" Jul 06 15:41:46.461685 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:41:46.461607 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-544dccfdb4-rl2tq" Jul 06 15:41:46.624813 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:41:46.624789 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-544dccfdb4-rl2tq"] Jul 06 15:41:46.627411 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:41:46.627380 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc2d0ef2f_f717_4854_8a6b_1f09b3e0868e.slice/crio-f7f1069f4ad68d4d44ffe7b383d957f46b735406e30f9ee7483b4026748b6032 WatchSource:0}: Error finding container f7f1069f4ad68d4d44ffe7b383d957f46b735406e30f9ee7483b4026748b6032: Status 404 returned error can't find the container with id f7f1069f4ad68d4d44ffe7b383d957f46b735406e30f9ee7483b4026748b6032 Jul 06 15:41:47.188869 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:41:47.188839 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-544dccfdb4-rl2tq" event={"ID":"c2d0ef2f-f717-4854-8a6b-1f09b3e0868e","Type":"ContainerStarted","Data":"f7f1069f4ad68d4d44ffe7b383d957f46b735406e30f9ee7483b4026748b6032"} Jul 06 15:41:48.293852 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:41:48.293817 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-5f6bdbfc9-hfwlk"] Jul 06 15:41:48.298668 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:41:48.298639 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5f6bdbfc9-hfwlk" Jul 06 15:41:48.316207 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:41:48.316136 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5f6bdbfc9-hfwlk"] Jul 06 15:41:48.356719 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:41:48.356680 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a3d590d1-bcc0-4c81-abb4-8aa13791bec6-console-serving-cert\") pod \"console-5f6bdbfc9-hfwlk\" (UID: \"a3d590d1-bcc0-4c81-abb4-8aa13791bec6\") " pod="openshift-console/console-5f6bdbfc9-hfwlk" Jul 06 15:41:48.356898 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:41:48.356724 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a3d590d1-bcc0-4c81-abb4-8aa13791bec6-service-ca\") pod \"console-5f6bdbfc9-hfwlk\" (UID: \"a3d590d1-bcc0-4c81-abb4-8aa13791bec6\") " pod="openshift-console/console-5f6bdbfc9-hfwlk" Jul 06 15:41:48.356898 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:41:48.356760 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a3d590d1-bcc0-4c81-abb4-8aa13791bec6-console-oauth-config\") pod \"console-5f6bdbfc9-hfwlk\" (UID: \"a3d590d1-bcc0-4c81-abb4-8aa13791bec6\") " pod="openshift-console/console-5f6bdbfc9-hfwlk" Jul 06 15:41:48.356898 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:41:48.356851 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a3d590d1-bcc0-4c81-abb4-8aa13791bec6-console-config\") pod \"console-5f6bdbfc9-hfwlk\" (UID: \"a3d590d1-bcc0-4c81-abb4-8aa13791bec6\") " pod="openshift-console/console-5f6bdbfc9-hfwlk" Jul 06 15:41:48.356898 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:41:48.356877 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a3d590d1-bcc0-4c81-abb4-8aa13791bec6-trusted-ca-bundle\") pod \"console-5f6bdbfc9-hfwlk\" (UID: \"a3d590d1-bcc0-4c81-abb4-8aa13791bec6\") " pod="openshift-console/console-5f6bdbfc9-hfwlk" Jul 06 15:41:48.357113 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:41:48.356952 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a3d590d1-bcc0-4c81-abb4-8aa13791bec6-oauth-serving-cert\") pod \"console-5f6bdbfc9-hfwlk\" (UID: \"a3d590d1-bcc0-4c81-abb4-8aa13791bec6\") " pod="openshift-console/console-5f6bdbfc9-hfwlk" Jul 06 15:41:48.357113 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:41:48.357016 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kvs84\" (UniqueName: \"kubernetes.io/projected/a3d590d1-bcc0-4c81-abb4-8aa13791bec6-kube-api-access-kvs84\") pod \"console-5f6bdbfc9-hfwlk\" (UID: \"a3d590d1-bcc0-4c81-abb4-8aa13791bec6\") " pod="openshift-console/console-5f6bdbfc9-hfwlk" Jul 06 15:41:48.457893 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:41:48.457859 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kvs84\" (UniqueName: \"kubernetes.io/projected/a3d590d1-bcc0-4c81-abb4-8aa13791bec6-kube-api-access-kvs84\") pod \"console-5f6bdbfc9-hfwlk\" (UID: \"a3d590d1-bcc0-4c81-abb4-8aa13791bec6\") " pod="openshift-console/console-5f6bdbfc9-hfwlk" Jul 06 15:41:48.458098 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:41:48.457937 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a3d590d1-bcc0-4c81-abb4-8aa13791bec6-console-serving-cert\") pod \"console-5f6bdbfc9-hfwlk\" (UID: \"a3d590d1-bcc0-4c81-abb4-8aa13791bec6\") " pod="openshift-console/console-5f6bdbfc9-hfwlk" Jul 06 15:41:48.458098 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:41:48.457970 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a3d590d1-bcc0-4c81-abb4-8aa13791bec6-service-ca\") pod \"console-5f6bdbfc9-hfwlk\" (UID: \"a3d590d1-bcc0-4c81-abb4-8aa13791bec6\") " pod="openshift-console/console-5f6bdbfc9-hfwlk" Jul 06 15:41:48.458098 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:41:48.458010 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a3d590d1-bcc0-4c81-abb4-8aa13791bec6-console-oauth-config\") pod \"console-5f6bdbfc9-hfwlk\" (UID: \"a3d590d1-bcc0-4c81-abb4-8aa13791bec6\") " pod="openshift-console/console-5f6bdbfc9-hfwlk" Jul 06 15:41:48.458098 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:41:48.458043 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a3d590d1-bcc0-4c81-abb4-8aa13791bec6-console-config\") pod \"console-5f6bdbfc9-hfwlk\" (UID: \"a3d590d1-bcc0-4c81-abb4-8aa13791bec6\") " pod="openshift-console/console-5f6bdbfc9-hfwlk" Jul 06 15:41:48.458098 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:41:48.458069 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a3d590d1-bcc0-4c81-abb4-8aa13791bec6-trusted-ca-bundle\") pod \"console-5f6bdbfc9-hfwlk\" (UID: \"a3d590d1-bcc0-4c81-abb4-8aa13791bec6\") " pod="openshift-console/console-5f6bdbfc9-hfwlk" Jul 06 15:41:48.458356 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:41:48.458126 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a3d590d1-bcc0-4c81-abb4-8aa13791bec6-oauth-serving-cert\") pod \"console-5f6bdbfc9-hfwlk\" (UID: \"a3d590d1-bcc0-4c81-abb4-8aa13791bec6\") " pod="openshift-console/console-5f6bdbfc9-hfwlk" Jul 06 15:41:48.458981 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:41:48.458950 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a3d590d1-bcc0-4c81-abb4-8aa13791bec6-oauth-serving-cert\") pod \"console-5f6bdbfc9-hfwlk\" (UID: \"a3d590d1-bcc0-4c81-abb4-8aa13791bec6\") " pod="openshift-console/console-5f6bdbfc9-hfwlk" Jul 06 15:41:48.459510 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:41:48.459478 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a3d590d1-bcc0-4c81-abb4-8aa13791bec6-console-config\") pod \"console-5f6bdbfc9-hfwlk\" (UID: \"a3d590d1-bcc0-4c81-abb4-8aa13791bec6\") " pod="openshift-console/console-5f6bdbfc9-hfwlk" Jul 06 15:41:48.459656 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:41:48.459633 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a3d590d1-bcc0-4c81-abb4-8aa13791bec6-service-ca\") pod \"console-5f6bdbfc9-hfwlk\" (UID: \"a3d590d1-bcc0-4c81-abb4-8aa13791bec6\") " pod="openshift-console/console-5f6bdbfc9-hfwlk" Jul 06 15:41:48.459930 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:41:48.459815 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a3d590d1-bcc0-4c81-abb4-8aa13791bec6-trusted-ca-bundle\") pod \"console-5f6bdbfc9-hfwlk\" (UID: \"a3d590d1-bcc0-4c81-abb4-8aa13791bec6\") " pod="openshift-console/console-5f6bdbfc9-hfwlk" Jul 06 15:41:48.461247 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:41:48.461225 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a3d590d1-bcc0-4c81-abb4-8aa13791bec6-console-oauth-config\") pod \"console-5f6bdbfc9-hfwlk\" (UID: \"a3d590d1-bcc0-4c81-abb4-8aa13791bec6\") " pod="openshift-console/console-5f6bdbfc9-hfwlk" Jul 06 15:41:48.461976 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:41:48.461955 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a3d590d1-bcc0-4c81-abb4-8aa13791bec6-console-serving-cert\") pod \"console-5f6bdbfc9-hfwlk\" (UID: \"a3d590d1-bcc0-4c81-abb4-8aa13791bec6\") " pod="openshift-console/console-5f6bdbfc9-hfwlk" Jul 06 15:41:48.477192 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:41:48.477144 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kvs84\" (UniqueName: \"kubernetes.io/projected/a3d590d1-bcc0-4c81-abb4-8aa13791bec6-kube-api-access-kvs84\") pod \"console-5f6bdbfc9-hfwlk\" (UID: \"a3d590d1-bcc0-4c81-abb4-8aa13791bec6\") " pod="openshift-console/console-5f6bdbfc9-hfwlk" Jul 06 15:41:48.613698 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:41:48.613604 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5f6bdbfc9-hfwlk" Jul 06 15:41:48.822242 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:41:48.822213 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5f6bdbfc9-hfwlk"] Jul 06 15:41:49.139317 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:41:49.139285 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda3d590d1_bcc0_4c81_abb4_8aa13791bec6.slice/crio-5ef93f8ad59c9dbe0d29935d44ecc32bfc4e26c7bfda8903c16fa574752aaa14 WatchSource:0}: Error finding container 5ef93f8ad59c9dbe0d29935d44ecc32bfc4e26c7bfda8903c16fa574752aaa14: Status 404 returned error can't find the container with id 5ef93f8ad59c9dbe0d29935d44ecc32bfc4e26c7bfda8903c16fa574752aaa14 Jul 06 15:41:49.202076 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:41:49.202034 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5f6bdbfc9-hfwlk" event={"ID":"a3d590d1-bcc0-4c81-abb4-8aa13791bec6","Type":"ContainerStarted","Data":"5ef93f8ad59c9dbe0d29935d44ecc32bfc4e26c7bfda8903c16fa574752aaa14"} Jul 06 15:41:50.208884 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:41:50.208837 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5f6bdbfc9-hfwlk" event={"ID":"a3d590d1-bcc0-4c81-abb4-8aa13791bec6","Type":"ContainerStarted","Data":"22125b2e7891d381925c1d9eac99beca39a46c316682bb0b1beda228d2f9dd90"} Jul 06 15:41:50.210366 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:41:50.210341 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-544dccfdb4-rl2tq" event={"ID":"c2d0ef2f-f717-4854-8a6b-1f09b3e0868e","Type":"ContainerStarted","Data":"404c330fe43821d10501f55b1389c9755b8d43a30446169987028c78aea15cf6"} Jul 06 15:41:50.210495 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:41:50.210397 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-544dccfdb4-rl2tq" Jul 06 15:41:50.235150 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:41:50.235094 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-5f6bdbfc9-hfwlk" podStartSLOduration=2.235079959 podStartE2EDuration="2.235079959s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 15:41:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 15:41:50.233041354 +0000 UTC m=+486.742274540" watchObservedRunningTime="2026-07-06 15:41:50.235079959 +0000 UTC m=+486.744313132" Jul 06 15:41:50.266119 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:41:50.266069 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-544dccfdb4-rl2tq" podStartSLOduration=1.695246863 podStartE2EDuration="4.266054566s" totalImagesPullingTime="2.570807703s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 15:41:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 15:41:50.265614656 +0000 UTC m=+486.774847830" watchObservedRunningTime="2026-07-06 15:41:50.266054566 +0000 UTC m=+486.775287740" Jul 06 15:41:55.181962 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:41:55.181932 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-s2j2x" Jul 06 15:41:58.614178 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:41:58.614132 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-5f6bdbfc9-hfwlk" Jul 06 15:41:58.614714 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:41:58.614243 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5f6bdbfc9-hfwlk" Jul 06 15:41:58.619009 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:41:58.618987 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-5f6bdbfc9-hfwlk" Jul 06 15:41:59.259964 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:41:59.259937 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-5f6bdbfc9-hfwlk" Jul 06 15:41:59.332287 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:41:59.332257 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-d9b4d8c44-zxfhk"] Jul 06 15:42:01.217843 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:42:01.217814 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-544dccfdb4-rl2tq" Jul 06 15:42:24.355013 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:42:24.354955 2563 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-d9b4d8c44-zxfhk" podUID="3cf66f47-ed5b-46c0-8728-9ac64f2ccdad" containerName="console" containerID="cri-o://939c033e6503b64516b03814fdaef9b360f75104e352046dccc7b3c3524ca8c7" gracePeriod=15 Jul 06 15:42:24.610777 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:42:24.610723 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-d9b4d8c44-zxfhk" Jul 06 15:42:24.691490 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:42:24.691458 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7fjmr\" (UniqueName: \"kubernetes.io/projected/3cf66f47-ed5b-46c0-8728-9ac64f2ccdad-kube-api-access-7fjmr\") pod \"3cf66f47-ed5b-46c0-8728-9ac64f2ccdad\" (UID: \"3cf66f47-ed5b-46c0-8728-9ac64f2ccdad\") " Jul 06 15:42:24.691681 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:42:24.691557 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3cf66f47-ed5b-46c0-8728-9ac64f2ccdad-console-oauth-config\") pod \"3cf66f47-ed5b-46c0-8728-9ac64f2ccdad\" (UID: \"3cf66f47-ed5b-46c0-8728-9ac64f2ccdad\") " Jul 06 15:42:24.691681 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:42:24.691590 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3cf66f47-ed5b-46c0-8728-9ac64f2ccdad-console-serving-cert\") pod \"3cf66f47-ed5b-46c0-8728-9ac64f2ccdad\" (UID: \"3cf66f47-ed5b-46c0-8728-9ac64f2ccdad\") " Jul 06 15:42:24.691681 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:42:24.691606 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3cf66f47-ed5b-46c0-8728-9ac64f2ccdad-oauth-serving-cert\") pod \"3cf66f47-ed5b-46c0-8728-9ac64f2ccdad\" (UID: \"3cf66f47-ed5b-46c0-8728-9ac64f2ccdad\") " Jul 06 15:42:24.691681 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:42:24.691634 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3cf66f47-ed5b-46c0-8728-9ac64f2ccdad-service-ca\") pod \"3cf66f47-ed5b-46c0-8728-9ac64f2ccdad\" (UID: \"3cf66f47-ed5b-46c0-8728-9ac64f2ccdad\") " Jul 06 15:42:24.691681 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:42:24.691665 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3cf66f47-ed5b-46c0-8728-9ac64f2ccdad-trusted-ca-bundle\") pod \"3cf66f47-ed5b-46c0-8728-9ac64f2ccdad\" (UID: \"3cf66f47-ed5b-46c0-8728-9ac64f2ccdad\") " Jul 06 15:42:24.691999 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:42:24.691688 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3cf66f47-ed5b-46c0-8728-9ac64f2ccdad-console-config\") pod \"3cf66f47-ed5b-46c0-8728-9ac64f2ccdad\" (UID: \"3cf66f47-ed5b-46c0-8728-9ac64f2ccdad\") " Jul 06 15:42:24.692095 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:42:24.692062 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3cf66f47-ed5b-46c0-8728-9ac64f2ccdad-service-ca" (OuterVolumeSpecName: "service-ca") pod "3cf66f47-ed5b-46c0-8728-9ac64f2ccdad" (UID: "3cf66f47-ed5b-46c0-8728-9ac64f2ccdad"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 15:42:24.692153 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:42:24.692066 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3cf66f47-ed5b-46c0-8728-9ac64f2ccdad-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "3cf66f47-ed5b-46c0-8728-9ac64f2ccdad" (UID: "3cf66f47-ed5b-46c0-8728-9ac64f2ccdad"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 15:42:24.692272 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:42:24.692118 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3cf66f47-ed5b-46c0-8728-9ac64f2ccdad-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "3cf66f47-ed5b-46c0-8728-9ac64f2ccdad" (UID: "3cf66f47-ed5b-46c0-8728-9ac64f2ccdad"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 15:42:24.692505 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:42:24.692438 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3cf66f47-ed5b-46c0-8728-9ac64f2ccdad-console-config" (OuterVolumeSpecName: "console-config") pod "3cf66f47-ed5b-46c0-8728-9ac64f2ccdad" (UID: "3cf66f47-ed5b-46c0-8728-9ac64f2ccdad"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 15:42:24.693881 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:42:24.693857 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3cf66f47-ed5b-46c0-8728-9ac64f2ccdad-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "3cf66f47-ed5b-46c0-8728-9ac64f2ccdad" (UID: "3cf66f47-ed5b-46c0-8728-9ac64f2ccdad"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 15:42:24.693881 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:42:24.693874 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3cf66f47-ed5b-46c0-8728-9ac64f2ccdad-kube-api-access-7fjmr" (OuterVolumeSpecName: "kube-api-access-7fjmr") pod "3cf66f47-ed5b-46c0-8728-9ac64f2ccdad" (UID: "3cf66f47-ed5b-46c0-8728-9ac64f2ccdad"). InnerVolumeSpecName "kube-api-access-7fjmr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 15:42:24.694005 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:42:24.693907 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3cf66f47-ed5b-46c0-8728-9ac64f2ccdad-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "3cf66f47-ed5b-46c0-8728-9ac64f2ccdad" (UID: "3cf66f47-ed5b-46c0-8728-9ac64f2ccdad"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 15:42:24.793001 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:42:24.792960 2563 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3cf66f47-ed5b-46c0-8728-9ac64f2ccdad-console-oauth-config\") on node \"ip-10-0-129-154.ec2.internal\" DevicePath \"\"" Jul 06 15:42:24.793001 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:42:24.792997 2563 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3cf66f47-ed5b-46c0-8728-9ac64f2ccdad-console-serving-cert\") on node \"ip-10-0-129-154.ec2.internal\" DevicePath \"\"" Jul 06 15:42:24.793001 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:42:24.793006 2563 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3cf66f47-ed5b-46c0-8728-9ac64f2ccdad-oauth-serving-cert\") on node \"ip-10-0-129-154.ec2.internal\" DevicePath \"\"" Jul 06 15:42:24.793001 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:42:24.793016 2563 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3cf66f47-ed5b-46c0-8728-9ac64f2ccdad-service-ca\") on node \"ip-10-0-129-154.ec2.internal\" DevicePath \"\"" Jul 06 15:42:24.793258 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:42:24.793024 2563 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3cf66f47-ed5b-46c0-8728-9ac64f2ccdad-trusted-ca-bundle\") on node \"ip-10-0-129-154.ec2.internal\" DevicePath \"\"" Jul 06 15:42:24.793258 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:42:24.793033 2563 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3cf66f47-ed5b-46c0-8728-9ac64f2ccdad-console-config\") on node \"ip-10-0-129-154.ec2.internal\" DevicePath \"\"" Jul 06 15:42:24.793258 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:42:24.793041 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7fjmr\" (UniqueName: \"kubernetes.io/projected/3cf66f47-ed5b-46c0-8728-9ac64f2ccdad-kube-api-access-7fjmr\") on node \"ip-10-0-129-154.ec2.internal\" DevicePath \"\"" Jul 06 15:42:25.385892 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:42:25.385808 2563 generic.go:356] "Generic (PLEG): container finished" podID="3cf66f47-ed5b-46c0-8728-9ac64f2ccdad" containerID="939c033e6503b64516b03814fdaef9b360f75104e352046dccc7b3c3524ca8c7" exitCode=0 Jul 06 15:42:25.385892 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:42:25.385847 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-d9b4d8c44-zxfhk" event={"ID":"3cf66f47-ed5b-46c0-8728-9ac64f2ccdad","Type":"ContainerDied","Data":"939c033e6503b64516b03814fdaef9b360f75104e352046dccc7b3c3524ca8c7"} Jul 06 15:42:25.385892 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:42:25.385857 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-d9b4d8c44-zxfhk" Jul 06 15:42:25.385892 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:42:25.385882 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-d9b4d8c44-zxfhk" event={"ID":"3cf66f47-ed5b-46c0-8728-9ac64f2ccdad","Type":"ContainerDied","Data":"f2bca1bcac8f6e8d7395e302f61a9cb292e0627222cd489010797d4d82ebd8e3"} Jul 06 15:42:25.386414 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:42:25.385901 2563 scope.go:117] "RemoveContainer" containerID="939c033e6503b64516b03814fdaef9b360f75104e352046dccc7b3c3524ca8c7" Jul 06 15:42:25.395990 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:42:25.395970 2563 scope.go:117] "RemoveContainer" containerID="939c033e6503b64516b03814fdaef9b360f75104e352046dccc7b3c3524ca8c7" Jul 06 15:42:25.396263 ip-10-0-129-154 kubenswrapper[2563]: E0706 15:42:25.396247 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"939c033e6503b64516b03814fdaef9b360f75104e352046dccc7b3c3524ca8c7\": container with ID starting with 939c033e6503b64516b03814fdaef9b360f75104e352046dccc7b3c3524ca8c7 not found: ID does not exist" containerID="939c033e6503b64516b03814fdaef9b360f75104e352046dccc7b3c3524ca8c7" Jul 06 15:42:25.396300 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:42:25.396272 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"939c033e6503b64516b03814fdaef9b360f75104e352046dccc7b3c3524ca8c7"} err="failed to get container status \"939c033e6503b64516b03814fdaef9b360f75104e352046dccc7b3c3524ca8c7\": rpc error: code = NotFound desc = could not find container \"939c033e6503b64516b03814fdaef9b360f75104e352046dccc7b3c3524ca8c7\": container with ID starting with 939c033e6503b64516b03814fdaef9b360f75104e352046dccc7b3c3524ca8c7 not found: ID does not exist" Jul 06 15:42:25.411096 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:42:25.411064 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-d9b4d8c44-zxfhk"] Jul 06 15:42:25.415084 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:42:25.415048 2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-d9b4d8c44-zxfhk"] Jul 06 15:42:25.946662 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:42:25.946629 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3cf66f47-ed5b-46c0-8728-9ac64f2ccdad" path="/var/lib/kubelet/pods/3cf66f47-ed5b-46c0-8728-9ac64f2ccdad/volumes" Jul 06 15:42:55.707190 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:42:55.707157 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0bc7cp"] Jul 06 15:42:55.707646 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:42:55.707588 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3cf66f47-ed5b-46c0-8728-9ac64f2ccdad" containerName="console" Jul 06 15:42:55.707646 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:42:55.707601 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="3cf66f47-ed5b-46c0-8728-9ac64f2ccdad" containerName="console" Jul 06 15:42:55.707716 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:42:55.707673 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="3cf66f47-ed5b-46c0-8728-9ac64f2ccdad" containerName="console" Jul 06 15:42:55.712209 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:42:55.712192 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0bc7cp" Jul 06 15:42:55.715124 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:42:55.715076 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 06 15:42:55.715238 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:42:55.715144 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 06 15:42:55.715238 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:42:55.715144 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-jk25k\"" Jul 06 15:42:55.720240 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:42:55.720221 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0bc7cp"] Jul 06 15:42:55.857251 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:42:55.857210 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f38f065d-1c16-4410-9a3b-801bc408a356-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0bc7cp\" (UID: \"f38f065d-1c16-4410-9a3b-801bc408a356\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0bc7cp" Jul 06 15:42:55.857456 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:42:55.857258 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jd582\" (UniqueName: \"kubernetes.io/projected/f38f065d-1c16-4410-9a3b-801bc408a356-kube-api-access-jd582\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0bc7cp\" (UID: \"f38f065d-1c16-4410-9a3b-801bc408a356\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0bc7cp" Jul 06 15:42:55.857456 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:42:55.857344 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f38f065d-1c16-4410-9a3b-801bc408a356-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0bc7cp\" (UID: \"f38f065d-1c16-4410-9a3b-801bc408a356\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0bc7cp" Jul 06 15:42:55.958698 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:42:55.958623 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f38f065d-1c16-4410-9a3b-801bc408a356-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0bc7cp\" (UID: \"f38f065d-1c16-4410-9a3b-801bc408a356\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0bc7cp" Jul 06 15:42:55.958698 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:42:55.958660 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jd582\" (UniqueName: \"kubernetes.io/projected/f38f065d-1c16-4410-9a3b-801bc408a356-kube-api-access-jd582\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0bc7cp\" (UID: \"f38f065d-1c16-4410-9a3b-801bc408a356\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0bc7cp" Jul 06 15:42:55.958698 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:42:55.958696 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f38f065d-1c16-4410-9a3b-801bc408a356-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0bc7cp\" (UID: \"f38f065d-1c16-4410-9a3b-801bc408a356\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0bc7cp" Jul 06 15:42:55.959053 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:42:55.959030 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f38f065d-1c16-4410-9a3b-801bc408a356-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0bc7cp\" (UID: \"f38f065d-1c16-4410-9a3b-801bc408a356\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0bc7cp" Jul 06 15:42:55.959053 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:42:55.959049 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f38f065d-1c16-4410-9a3b-801bc408a356-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0bc7cp\" (UID: \"f38f065d-1c16-4410-9a3b-801bc408a356\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0bc7cp" Jul 06 15:42:55.967529 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:42:55.967508 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jd582\" (UniqueName: \"kubernetes.io/projected/f38f065d-1c16-4410-9a3b-801bc408a356-kube-api-access-jd582\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0bc7cp\" (UID: \"f38f065d-1c16-4410-9a3b-801bc408a356\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0bc7cp" Jul 06 15:42:56.023315 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:42:56.023268 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0bc7cp" Jul 06 15:42:56.168373 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:42:56.168347 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0bc7cp"] Jul 06 15:42:56.170628 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:42:56.170601 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf38f065d_1c16_4410_9a3b_801bc408a356.slice/crio-23783eb97a62962e392649ce9acc622e88fb5617de6288cb31681cabcd65e917 WatchSource:0}: Error finding container 23783eb97a62962e392649ce9acc622e88fb5617de6288cb31681cabcd65e917: Status 404 returned error can't find the container with id 23783eb97a62962e392649ce9acc622e88fb5617de6288cb31681cabcd65e917 Jul 06 15:42:56.306085 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:42:56.306048 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qgpb2"] Jul 06 15:42:56.309607 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:42:56.309591 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qgpb2" Jul 06 15:42:56.321187 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:42:56.321158 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qgpb2"] Jul 06 15:42:56.362223 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:42:56.361971 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f7b75\" (UniqueName: \"kubernetes.io/projected/605191eb-5b8b-4737-a33d-685711524363-kube-api-access-f7b75\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qgpb2\" (UID: \"605191eb-5b8b-4737-a33d-685711524363\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qgpb2" Jul 06 15:42:56.362223 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:42:56.362015 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/605191eb-5b8b-4737-a33d-685711524363-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qgpb2\" (UID: \"605191eb-5b8b-4737-a33d-685711524363\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qgpb2" Jul 06 15:42:56.362223 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:42:56.362120 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/605191eb-5b8b-4737-a33d-685711524363-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qgpb2\" (UID: \"605191eb-5b8b-4737-a33d-685711524363\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qgpb2" Jul 06 15:42:56.463064 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:42:56.463033 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/605191eb-5b8b-4737-a33d-685711524363-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qgpb2\" (UID: \"605191eb-5b8b-4737-a33d-685711524363\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qgpb2" Jul 06 15:42:56.463255 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:42:56.463096 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f7b75\" (UniqueName: \"kubernetes.io/projected/605191eb-5b8b-4737-a33d-685711524363-kube-api-access-f7b75\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qgpb2\" (UID: \"605191eb-5b8b-4737-a33d-685711524363\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qgpb2" Jul 06 15:42:56.463255 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:42:56.463119 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/605191eb-5b8b-4737-a33d-685711524363-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qgpb2\" (UID: \"605191eb-5b8b-4737-a33d-685711524363\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qgpb2" Jul 06 15:42:56.463450 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:42:56.463401 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/605191eb-5b8b-4737-a33d-685711524363-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qgpb2\" (UID: \"605191eb-5b8b-4737-a33d-685711524363\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qgpb2" Jul 06 15:42:56.463515 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:42:56.463476 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/605191eb-5b8b-4737-a33d-685711524363-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qgpb2\" (UID: \"605191eb-5b8b-4737-a33d-685711524363\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qgpb2" Jul 06 15:42:56.473137 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:42:56.473090 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f7b75\" (UniqueName: \"kubernetes.io/projected/605191eb-5b8b-4737-a33d-685711524363-kube-api-access-f7b75\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qgpb2\" (UID: \"605191eb-5b8b-4737-a33d-685711524363\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qgpb2" Jul 06 15:42:56.543598 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:42:56.543566 2563 generic.go:356] "Generic (PLEG): container finished" podID="f38f065d-1c16-4410-9a3b-801bc408a356" containerID="8800040bdb9f4f3e400164778566cef50bc352dc2b0c4d7a7a21f6bd1e49a032" exitCode=0 Jul 06 15:42:56.543774 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:42:56.543660 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0bc7cp" event={"ID":"f38f065d-1c16-4410-9a3b-801bc408a356","Type":"ContainerDied","Data":"8800040bdb9f4f3e400164778566cef50bc352dc2b0c4d7a7a21f6bd1e49a032"} Jul 06 15:42:56.543774 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:42:56.543720 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0bc7cp" event={"ID":"f38f065d-1c16-4410-9a3b-801bc408a356","Type":"ContainerStarted","Data":"23783eb97a62962e392649ce9acc622e88fb5617de6288cb31681cabcd65e917"} Jul 06 15:42:56.664360 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:42:56.664324 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qgpb2" Jul 06 15:42:56.803202 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:42:56.803176 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qgpb2"] Jul 06 15:42:56.805181 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:42:56.805158 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod605191eb_5b8b_4737_a33d_685711524363.slice/crio-f142b6bf61256aeef48789becd9ffa34dae6258aa7a7d443743c16374de5e6bb WatchSource:0}: Error finding container f142b6bf61256aeef48789becd9ffa34dae6258aa7a7d443743c16374de5e6bb: Status 404 returned error can't find the container with id f142b6bf61256aeef48789becd9ffa34dae6258aa7a7d443743c16374de5e6bb Jul 06 15:42:56.902070 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:42:56.902028 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759vdg26"] Jul 06 15:42:56.905911 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:42:56.905892 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759vdg26" Jul 06 15:42:56.916469 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:42:56.916446 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759vdg26"] Jul 06 15:42:56.968127 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:42:56.968100 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rctgl\" (UniqueName: \"kubernetes.io/projected/7d1719d6-6fb3-4557-a9d5-a384fcf5786e-kube-api-access-rctgl\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759vdg26\" (UID: \"7d1719d6-6fb3-4557-a9d5-a384fcf5786e\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759vdg26" Jul 06 15:42:56.968242 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:42:56.968155 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7d1719d6-6fb3-4557-a9d5-a384fcf5786e-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759vdg26\" (UID: \"7d1719d6-6fb3-4557-a9d5-a384fcf5786e\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759vdg26" Jul 06 15:42:56.968286 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:42:56.968254 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7d1719d6-6fb3-4557-a9d5-a384fcf5786e-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759vdg26\" (UID: \"7d1719d6-6fb3-4557-a9d5-a384fcf5786e\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759vdg26" Jul 06 15:42:57.069067 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:42:57.068974 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7d1719d6-6fb3-4557-a9d5-a384fcf5786e-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759vdg26\" (UID: \"7d1719d6-6fb3-4557-a9d5-a384fcf5786e\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759vdg26" Jul 06 15:42:57.069250 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:42:57.069082 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rctgl\" (UniqueName: \"kubernetes.io/projected/7d1719d6-6fb3-4557-a9d5-a384fcf5786e-kube-api-access-rctgl\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759vdg26\" (UID: \"7d1719d6-6fb3-4557-a9d5-a384fcf5786e\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759vdg26" Jul 06 15:42:57.069250 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:42:57.069130 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7d1719d6-6fb3-4557-a9d5-a384fcf5786e-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759vdg26\" (UID: \"7d1719d6-6fb3-4557-a9d5-a384fcf5786e\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759vdg26" Jul 06 15:42:57.069475 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:42:57.069411 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7d1719d6-6fb3-4557-a9d5-a384fcf5786e-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759vdg26\" (UID: \"7d1719d6-6fb3-4557-a9d5-a384fcf5786e\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759vdg26" Jul 06 15:42:57.069560 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:42:57.069543 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7d1719d6-6fb3-4557-a9d5-a384fcf5786e-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759vdg26\" (UID: \"7d1719d6-6fb3-4557-a9d5-a384fcf5786e\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759vdg26" Jul 06 15:42:57.080486 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:42:57.080467 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rctgl\" (UniqueName: \"kubernetes.io/projected/7d1719d6-6fb3-4557-a9d5-a384fcf5786e-kube-api-access-rctgl\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759vdg26\" (UID: \"7d1719d6-6fb3-4557-a9d5-a384fcf5786e\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759vdg26" Jul 06 15:42:57.279720 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:42:57.279688 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759vdg26" Jul 06 15:42:57.333011 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:42:57.332974 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73fsrrm"] Jul 06 15:42:57.336950 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:42:57.336928 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73fsrrm" Jul 06 15:42:57.347263 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:42:57.347240 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73fsrrm"] Jul 06 15:42:57.465538 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:42:57.465514 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759vdg26"] Jul 06 15:42:57.467705 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:42:57.467680 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7d1719d6_6fb3_4557_a9d5_a384fcf5786e.slice/crio-52723a11f7756074d25693a2cfd067b7098974c295d3dbb9ca36e64169274863 WatchSource:0}: Error finding container 52723a11f7756074d25693a2cfd067b7098974c295d3dbb9ca36e64169274863: Status 404 returned error can't find the container with id 52723a11f7756074d25693a2cfd067b7098974c295d3dbb9ca36e64169274863 Jul 06 15:42:57.473185 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:42:57.473156 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/92d77489-516a-4f5a-9b29-72ff30ab12c1-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73fsrrm\" (UID: \"92d77489-516a-4f5a-9b29-72ff30ab12c1\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73fsrrm" Jul 06 15:42:57.473271 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:42:57.473202 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bjc7d\" (UniqueName: \"kubernetes.io/projected/92d77489-516a-4f5a-9b29-72ff30ab12c1-kube-api-access-bjc7d\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73fsrrm\" (UID: \"92d77489-516a-4f5a-9b29-72ff30ab12c1\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73fsrrm" Jul 06 15:42:57.473271 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:42:57.473263 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/92d77489-516a-4f5a-9b29-72ff30ab12c1-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73fsrrm\" (UID: \"92d77489-516a-4f5a-9b29-72ff30ab12c1\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73fsrrm" Jul 06 15:42:57.554319 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:42:57.554294 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759vdg26" event={"ID":"7d1719d6-6fb3-4557-a9d5-a384fcf5786e","Type":"ContainerStarted","Data":"52723a11f7756074d25693a2cfd067b7098974c295d3dbb9ca36e64169274863"} Jul 06 15:42:57.556168 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:42:57.556142 2563 generic.go:356] "Generic (PLEG): container finished" podID="f38f065d-1c16-4410-9a3b-801bc408a356" containerID="4f57771688b941d7734b0e6429218c3a1bf197cc31b2436c6321f5013b024fd5" exitCode=0 Jul 06 15:42:57.556278 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:42:57.556220 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0bc7cp" event={"ID":"f38f065d-1c16-4410-9a3b-801bc408a356","Type":"ContainerDied","Data":"4f57771688b941d7734b0e6429218c3a1bf197cc31b2436c6321f5013b024fd5"} Jul 06 15:42:57.557999 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:42:57.557976 2563 generic.go:356] "Generic (PLEG): container finished" podID="605191eb-5b8b-4737-a33d-685711524363" containerID="ef22d41e475ec2957ea016a72200ca1510b6fb0d79fea5d4a08ff86390a3c187" exitCode=0 Jul 06 15:42:57.558087 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:42:57.558053 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qgpb2" event={"ID":"605191eb-5b8b-4737-a33d-685711524363","Type":"ContainerDied","Data":"ef22d41e475ec2957ea016a72200ca1510b6fb0d79fea5d4a08ff86390a3c187"} Jul 06 15:42:57.558087 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:42:57.558078 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qgpb2" event={"ID":"605191eb-5b8b-4737-a33d-685711524363","Type":"ContainerStarted","Data":"f142b6bf61256aeef48789becd9ffa34dae6258aa7a7d443743c16374de5e6bb"} Jul 06 15:42:57.574796 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:42:57.574739 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/92d77489-516a-4f5a-9b29-72ff30ab12c1-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73fsrrm\" (UID: \"92d77489-516a-4f5a-9b29-72ff30ab12c1\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73fsrrm" Jul 06 15:42:57.574796 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:42:57.574788 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bjc7d\" (UniqueName: \"kubernetes.io/projected/92d77489-516a-4f5a-9b29-72ff30ab12c1-kube-api-access-bjc7d\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73fsrrm\" (UID: \"92d77489-516a-4f5a-9b29-72ff30ab12c1\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73fsrrm" Jul 06 15:42:57.574911 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:42:57.574829 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/92d77489-516a-4f5a-9b29-72ff30ab12c1-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73fsrrm\" (UID: \"92d77489-516a-4f5a-9b29-72ff30ab12c1\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73fsrrm" Jul 06 15:42:57.575103 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:42:57.575083 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/92d77489-516a-4f5a-9b29-72ff30ab12c1-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73fsrrm\" (UID: \"92d77489-516a-4f5a-9b29-72ff30ab12c1\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73fsrrm" Jul 06 15:42:57.575180 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:42:57.575154 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/92d77489-516a-4f5a-9b29-72ff30ab12c1-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73fsrrm\" (UID: \"92d77489-516a-4f5a-9b29-72ff30ab12c1\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73fsrrm" Jul 06 15:42:57.584504 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:42:57.584476 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bjc7d\" (UniqueName: \"kubernetes.io/projected/92d77489-516a-4f5a-9b29-72ff30ab12c1-kube-api-access-bjc7d\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73fsrrm\" (UID: \"92d77489-516a-4f5a-9b29-72ff30ab12c1\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73fsrrm" Jul 06 15:42:57.647013 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:42:57.646992 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73fsrrm" Jul 06 15:42:57.827279 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:42:57.827249 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73fsrrm"] Jul 06 15:42:57.828153 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:42:57.828122 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod92d77489_516a_4f5a_9b29_72ff30ab12c1.slice/crio-3caa424229ef387e4b8b4065dea2f497a40a00ec8807b11f2dad893229588f23 WatchSource:0}: Error finding container 3caa424229ef387e4b8b4065dea2f497a40a00ec8807b11f2dad893229588f23: Status 404 returned error can't find the container with id 3caa424229ef387e4b8b4065dea2f497a40a00ec8807b11f2dad893229588f23 Jul 06 15:42:58.565791 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:42:58.565755 2563 generic.go:356] "Generic (PLEG): container finished" podID="92d77489-516a-4f5a-9b29-72ff30ab12c1" containerID="196c57685f4eae0583b1aee616416479dad30bf1570d60b470eac0307cf1a0ce" exitCode=0 Jul 06 15:42:58.565979 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:42:58.565801 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73fsrrm" event={"ID":"92d77489-516a-4f5a-9b29-72ff30ab12c1","Type":"ContainerDied","Data":"196c57685f4eae0583b1aee616416479dad30bf1570d60b470eac0307cf1a0ce"} Jul 06 15:42:58.565979 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:42:58.565855 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73fsrrm" event={"ID":"92d77489-516a-4f5a-9b29-72ff30ab12c1","Type":"ContainerStarted","Data":"3caa424229ef387e4b8b4065dea2f497a40a00ec8807b11f2dad893229588f23"} Jul 06 15:42:58.567567 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:42:58.567543 2563 generic.go:356] "Generic (PLEG): container finished" podID="7d1719d6-6fb3-4557-a9d5-a384fcf5786e" containerID="2977d5c6b45b156aa1d26c259173831e38dd445de3086dba1e4df5ce9ad64441" exitCode=0 Jul 06 15:42:58.567677 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:42:58.567632 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759vdg26" event={"ID":"7d1719d6-6fb3-4557-a9d5-a384fcf5786e","Type":"ContainerDied","Data":"2977d5c6b45b156aa1d26c259173831e38dd445de3086dba1e4df5ce9ad64441"} Jul 06 15:42:58.569860 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:42:58.569788 2563 generic.go:356] "Generic (PLEG): container finished" podID="f38f065d-1c16-4410-9a3b-801bc408a356" containerID="519cea8f6d8f77de1256460775d184c05580b780a589142440ce3598e6c268f1" exitCode=0 Jul 06 15:42:58.569927 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:42:58.569862 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0bc7cp" event={"ID":"f38f065d-1c16-4410-9a3b-801bc408a356","Type":"ContainerDied","Data":"519cea8f6d8f77de1256460775d184c05580b780a589142440ce3598e6c268f1"} Jul 06 15:42:58.571749 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:42:58.571727 2563 generic.go:356] "Generic (PLEG): container finished" podID="605191eb-5b8b-4737-a33d-685711524363" containerID="6b09dac6b734f5a8402244481755104d61082c4fad29c3e9666bb2fddc70bd16" exitCode=0 Jul 06 15:42:58.571835 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:42:58.571806 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qgpb2" event={"ID":"605191eb-5b8b-4737-a33d-685711524363","Type":"ContainerDied","Data":"6b09dac6b734f5a8402244481755104d61082c4fad29c3e9666bb2fddc70bd16"} Jul 06 15:42:59.578925 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:42:59.578861 2563 generic.go:356] "Generic (PLEG): container finished" podID="605191eb-5b8b-4737-a33d-685711524363" containerID="509cd5d451bc15b3b9ded26e903a61b172779fe9ddb2afe4d4a62d5c646e2c00" exitCode=0 Jul 06 15:42:59.579320 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:42:59.578942 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qgpb2" event={"ID":"605191eb-5b8b-4737-a33d-685711524363","Type":"ContainerDied","Data":"509cd5d451bc15b3b9ded26e903a61b172779fe9ddb2afe4d4a62d5c646e2c00"} Jul 06 15:42:59.580802 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:42:59.580777 2563 generic.go:356] "Generic (PLEG): container finished" podID="7d1719d6-6fb3-4557-a9d5-a384fcf5786e" containerID="a6f5fb4b423c36f197d6270a77e0ffe14aad8c8fc0c71a9d184cf9bbcf2c6250" exitCode=0 Jul 06 15:42:59.580928 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:42:59.580869 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759vdg26" event={"ID":"7d1719d6-6fb3-4557-a9d5-a384fcf5786e","Type":"ContainerDied","Data":"a6f5fb4b423c36f197d6270a77e0ffe14aad8c8fc0c71a9d184cf9bbcf2c6250"} Jul 06 15:42:59.728703 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:42:59.728682 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0bc7cp" Jul 06 15:42:59.794441 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:42:59.794393 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jd582\" (UniqueName: \"kubernetes.io/projected/f38f065d-1c16-4410-9a3b-801bc408a356-kube-api-access-jd582\") pod \"f38f065d-1c16-4410-9a3b-801bc408a356\" (UID: \"f38f065d-1c16-4410-9a3b-801bc408a356\") " Jul 06 15:42:59.794556 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:42:59.794463 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f38f065d-1c16-4410-9a3b-801bc408a356-util\") pod \"f38f065d-1c16-4410-9a3b-801bc408a356\" (UID: \"f38f065d-1c16-4410-9a3b-801bc408a356\") " Jul 06 15:42:59.794556 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:42:59.794515 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f38f065d-1c16-4410-9a3b-801bc408a356-bundle\") pod \"f38f065d-1c16-4410-9a3b-801bc408a356\" (UID: \"f38f065d-1c16-4410-9a3b-801bc408a356\") " Jul 06 15:42:59.795001 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:42:59.794966 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f38f065d-1c16-4410-9a3b-801bc408a356-bundle" (OuterVolumeSpecName: "bundle") pod "f38f065d-1c16-4410-9a3b-801bc408a356" (UID: "f38f065d-1c16-4410-9a3b-801bc408a356"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 15:42:59.796662 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:42:59.796642 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f38f065d-1c16-4410-9a3b-801bc408a356-kube-api-access-jd582" (OuterVolumeSpecName: "kube-api-access-jd582") pod "f38f065d-1c16-4410-9a3b-801bc408a356" (UID: "f38f065d-1c16-4410-9a3b-801bc408a356"). InnerVolumeSpecName "kube-api-access-jd582". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 15:42:59.800457 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:42:59.800401 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f38f065d-1c16-4410-9a3b-801bc408a356-util" (OuterVolumeSpecName: "util") pod "f38f065d-1c16-4410-9a3b-801bc408a356" (UID: "f38f065d-1c16-4410-9a3b-801bc408a356"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 15:42:59.895679 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:42:59.895603 2563 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f38f065d-1c16-4410-9a3b-801bc408a356-bundle\") on node \"ip-10-0-129-154.ec2.internal\" DevicePath \"\"" Jul 06 15:42:59.895679 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:42:59.895630 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jd582\" (UniqueName: \"kubernetes.io/projected/f38f065d-1c16-4410-9a3b-801bc408a356-kube-api-access-jd582\") on node \"ip-10-0-129-154.ec2.internal\" DevicePath \"\"" Jul 06 15:42:59.895679 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:42:59.895639 2563 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f38f065d-1c16-4410-9a3b-801bc408a356-util\") on node \"ip-10-0-129-154.ec2.internal\" DevicePath \"\"" Jul 06 15:43:00.587517 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:43:00.587447 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0bc7cp" Jul 06 15:43:00.587517 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:43:00.587451 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0bc7cp" event={"ID":"f38f065d-1c16-4410-9a3b-801bc408a356","Type":"ContainerDied","Data":"23783eb97a62962e392649ce9acc622e88fb5617de6288cb31681cabcd65e917"} Jul 06 15:43:00.587517 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:43:00.587497 2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="23783eb97a62962e392649ce9acc622e88fb5617de6288cb31681cabcd65e917" Jul 06 15:43:00.589313 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:43:00.589287 2563 generic.go:356] "Generic (PLEG): container finished" podID="92d77489-516a-4f5a-9b29-72ff30ab12c1" containerID="ef003ed312ed411cf2f6f2571c22319997346d6883844c08d88677b6e043f090" exitCode=0 Jul 06 15:43:00.589462 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:43:00.589369 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73fsrrm" event={"ID":"92d77489-516a-4f5a-9b29-72ff30ab12c1","Type":"ContainerDied","Data":"ef003ed312ed411cf2f6f2571c22319997346d6883844c08d88677b6e043f090"} Jul 06 15:43:00.591900 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:43:00.591838 2563 generic.go:356] "Generic (PLEG): container finished" podID="7d1719d6-6fb3-4557-a9d5-a384fcf5786e" containerID="2f30b4dc3b588f3e89e1649855e36cbd96214eb427c08df4d6f16ba1ebb45353" exitCode=0 Jul 06 15:43:00.591900 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:43:00.591891 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759vdg26" event={"ID":"7d1719d6-6fb3-4557-a9d5-a384fcf5786e","Type":"ContainerDied","Data":"2f30b4dc3b588f3e89e1649855e36cbd96214eb427c08df4d6f16ba1ebb45353"} Jul 06 15:43:00.736012 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:43:00.735990 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qgpb2" Jul 06 15:43:00.803089 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:43:00.803059 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-f7b75\" (UniqueName: \"kubernetes.io/projected/605191eb-5b8b-4737-a33d-685711524363-kube-api-access-f7b75\") pod \"605191eb-5b8b-4737-a33d-685711524363\" (UID: \"605191eb-5b8b-4737-a33d-685711524363\") " Jul 06 15:43:00.803216 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:43:00.803105 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/605191eb-5b8b-4737-a33d-685711524363-util\") pod \"605191eb-5b8b-4737-a33d-685711524363\" (UID: \"605191eb-5b8b-4737-a33d-685711524363\") " Jul 06 15:43:00.803216 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:43:00.803125 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/605191eb-5b8b-4737-a33d-685711524363-bundle\") pod \"605191eb-5b8b-4737-a33d-685711524363\" (UID: \"605191eb-5b8b-4737-a33d-685711524363\") " Jul 06 15:43:00.803704 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:43:00.803669 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/605191eb-5b8b-4737-a33d-685711524363-bundle" (OuterVolumeSpecName: "bundle") pod "605191eb-5b8b-4737-a33d-685711524363" (UID: "605191eb-5b8b-4737-a33d-685711524363"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 15:43:00.805228 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:43:00.805188 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/605191eb-5b8b-4737-a33d-685711524363-kube-api-access-f7b75" (OuterVolumeSpecName: "kube-api-access-f7b75") pod "605191eb-5b8b-4737-a33d-685711524363" (UID: "605191eb-5b8b-4737-a33d-685711524363"). InnerVolumeSpecName "kube-api-access-f7b75". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 15:43:00.808639 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:43:00.808615 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/605191eb-5b8b-4737-a33d-685711524363-util" (OuterVolumeSpecName: "util") pod "605191eb-5b8b-4737-a33d-685711524363" (UID: "605191eb-5b8b-4737-a33d-685711524363"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 15:43:00.904390 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:43:00.904360 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f7b75\" (UniqueName: \"kubernetes.io/projected/605191eb-5b8b-4737-a33d-685711524363-kube-api-access-f7b75\") on node \"ip-10-0-129-154.ec2.internal\" DevicePath \"\"" Jul 06 15:43:00.904390 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:43:00.904387 2563 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/605191eb-5b8b-4737-a33d-685711524363-util\") on node \"ip-10-0-129-154.ec2.internal\" DevicePath \"\"" Jul 06 15:43:00.904587 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:43:00.904398 2563 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/605191eb-5b8b-4737-a33d-685711524363-bundle\") on node \"ip-10-0-129-154.ec2.internal\" DevicePath \"\"" Jul 06 15:43:01.598785 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:43:01.598691 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qgpb2" Jul 06 15:43:01.598785 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:43:01.598723 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qgpb2" event={"ID":"605191eb-5b8b-4737-a33d-685711524363","Type":"ContainerDied","Data":"f142b6bf61256aeef48789becd9ffa34dae6258aa7a7d443743c16374de5e6bb"} Jul 06 15:43:01.598785 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:43:01.598779 2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f142b6bf61256aeef48789becd9ffa34dae6258aa7a7d443743c16374de5e6bb" Jul 06 15:43:01.600978 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:43:01.600953 2563 generic.go:356] "Generic (PLEG): container finished" podID="92d77489-516a-4f5a-9b29-72ff30ab12c1" containerID="02009ff8c8d40c266da8c644481cb027dc9c92a2bdde9c8f4ed98ddc11d73170" exitCode=0 Jul 06 15:43:01.601110 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:43:01.601001 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73fsrrm" event={"ID":"92d77489-516a-4f5a-9b29-72ff30ab12c1","Type":"ContainerDied","Data":"02009ff8c8d40c266da8c644481cb027dc9c92a2bdde9c8f4ed98ddc11d73170"} Jul 06 15:43:01.735755 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:43:01.735730 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759vdg26" Jul 06 15:43:01.811584 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:43:01.811553 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7d1719d6-6fb3-4557-a9d5-a384fcf5786e-util\") pod \"7d1719d6-6fb3-4557-a9d5-a384fcf5786e\" (UID: \"7d1719d6-6fb3-4557-a9d5-a384fcf5786e\") " Jul 06 15:43:01.811744 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:43:01.811611 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rctgl\" (UniqueName: \"kubernetes.io/projected/7d1719d6-6fb3-4557-a9d5-a384fcf5786e-kube-api-access-rctgl\") pod \"7d1719d6-6fb3-4557-a9d5-a384fcf5786e\" (UID: \"7d1719d6-6fb3-4557-a9d5-a384fcf5786e\") " Jul 06 15:43:01.811744 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:43:01.811642 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7d1719d6-6fb3-4557-a9d5-a384fcf5786e-bundle\") pod \"7d1719d6-6fb3-4557-a9d5-a384fcf5786e\" (UID: \"7d1719d6-6fb3-4557-a9d5-a384fcf5786e\") " Jul 06 15:43:01.812138 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:43:01.812110 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7d1719d6-6fb3-4557-a9d5-a384fcf5786e-bundle" (OuterVolumeSpecName: "bundle") pod "7d1719d6-6fb3-4557-a9d5-a384fcf5786e" (UID: "7d1719d6-6fb3-4557-a9d5-a384fcf5786e"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 15:43:01.813904 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:43:01.813875 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7d1719d6-6fb3-4557-a9d5-a384fcf5786e-kube-api-access-rctgl" (OuterVolumeSpecName: "kube-api-access-rctgl") pod "7d1719d6-6fb3-4557-a9d5-a384fcf5786e" (UID: "7d1719d6-6fb3-4557-a9d5-a384fcf5786e"). InnerVolumeSpecName "kube-api-access-rctgl". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 15:43:01.816805 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:43:01.816782 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7d1719d6-6fb3-4557-a9d5-a384fcf5786e-util" (OuterVolumeSpecName: "util") pod "7d1719d6-6fb3-4557-a9d5-a384fcf5786e" (UID: "7d1719d6-6fb3-4557-a9d5-a384fcf5786e"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 15:43:01.912645 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:43:01.912612 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rctgl\" (UniqueName: \"kubernetes.io/projected/7d1719d6-6fb3-4557-a9d5-a384fcf5786e-kube-api-access-rctgl\") on node \"ip-10-0-129-154.ec2.internal\" DevicePath \"\"" Jul 06 15:43:01.912645 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:43:01.912642 2563 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7d1719d6-6fb3-4557-a9d5-a384fcf5786e-bundle\") on node \"ip-10-0-129-154.ec2.internal\" DevicePath \"\"" Jul 06 15:43:01.912645 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:43:01.912653 2563 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7d1719d6-6fb3-4557-a9d5-a384fcf5786e-util\") on node \"ip-10-0-129-154.ec2.internal\" DevicePath \"\"" Jul 06 15:43:02.607760 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:43:02.607729 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759vdg26" Jul 06 15:43:02.607760 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:43:02.607752 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759vdg26" event={"ID":"7d1719d6-6fb3-4557-a9d5-a384fcf5786e","Type":"ContainerDied","Data":"52723a11f7756074d25693a2cfd067b7098974c295d3dbb9ca36e64169274863"} Jul 06 15:43:02.608166 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:43:02.607793 2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="52723a11f7756074d25693a2cfd067b7098974c295d3dbb9ca36e64169274863" Jul 06 15:43:02.745446 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:43:02.745403 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73fsrrm" Jul 06 15:43:02.822031 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:43:02.821999 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/92d77489-516a-4f5a-9b29-72ff30ab12c1-util\") pod \"92d77489-516a-4f5a-9b29-72ff30ab12c1\" (UID: \"92d77489-516a-4f5a-9b29-72ff30ab12c1\") " Jul 06 15:43:02.822223 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:43:02.822085 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bjc7d\" (UniqueName: \"kubernetes.io/projected/92d77489-516a-4f5a-9b29-72ff30ab12c1-kube-api-access-bjc7d\") pod \"92d77489-516a-4f5a-9b29-72ff30ab12c1\" (UID: \"92d77489-516a-4f5a-9b29-72ff30ab12c1\") " Jul 06 15:43:02.822223 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:43:02.822105 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/92d77489-516a-4f5a-9b29-72ff30ab12c1-bundle\") pod \"92d77489-516a-4f5a-9b29-72ff30ab12c1\" (UID: \"92d77489-516a-4f5a-9b29-72ff30ab12c1\") " Jul 06 15:43:02.822794 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:43:02.822770 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/92d77489-516a-4f5a-9b29-72ff30ab12c1-bundle" (OuterVolumeSpecName: "bundle") pod "92d77489-516a-4f5a-9b29-72ff30ab12c1" (UID: "92d77489-516a-4f5a-9b29-72ff30ab12c1"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 15:43:02.824316 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:43:02.824281 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/92d77489-516a-4f5a-9b29-72ff30ab12c1-kube-api-access-bjc7d" (OuterVolumeSpecName: "kube-api-access-bjc7d") pod "92d77489-516a-4f5a-9b29-72ff30ab12c1" (UID: "92d77489-516a-4f5a-9b29-72ff30ab12c1"). InnerVolumeSpecName "kube-api-access-bjc7d". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 15:43:02.827363 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:43:02.827339 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/92d77489-516a-4f5a-9b29-72ff30ab12c1-util" (OuterVolumeSpecName: "util") pod "92d77489-516a-4f5a-9b29-72ff30ab12c1" (UID: "92d77489-516a-4f5a-9b29-72ff30ab12c1"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 15:43:02.922883 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:43:02.922849 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bjc7d\" (UniqueName: \"kubernetes.io/projected/92d77489-516a-4f5a-9b29-72ff30ab12c1-kube-api-access-bjc7d\") on node \"ip-10-0-129-154.ec2.internal\" DevicePath \"\"" Jul 06 15:43:02.922883 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:43:02.922880 2563 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/92d77489-516a-4f5a-9b29-72ff30ab12c1-bundle\") on node \"ip-10-0-129-154.ec2.internal\" DevicePath \"\"" Jul 06 15:43:02.923066 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:43:02.922894 2563 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/92d77489-516a-4f5a-9b29-72ff30ab12c1-util\") on node \"ip-10-0-129-154.ec2.internal\" DevicePath \"\"" Jul 06 15:43:03.615582 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:43:03.615554 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73fsrrm" Jul 06 15:43:03.615956 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:43:03.615580 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73fsrrm" event={"ID":"92d77489-516a-4f5a-9b29-72ff30ab12c1","Type":"ContainerDied","Data":"3caa424229ef387e4b8b4065dea2f497a40a00ec8807b11f2dad893229588f23"} Jul 06 15:43:03.615956 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:43:03.615624 2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3caa424229ef387e4b8b4065dea2f497a40a00ec8807b11f2dad893229588f23" Jul 06 15:43:43.857898 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:43:43.857867 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-64786d9cbb-trjfs_bf5c30e2-bf88-468b-adf2-2e8f73f136b4/cluster-monitoring-operator/0.log" Jul 06 15:43:43.858554 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:43:43.858324 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-64786d9cbb-trjfs_bf5c30e2-bf88-468b-adf2-2e8f73f136b4/cluster-monitoring-operator/0.log" Jul 06 15:43:43.861265 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:43:43.861241 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-l47bf_9ad7e968-ed2e-4cbe-9b4b-780bc9d01b88/console-operator/2.log" Jul 06 15:43:43.861754 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:43:43.861729 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-l47bf_9ad7e968-ed2e-4cbe-9b4b-780bc9d01b88/console-operator/2.log" Jul 06 15:44:19.177499 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:44:19.177457 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-58cd955675-zhzv4"] Jul 06 15:44:19.178105 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:44:19.177949 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f38f065d-1c16-4410-9a3b-801bc408a356" containerName="util" Jul 06 15:44:19.178105 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:44:19.177968 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="f38f065d-1c16-4410-9a3b-801bc408a356" containerName="util" Jul 06 15:44:19.178105 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:44:19.177982 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="605191eb-5b8b-4737-a33d-685711524363" containerName="extract" Jul 06 15:44:19.178105 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:44:19.177991 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="605191eb-5b8b-4737-a33d-685711524363" containerName="extract" Jul 06 15:44:19.178105 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:44:19.178001 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="92d77489-516a-4f5a-9b29-72ff30ab12c1" containerName="pull" Jul 06 15:44:19.178105 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:44:19.178009 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="92d77489-516a-4f5a-9b29-72ff30ab12c1" containerName="pull" Jul 06 15:44:19.178105 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:44:19.178020 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="92d77489-516a-4f5a-9b29-72ff30ab12c1" containerName="extract" Jul 06 15:44:19.178105 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:44:19.178028 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="92d77489-516a-4f5a-9b29-72ff30ab12c1" containerName="extract" Jul 06 15:44:19.178105 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:44:19.178051 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="605191eb-5b8b-4737-a33d-685711524363" containerName="util" Jul 06 15:44:19.178105 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:44:19.178059 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="605191eb-5b8b-4737-a33d-685711524363" containerName="util" Jul 06 15:44:19.178105 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:44:19.178072 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f38f065d-1c16-4410-9a3b-801bc408a356" containerName="pull" Jul 06 15:44:19.178105 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:44:19.178080 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="f38f065d-1c16-4410-9a3b-801bc408a356" containerName="pull" Jul 06 15:44:19.178105 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:44:19.178093 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="92d77489-516a-4f5a-9b29-72ff30ab12c1" containerName="util" Jul 06 15:44:19.178105 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:44:19.178102 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="92d77489-516a-4f5a-9b29-72ff30ab12c1" containerName="util" Jul 06 15:44:19.178105 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:44:19.178112 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7d1719d6-6fb3-4557-a9d5-a384fcf5786e" containerName="extract" Jul 06 15:44:19.178894 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:44:19.178120 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="7d1719d6-6fb3-4557-a9d5-a384fcf5786e" containerName="extract" Jul 06 15:44:19.178894 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:44:19.178133 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f38f065d-1c16-4410-9a3b-801bc408a356" containerName="extract" Jul 06 15:44:19.178894 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:44:19.178141 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="f38f065d-1c16-4410-9a3b-801bc408a356" containerName="extract" Jul 06 15:44:19.178894 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:44:19.178149 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="605191eb-5b8b-4737-a33d-685711524363" containerName="pull" Jul 06 15:44:19.178894 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:44:19.178157 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="605191eb-5b8b-4737-a33d-685711524363" containerName="pull" Jul 06 15:44:19.178894 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:44:19.178170 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7d1719d6-6fb3-4557-a9d5-a384fcf5786e" containerName="util" Jul 06 15:44:19.178894 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:44:19.178179 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="7d1719d6-6fb3-4557-a9d5-a384fcf5786e" containerName="util" Jul 06 15:44:19.178894 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:44:19.178188 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7d1719d6-6fb3-4557-a9d5-a384fcf5786e" containerName="pull" Jul 06 15:44:19.178894 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:44:19.178196 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="7d1719d6-6fb3-4557-a9d5-a384fcf5786e" containerName="pull" Jul 06 15:44:19.178894 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:44:19.178285 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="f38f065d-1c16-4410-9a3b-801bc408a356" containerName="extract" Jul 06 15:44:19.178894 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:44:19.178299 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="92d77489-516a-4f5a-9b29-72ff30ab12c1" containerName="extract" Jul 06 15:44:19.178894 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:44:19.178309 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="605191eb-5b8b-4737-a33d-685711524363" containerName="extract" Jul 06 15:44:19.178894 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:44:19.178319 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="7d1719d6-6fb3-4557-a9d5-a384fcf5786e" containerName="extract" Jul 06 15:44:19.181819 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:44:19.181799 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-zhzv4" Jul 06 15:44:19.186412 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:44:19.186392 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 06 15:44:19.187664 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:44:19.187632 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-5g5h8\"" Jul 06 15:44:19.187767 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:44:19.187673 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 06 15:44:19.190399 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:44:19.190380 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-zhzv4"] Jul 06 15:44:19.288563 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:44:19.288525 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-npdb8\" (UniqueName: \"kubernetes.io/projected/062864df-9e54-4b95-862a-1e39720592c8-kube-api-access-npdb8\") pod \"authorino-58cd955675-zhzv4\" (UID: \"062864df-9e54-4b95-862a-1e39720592c8\") " pod="kuadrant-system/authorino-58cd955675-zhzv4" Jul 06 15:44:19.390075 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:44:19.390047 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-npdb8\" (UniqueName: \"kubernetes.io/projected/062864df-9e54-4b95-862a-1e39720592c8-kube-api-access-npdb8\") pod \"authorino-58cd955675-zhzv4\" (UID: \"062864df-9e54-4b95-862a-1e39720592c8\") " pod="kuadrant-system/authorino-58cd955675-zhzv4" Jul 06 15:44:19.417258 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:44:19.417217 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-npdb8\" (UniqueName: \"kubernetes.io/projected/062864df-9e54-4b95-862a-1e39720592c8-kube-api-access-npdb8\") pod \"authorino-58cd955675-zhzv4\" (UID: \"062864df-9e54-4b95-862a-1e39720592c8\") " pod="kuadrant-system/authorino-58cd955675-zhzv4" Jul 06 15:44:19.491016 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:44:19.490946 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-zhzv4" Jul 06 15:44:19.632471 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:44:19.632440 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-zhzv4"] Jul 06 15:44:19.634156 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:44:19.634125 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod062864df_9e54_4b95_862a_1e39720592c8.slice/crio-8ff593263bc95c21972cd76d23087c6db220089d94f18eae17ca468e9e39605b WatchSource:0}: Error finding container 8ff593263bc95c21972cd76d23087c6db220089d94f18eae17ca468e9e39605b: Status 404 returned error can't find the container with id 8ff593263bc95c21972cd76d23087c6db220089d94f18eae17ca468e9e39605b Jul 06 15:44:20.026276 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:44:20.026243 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-zhzv4" event={"ID":"062864df-9e54-4b95-862a-1e39720592c8","Type":"ContainerStarted","Data":"8ff593263bc95c21972cd76d23087c6db220089d94f18eae17ca468e9e39605b"} Jul 06 15:44:24.052779 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:44:24.052736 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-zhzv4" event={"ID":"062864df-9e54-4b95-862a-1e39720592c8","Type":"ContainerStarted","Data":"15a8d24e0ab750661eb66f41eddb5a97588c8df9000d5b9eab6632f570d087e8"} Jul 06 15:44:24.076278 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:44:24.076225 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-58cd955675-zhzv4" podStartSLOduration=1.4773908900000001 podStartE2EDuration="5.076210303s" totalImagesPullingTime="3.598819413s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 15:44:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 15:44:24.07457456 +0000 UTC m=+640.583807735" watchObservedRunningTime="2026-07-06 15:44:24.076210303 +0000 UTC m=+640.585443474" Jul 06 15:44:25.171588 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:44:25.171504 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-zhzv4"] Jul 06 15:44:26.063374 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:44:26.063339 2563 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-58cd955675-zhzv4" podUID="062864df-9e54-4b95-862a-1e39720592c8" containerName="authorino" containerID="cri-o://15a8d24e0ab750661eb66f41eddb5a97588c8df9000d5b9eab6632f570d087e8" gracePeriod=30 Jul 06 15:44:26.312794 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:44:26.312768 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-zhzv4" Jul 06 15:44:26.354567 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:44:26.354487 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-npdb8\" (UniqueName: \"kubernetes.io/projected/062864df-9e54-4b95-862a-1e39720592c8-kube-api-access-npdb8\") pod \"062864df-9e54-4b95-862a-1e39720592c8\" (UID: \"062864df-9e54-4b95-862a-1e39720592c8\") " Jul 06 15:44:26.356682 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:44:26.356657 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/062864df-9e54-4b95-862a-1e39720592c8-kube-api-access-npdb8" (OuterVolumeSpecName: "kube-api-access-npdb8") pod "062864df-9e54-4b95-862a-1e39720592c8" (UID: "062864df-9e54-4b95-862a-1e39720592c8"). InnerVolumeSpecName "kube-api-access-npdb8". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 15:44:26.455284 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:44:26.455249 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-npdb8\" (UniqueName: \"kubernetes.io/projected/062864df-9e54-4b95-862a-1e39720592c8-kube-api-access-npdb8\") on node \"ip-10-0-129-154.ec2.internal\" DevicePath \"\"" Jul 06 15:44:27.070006 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:44:27.069972 2563 generic.go:356] "Generic (PLEG): container finished" podID="062864df-9e54-4b95-862a-1e39720592c8" containerID="15a8d24e0ab750661eb66f41eddb5a97588c8df9000d5b9eab6632f570d087e8" exitCode=0 Jul 06 15:44:27.070183 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:44:27.070009 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-zhzv4" event={"ID":"062864df-9e54-4b95-862a-1e39720592c8","Type":"ContainerDied","Data":"15a8d24e0ab750661eb66f41eddb5a97588c8df9000d5b9eab6632f570d087e8"} Jul 06 15:44:27.070183 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:44:27.070032 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-zhzv4" Jul 06 15:44:27.070183 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:44:27.070064 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-zhzv4" event={"ID":"062864df-9e54-4b95-862a-1e39720592c8","Type":"ContainerDied","Data":"8ff593263bc95c21972cd76d23087c6db220089d94f18eae17ca468e9e39605b"} Jul 06 15:44:27.070183 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:44:27.070086 2563 scope.go:117] "RemoveContainer" containerID="15a8d24e0ab750661eb66f41eddb5a97588c8df9000d5b9eab6632f570d087e8" Jul 06 15:44:27.080436 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:44:27.080401 2563 scope.go:117] "RemoveContainer" containerID="15a8d24e0ab750661eb66f41eddb5a97588c8df9000d5b9eab6632f570d087e8" Jul 06 15:44:27.080703 ip-10-0-129-154 kubenswrapper[2563]: E0706 15:44:27.080684 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"15a8d24e0ab750661eb66f41eddb5a97588c8df9000d5b9eab6632f570d087e8\": container with ID starting with 15a8d24e0ab750661eb66f41eddb5a97588c8df9000d5b9eab6632f570d087e8 not found: ID does not exist" containerID="15a8d24e0ab750661eb66f41eddb5a97588c8df9000d5b9eab6632f570d087e8" Jul 06 15:44:27.080767 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:44:27.080712 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"15a8d24e0ab750661eb66f41eddb5a97588c8df9000d5b9eab6632f570d087e8"} err="failed to get container status \"15a8d24e0ab750661eb66f41eddb5a97588c8df9000d5b9eab6632f570d087e8\": rpc error: code = NotFound desc = could not find container \"15a8d24e0ab750661eb66f41eddb5a97588c8df9000d5b9eab6632f570d087e8\": container with ID starting with 15a8d24e0ab750661eb66f41eddb5a97588c8df9000d5b9eab6632f570d087e8 not found: ID does not exist" Jul 06 15:44:27.093688 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:44:27.093646 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-zhzv4"] Jul 06 15:44:27.097639 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:44:27.097618 2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-58cd955675-zhzv4"] Jul 06 15:44:27.945594 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:44:27.945553 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="062864df-9e54-4b95-862a-1e39720592c8" path="/var/lib/kubelet/pods/062864df-9e54-4b95-862a-1e39720592c8/volumes" Jul 06 15:44:52.805204 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:44:52.805169 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5d4b67c446-qkg2t"] Jul 06 15:44:52.805709 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:44:52.805547 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="062864df-9e54-4b95-862a-1e39720592c8" containerName="authorino" Jul 06 15:44:52.805709 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:44:52.805562 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="062864df-9e54-4b95-862a-1e39720592c8" containerName="authorino" Jul 06 15:44:52.805709 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:44:52.805621 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="062864df-9e54-4b95-862a-1e39720592c8" containerName="authorino" Jul 06 15:44:52.809745 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:44:52.809730 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-qkg2t" Jul 06 15:44:52.812632 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:44:52.812608 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 06 15:44:52.813549 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:44:52.813526 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-5g5h8\"" Jul 06 15:44:52.813667 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:44:52.813584 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 06 15:44:52.815918 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:44:52.815880 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-qkg2t"] Jul 06 15:44:52.991172 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:44:52.991139 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7dgz2\" (UniqueName: \"kubernetes.io/projected/c367a2ab-08f0-429d-b03d-b81e4d75e055-kube-api-access-7dgz2\") pod \"authorino-5d4b67c446-qkg2t\" (UID: \"c367a2ab-08f0-429d-b03d-b81e4d75e055\") " pod="kuadrant-system/authorino-5d4b67c446-qkg2t" Jul 06 15:44:53.091483 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:44:53.091376 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-qkg2t"] Jul 06 15:44:53.091682 ip-10-0-129-154 kubenswrapper[2563]: E0706 15:44:53.091657 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-7dgz2], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-5d4b67c446-qkg2t" podUID="c367a2ab-08f0-429d-b03d-b81e4d75e055" Jul 06 15:44:53.091725 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:44:53.091705 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7dgz2\" (UniqueName: \"kubernetes.io/projected/c367a2ab-08f0-429d-b03d-b81e4d75e055-kube-api-access-7dgz2\") pod \"authorino-5d4b67c446-qkg2t\" (UID: \"c367a2ab-08f0-429d-b03d-b81e4d75e055\") " pod="kuadrant-system/authorino-5d4b67c446-qkg2t" Jul 06 15:44:53.104951 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:44:53.104922 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7dgz2\" (UniqueName: \"kubernetes.io/projected/c367a2ab-08f0-429d-b03d-b81e4d75e055-kube-api-access-7dgz2\") pod \"authorino-5d4b67c446-qkg2t\" (UID: \"c367a2ab-08f0-429d-b03d-b81e4d75e055\") " pod="kuadrant-system/authorino-5d4b67c446-qkg2t" Jul 06 15:44:53.123831 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:44:53.123809 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-676969c7b4-gzp6w"] Jul 06 15:44:53.127275 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:44:53.127261 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-676969c7b4-gzp6w" Jul 06 15:44:53.129795 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:44:53.129777 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\"" Jul 06 15:44:53.138107 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:44:53.138082 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-676969c7b4-gzp6w"] Jul 06 15:44:53.205398 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:44:53.205366 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-qkg2t" Jul 06 15:44:53.212256 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:44:53.212234 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-qkg2t" Jul 06 15:44:53.293386 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:44:53.293351 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rsg8t\" (UniqueName: \"kubernetes.io/projected/3d478fe7-51aa-49aa-8c96-b8ae66640974-kube-api-access-rsg8t\") pod \"authorino-676969c7b4-gzp6w\" (UID: \"3d478fe7-51aa-49aa-8c96-b8ae66640974\") " pod="kuadrant-system/authorino-676969c7b4-gzp6w" Jul 06 15:44:53.293386 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:44:53.293392 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/3d478fe7-51aa-49aa-8c96-b8ae66640974-tls-cert\") pod \"authorino-676969c7b4-gzp6w\" (UID: \"3d478fe7-51aa-49aa-8c96-b8ae66640974\") " pod="kuadrant-system/authorino-676969c7b4-gzp6w" Jul 06 15:44:53.394415 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:44:53.394332 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7dgz2\" (UniqueName: \"kubernetes.io/projected/c367a2ab-08f0-429d-b03d-b81e4d75e055-kube-api-access-7dgz2\") pod \"c367a2ab-08f0-429d-b03d-b81e4d75e055\" (UID: \"c367a2ab-08f0-429d-b03d-b81e4d75e055\") " Jul 06 15:44:53.394603 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:44:53.394569 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rsg8t\" (UniqueName: \"kubernetes.io/projected/3d478fe7-51aa-49aa-8c96-b8ae66640974-kube-api-access-rsg8t\") pod \"authorino-676969c7b4-gzp6w\" (UID: \"3d478fe7-51aa-49aa-8c96-b8ae66640974\") " pod="kuadrant-system/authorino-676969c7b4-gzp6w" Jul 06 15:44:53.394674 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:44:53.394620 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/3d478fe7-51aa-49aa-8c96-b8ae66640974-tls-cert\") pod \"authorino-676969c7b4-gzp6w\" (UID: \"3d478fe7-51aa-49aa-8c96-b8ae66640974\") " pod="kuadrant-system/authorino-676969c7b4-gzp6w" Jul 06 15:44:53.396632 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:44:53.396602 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c367a2ab-08f0-429d-b03d-b81e4d75e055-kube-api-access-7dgz2" (OuterVolumeSpecName: "kube-api-access-7dgz2") pod "c367a2ab-08f0-429d-b03d-b81e4d75e055" (UID: "c367a2ab-08f0-429d-b03d-b81e4d75e055"). InnerVolumeSpecName "kube-api-access-7dgz2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 15:44:53.397107 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:44:53.397092 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/3d478fe7-51aa-49aa-8c96-b8ae66640974-tls-cert\") pod \"authorino-676969c7b4-gzp6w\" (UID: \"3d478fe7-51aa-49aa-8c96-b8ae66640974\") " pod="kuadrant-system/authorino-676969c7b4-gzp6w" Jul 06 15:44:53.403553 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:44:53.403529 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rsg8t\" (UniqueName: \"kubernetes.io/projected/3d478fe7-51aa-49aa-8c96-b8ae66640974-kube-api-access-rsg8t\") pod \"authorino-676969c7b4-gzp6w\" (UID: \"3d478fe7-51aa-49aa-8c96-b8ae66640974\") " pod="kuadrant-system/authorino-676969c7b4-gzp6w" Jul 06 15:44:53.437098 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:44:53.437071 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-676969c7b4-gzp6w" Jul 06 15:44:53.495261 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:44:53.495234 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7dgz2\" (UniqueName: \"kubernetes.io/projected/c367a2ab-08f0-429d-b03d-b81e4d75e055-kube-api-access-7dgz2\") on node \"ip-10-0-129-154.ec2.internal\" DevicePath \"\"" Jul 06 15:44:53.570861 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:44:53.570835 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-676969c7b4-gzp6w"] Jul 06 15:44:53.572910 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:44:53.572873 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3d478fe7_51aa_49aa_8c96_b8ae66640974.slice/crio-f164d322162bbe8fadb2c65bbd8cb22612fe09cc9a1327baaf22d93c1a013c4b WatchSource:0}: Error finding container f164d322162bbe8fadb2c65bbd8cb22612fe09cc9a1327baaf22d93c1a013c4b: Status 404 returned error can't find the container with id f164d322162bbe8fadb2c65bbd8cb22612fe09cc9a1327baaf22d93c1a013c4b Jul 06 15:44:54.212481 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:44:54.212449 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-676969c7b4-gzp6w" event={"ID":"3d478fe7-51aa-49aa-8c96-b8ae66640974","Type":"ContainerStarted","Data":"d73cf6911b423fa8cbe9715c04e44f6e86f4a475958e4fab04777ee80b0b13d4"} Jul 06 15:44:54.212879 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:44:54.212490 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-676969c7b4-gzp6w" event={"ID":"3d478fe7-51aa-49aa-8c96-b8ae66640974","Type":"ContainerStarted","Data":"f164d322162bbe8fadb2c65bbd8cb22612fe09cc9a1327baaf22d93c1a013c4b"} Jul 06 15:44:54.212879 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:44:54.212496 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-qkg2t" Jul 06 15:44:54.236018 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:44:54.235973 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-676969c7b4-gzp6w" podStartSLOduration=0.933263592 podStartE2EDuration="1.235958523s" totalImagesPullingTime="302.694931ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 15:44:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 15:44:54.233742698 +0000 UTC m=+670.742975872" watchObservedRunningTime="2026-07-06 15:44:54.235958523 +0000 UTC m=+670.745191713" Jul 06 15:44:54.268082 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:44:54.268047 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-qkg2t"] Jul 06 15:44:54.290597 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:44:54.290557 2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-qkg2t"] Jul 06 15:44:55.945867 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:44:55.945835 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c367a2ab-08f0-429d-b03d-b81e4d75e055" path="/var/lib/kubelet/pods/c367a2ab-08f0-429d-b03d-b81e4d75e055/volumes" Jul 06 15:46:22.646559 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:46:22.646523 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-7sgfq"] Jul 06 15:46:22.650330 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:46:22.650314 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-7sgfq" Jul 06 15:46:22.668788 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:46:22.668755 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 06 15:46:22.670312 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:46:22.670287 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\"" Jul 06 15:46:22.670312 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:46:22.670304 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 06 15:46:22.670517 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:46:22.670304 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-cmsdq\"" Jul 06 15:46:22.670940 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:46:22.670919 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-7sgfq"] Jul 06 15:46:22.772968 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:46:22.772928 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d4c96862-d4b8-4e69-a536-b13867b7f2bf-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-7sgfq\" (UID: \"d4c96862-d4b8-4e69-a536-b13867b7f2bf\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-7sgfq" Jul 06 15:46:22.773134 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:46:22.772993 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d4c96862-d4b8-4e69-a536-b13867b7f2bf-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-7sgfq\" (UID: \"d4c96862-d4b8-4e69-a536-b13867b7f2bf\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-7sgfq" Jul 06 15:46:22.773134 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:46:22.773061 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d4c96862-d4b8-4e69-a536-b13867b7f2bf-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-7sgfq\" (UID: \"d4c96862-d4b8-4e69-a536-b13867b7f2bf\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-7sgfq" Jul 06 15:46:22.773134 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:46:22.773111 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d4c96862-d4b8-4e69-a536-b13867b7f2bf-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-7sgfq\" (UID: \"d4c96862-d4b8-4e69-a536-b13867b7f2bf\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-7sgfq" Jul 06 15:46:22.773284 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:46:22.773148 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p55wr\" (UniqueName: \"kubernetes.io/projected/d4c96862-d4b8-4e69-a536-b13867b7f2bf-kube-api-access-p55wr\") pod \"e2e-distinct-simulated-kserve-59975d69f9-7sgfq\" (UID: \"d4c96862-d4b8-4e69-a536-b13867b7f2bf\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-7sgfq" Jul 06 15:46:22.773284 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:46:22.773270 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d4c96862-d4b8-4e69-a536-b13867b7f2bf-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-7sgfq\" (UID: \"d4c96862-d4b8-4e69-a536-b13867b7f2bf\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-7sgfq" Jul 06 15:46:22.874109 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:46:22.874071 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d4c96862-d4b8-4e69-a536-b13867b7f2bf-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-7sgfq\" (UID: \"d4c96862-d4b8-4e69-a536-b13867b7f2bf\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-7sgfq" Jul 06 15:46:22.874109 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:46:22.874110 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d4c96862-d4b8-4e69-a536-b13867b7f2bf-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-7sgfq\" (UID: \"d4c96862-d4b8-4e69-a536-b13867b7f2bf\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-7sgfq" Jul 06 15:46:22.874348 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:46:22.874142 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d4c96862-d4b8-4e69-a536-b13867b7f2bf-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-7sgfq\" (UID: \"d4c96862-d4b8-4e69-a536-b13867b7f2bf\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-7sgfq" Jul 06 15:46:22.874348 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:46:22.874183 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d4c96862-d4b8-4e69-a536-b13867b7f2bf-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-7sgfq\" (UID: \"d4c96862-d4b8-4e69-a536-b13867b7f2bf\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-7sgfq" Jul 06 15:46:22.874348 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:46:22.874200 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d4c96862-d4b8-4e69-a536-b13867b7f2bf-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-7sgfq\" (UID: \"d4c96862-d4b8-4e69-a536-b13867b7f2bf\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-7sgfq" Jul 06 15:46:22.874348 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:46:22.874229 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p55wr\" (UniqueName: \"kubernetes.io/projected/d4c96862-d4b8-4e69-a536-b13867b7f2bf-kube-api-access-p55wr\") pod \"e2e-distinct-simulated-kserve-59975d69f9-7sgfq\" (UID: \"d4c96862-d4b8-4e69-a536-b13867b7f2bf\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-7sgfq" Jul 06 15:46:22.874594 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:46:22.874568 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d4c96862-d4b8-4e69-a536-b13867b7f2bf-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-7sgfq\" (UID: \"d4c96862-d4b8-4e69-a536-b13867b7f2bf\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-7sgfq" Jul 06 15:46:22.874658 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:46:22.874615 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d4c96862-d4b8-4e69-a536-b13867b7f2bf-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-7sgfq\" (UID: \"d4c96862-d4b8-4e69-a536-b13867b7f2bf\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-7sgfq" Jul 06 15:46:22.874715 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:46:22.874691 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d4c96862-d4b8-4e69-a536-b13867b7f2bf-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-7sgfq\" (UID: \"d4c96862-d4b8-4e69-a536-b13867b7f2bf\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-7sgfq" Jul 06 15:46:22.876771 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:46:22.876749 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d4c96862-d4b8-4e69-a536-b13867b7f2bf-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-7sgfq\" (UID: \"d4c96862-d4b8-4e69-a536-b13867b7f2bf\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-7sgfq" Jul 06 15:46:22.876965 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:46:22.876947 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d4c96862-d4b8-4e69-a536-b13867b7f2bf-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-7sgfq\" (UID: \"d4c96862-d4b8-4e69-a536-b13867b7f2bf\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-7sgfq" Jul 06 15:46:22.886328 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:46:22.886301 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p55wr\" (UniqueName: \"kubernetes.io/projected/d4c96862-d4b8-4e69-a536-b13867b7f2bf-kube-api-access-p55wr\") pod \"e2e-distinct-simulated-kserve-59975d69f9-7sgfq\" (UID: \"d4c96862-d4b8-4e69-a536-b13867b7f2bf\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-7sgfq" Jul 06 15:46:22.960444 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:46:22.960399 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-7sgfq" Jul 06 15:46:23.111654 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:46:23.111553 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-7sgfq"] Jul 06 15:46:23.114289 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:46:23.114258 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd4c96862_d4b8_4e69_a536_b13867b7f2bf.slice/crio-3de14694ce602b4bcada23894860f2d3a94198bdd5332df85e2f01aff741b207 WatchSource:0}: Error finding container 3de14694ce602b4bcada23894860f2d3a94198bdd5332df85e2f01aff741b207: Status 404 returned error can't find the container with id 3de14694ce602b4bcada23894860f2d3a94198bdd5332df85e2f01aff741b207 Jul 06 15:46:23.115580 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:46:23.115561 2563 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 06 15:46:23.699826 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:46:23.699791 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-7sgfq" event={"ID":"d4c96862-d4b8-4e69-a536-b13867b7f2bf","Type":"ContainerStarted","Data":"3de14694ce602b4bcada23894860f2d3a94198bdd5332df85e2f01aff741b207"} Jul 06 15:46:27.728724 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:46:27.728675 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-nx96r"] Jul 06 15:46:27.733322 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:46:27.733303 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-nx96r" Jul 06 15:46:27.736638 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:46:27.736617 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\"" Jul 06 15:46:27.744608 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:46:27.744588 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-nx96r"] Jul 06 15:46:27.926229 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:46:27.926193 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6926bf8b-8350-4ce6-aab1-92f7ecf6e970-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-nx96r\" (UID: \"6926bf8b-8350-4ce6-aab1-92f7ecf6e970\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-nx96r" Jul 06 15:46:27.926229 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:46:27.926233 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6926bf8b-8350-4ce6-aab1-92f7ecf6e970-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-nx96r\" (UID: \"6926bf8b-8350-4ce6-aab1-92f7ecf6e970\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-nx96r" Jul 06 15:46:27.926507 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:46:27.926269 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6926bf8b-8350-4ce6-aab1-92f7ecf6e970-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-nx96r\" (UID: \"6926bf8b-8350-4ce6-aab1-92f7ecf6e970\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-nx96r" Jul 06 15:46:27.926507 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:46:27.926364 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-plhrr\" (UniqueName: \"kubernetes.io/projected/6926bf8b-8350-4ce6-aab1-92f7ecf6e970-kube-api-access-plhrr\") pod \"e2e-trlp-test-simulated-kserve-598466548d-nx96r\" (UID: \"6926bf8b-8350-4ce6-aab1-92f7ecf6e970\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-nx96r" Jul 06 15:46:27.926507 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:46:27.926468 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6926bf8b-8350-4ce6-aab1-92f7ecf6e970-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-nx96r\" (UID: \"6926bf8b-8350-4ce6-aab1-92f7ecf6e970\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-nx96r" Jul 06 15:46:27.926507 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:46:27.926500 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6926bf8b-8350-4ce6-aab1-92f7ecf6e970-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-nx96r\" (UID: \"6926bf8b-8350-4ce6-aab1-92f7ecf6e970\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-nx96r" Jul 06 15:46:28.027740 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:46:28.027651 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6926bf8b-8350-4ce6-aab1-92f7ecf6e970-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-nx96r\" (UID: \"6926bf8b-8350-4ce6-aab1-92f7ecf6e970\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-nx96r" Jul 06 15:46:28.027740 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:46:28.027700 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-plhrr\" (UniqueName: \"kubernetes.io/projected/6926bf8b-8350-4ce6-aab1-92f7ecf6e970-kube-api-access-plhrr\") pod \"e2e-trlp-test-simulated-kserve-598466548d-nx96r\" (UID: \"6926bf8b-8350-4ce6-aab1-92f7ecf6e970\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-nx96r" Jul 06 15:46:28.027740 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:46:28.027741 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6926bf8b-8350-4ce6-aab1-92f7ecf6e970-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-nx96r\" (UID: \"6926bf8b-8350-4ce6-aab1-92f7ecf6e970\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-nx96r" Jul 06 15:46:28.028006 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:46:28.027764 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6926bf8b-8350-4ce6-aab1-92f7ecf6e970-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-nx96r\" (UID: \"6926bf8b-8350-4ce6-aab1-92f7ecf6e970\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-nx96r" Jul 06 15:46:28.028006 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:46:28.027790 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6926bf8b-8350-4ce6-aab1-92f7ecf6e970-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-nx96r\" (UID: \"6926bf8b-8350-4ce6-aab1-92f7ecf6e970\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-nx96r" Jul 06 15:46:28.028006 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:46:28.027806 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6926bf8b-8350-4ce6-aab1-92f7ecf6e970-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-nx96r\" (UID: \"6926bf8b-8350-4ce6-aab1-92f7ecf6e970\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-nx96r" Jul 06 15:46:28.028266 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:46:28.028237 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6926bf8b-8350-4ce6-aab1-92f7ecf6e970-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-nx96r\" (UID: \"6926bf8b-8350-4ce6-aab1-92f7ecf6e970\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-nx96r" Jul 06 15:46:28.028405 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:46:28.028274 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6926bf8b-8350-4ce6-aab1-92f7ecf6e970-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-nx96r\" (UID: \"6926bf8b-8350-4ce6-aab1-92f7ecf6e970\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-nx96r" Jul 06 15:46:28.028405 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:46:28.028328 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6926bf8b-8350-4ce6-aab1-92f7ecf6e970-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-nx96r\" (UID: \"6926bf8b-8350-4ce6-aab1-92f7ecf6e970\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-nx96r" Jul 06 15:46:28.030227 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:46:28.030207 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6926bf8b-8350-4ce6-aab1-92f7ecf6e970-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-nx96r\" (UID: \"6926bf8b-8350-4ce6-aab1-92f7ecf6e970\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-nx96r" Jul 06 15:46:28.030379 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:46:28.030364 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6926bf8b-8350-4ce6-aab1-92f7ecf6e970-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-nx96r\" (UID: \"6926bf8b-8350-4ce6-aab1-92f7ecf6e970\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-nx96r" Jul 06 15:46:28.036473 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:46:28.036450 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-plhrr\" (UniqueName: \"kubernetes.io/projected/6926bf8b-8350-4ce6-aab1-92f7ecf6e970-kube-api-access-plhrr\") pod \"e2e-trlp-test-simulated-kserve-598466548d-nx96r\" (UID: \"6926bf8b-8350-4ce6-aab1-92f7ecf6e970\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-nx96r" Jul 06 15:46:28.045340 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:46:28.045321 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-nx96r" Jul 06 15:46:28.186666 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:46:28.186638 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-nx96r"] Jul 06 15:46:28.187716 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:46:28.187693 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6926bf8b_8350_4ce6_aab1_92f7ecf6e970.slice/crio-fa242fa2fdb154b598d1a57bc5d25a9a29fb99ceaa6811da532911c302d6bd53 WatchSource:0}: Error finding container fa242fa2fdb154b598d1a57bc5d25a9a29fb99ceaa6811da532911c302d6bd53: Status 404 returned error can't find the container with id fa242fa2fdb154b598d1a57bc5d25a9a29fb99ceaa6811da532911c302d6bd53 Jul 06 15:46:28.730226 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:46:28.730185 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-nx96r" event={"ID":"6926bf8b-8350-4ce6-aab1-92f7ecf6e970","Type":"ContainerStarted","Data":"fa242fa2fdb154b598d1a57bc5d25a9a29fb99ceaa6811da532911c302d6bd53"} Jul 06 15:46:29.737414 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:46:29.737377 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-nx96r" event={"ID":"6926bf8b-8350-4ce6-aab1-92f7ecf6e970","Type":"ContainerStarted","Data":"1274f0731b595b154c526ac927d800c49b49329ca6552bd83c489dca1bf2f12f"} Jul 06 15:46:29.760807 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:46:29.760755 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-nx96r" podStartSLOduration=1.57215806 podStartE2EDuration="2.76073867s" totalImagesPullingTime="1.18858061s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 15:46:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 15:46:29.75929106 +0000 UTC m=+766.268524254" watchObservedRunningTime="2026-07-06 15:46:29.76073867 +0000 UTC m=+766.269971822" Jul 06 15:46:31.750332 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:46:31.750287 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-7sgfq" event={"ID":"d4c96862-d4b8-4e69-a536-b13867b7f2bf","Type":"ContainerStarted","Data":"9d83d8e1f2133d336cf1465e004183c71815f64570249e3a8d55d89b709e1709"} Jul 06 15:46:31.772567 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:46:31.772518 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-7sgfq" podStartSLOduration=1.276777046 podStartE2EDuration="9.772501034s" totalImagesPullingTime="8.495723988s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 15:46:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 15:46:31.769603523 +0000 UTC m=+768.278836698" watchObservedRunningTime="2026-07-06 15:46:31.772501034 +0000 UTC m=+768.281734209" Jul 06 15:46:32.961432 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:46:32.961397 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-7sgfq" Jul 06 15:46:32.961870 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:46:32.961466 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-7sgfq" Jul 06 15:46:32.974280 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:46:32.974256 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-7sgfq" Jul 06 15:46:33.772182 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:46:33.772155 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-7sgfq" Jul 06 15:46:38.046314 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:46:38.046276 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-nx96r" Jul 06 15:46:38.046777 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:46:38.046343 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-nx96r" Jul 06 15:46:38.058721 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:46:38.058695 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-nx96r" Jul 06 15:46:38.799818 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:46:38.799785 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-nx96r" Jul 06 15:48:11.346531 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:48:11.346501 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5497d684cc-fpvql"] Jul 06 15:48:11.349856 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:48:11.349839 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5497d684cc-fpvql" Jul 06 15:48:11.359489 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:48:11.359463 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5497d684cc-fpvql"] Jul 06 15:48:11.479921 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:48:11.479886 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5wdq4\" (UniqueName: \"kubernetes.io/projected/3d9bb9b2-66b1-4a36-aa3d-e48975c8f61d-kube-api-access-5wdq4\") pod \"authorino-5497d684cc-fpvql\" (UID: \"3d9bb9b2-66b1-4a36-aa3d-e48975c8f61d\") " pod="kuadrant-system/authorino-5497d684cc-fpvql" Jul 06 15:48:11.480092 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:48:11.479973 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/3d9bb9b2-66b1-4a36-aa3d-e48975c8f61d-tls-cert\") pod \"authorino-5497d684cc-fpvql\" (UID: \"3d9bb9b2-66b1-4a36-aa3d-e48975c8f61d\") " pod="kuadrant-system/authorino-5497d684cc-fpvql" Jul 06 15:48:11.580720 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:48:11.580690 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/3d9bb9b2-66b1-4a36-aa3d-e48975c8f61d-tls-cert\") pod \"authorino-5497d684cc-fpvql\" (UID: \"3d9bb9b2-66b1-4a36-aa3d-e48975c8f61d\") " pod="kuadrant-system/authorino-5497d684cc-fpvql" Jul 06 15:48:11.580885 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:48:11.580736 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5wdq4\" (UniqueName: \"kubernetes.io/projected/3d9bb9b2-66b1-4a36-aa3d-e48975c8f61d-kube-api-access-5wdq4\") pod \"authorino-5497d684cc-fpvql\" (UID: \"3d9bb9b2-66b1-4a36-aa3d-e48975c8f61d\") " pod="kuadrant-system/authorino-5497d684cc-fpvql" Jul 06 15:48:11.583264 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:48:11.583243 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/3d9bb9b2-66b1-4a36-aa3d-e48975c8f61d-tls-cert\") pod \"authorino-5497d684cc-fpvql\" (UID: \"3d9bb9b2-66b1-4a36-aa3d-e48975c8f61d\") " pod="kuadrant-system/authorino-5497d684cc-fpvql" Jul 06 15:48:11.589664 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:48:11.589644 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5wdq4\" (UniqueName: \"kubernetes.io/projected/3d9bb9b2-66b1-4a36-aa3d-e48975c8f61d-kube-api-access-5wdq4\") pod \"authorino-5497d684cc-fpvql\" (UID: \"3d9bb9b2-66b1-4a36-aa3d-e48975c8f61d\") " pod="kuadrant-system/authorino-5497d684cc-fpvql" Jul 06 15:48:11.659516 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:48:11.659488 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5497d684cc-fpvql" Jul 06 15:48:11.798852 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:48:11.798822 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5497d684cc-fpvql"] Jul 06 15:48:11.801077 ip-10-0-129-154 kubenswrapper[2563]: W0706 15:48:11.801045 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3d9bb9b2_66b1_4a36_aa3d_e48975c8f61d.slice/crio-f8db9bfc43f81f50b01cacb49f1089ab2d41e53abfb223617b824c63e019a8e2 WatchSource:0}: Error finding container f8db9bfc43f81f50b01cacb49f1089ab2d41e53abfb223617b824c63e019a8e2: Status 404 returned error can't find the container with id f8db9bfc43f81f50b01cacb49f1089ab2d41e53abfb223617b824c63e019a8e2 Jul 06 15:48:12.320247 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:48:12.320220 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5497d684cc-fpvql" event={"ID":"3d9bb9b2-66b1-4a36-aa3d-e48975c8f61d","Type":"ContainerStarted","Data":"f8db9bfc43f81f50b01cacb49f1089ab2d41e53abfb223617b824c63e019a8e2"} Jul 06 15:48:13.329077 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:48:13.329042 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5497d684cc-fpvql" event={"ID":"3d9bb9b2-66b1-4a36-aa3d-e48975c8f61d","Type":"ContainerStarted","Data":"8bfa1fd9d841c7109c2377f443f61c7415b46121184eb7b0c1d5af63278a9a43"} Jul 06 15:48:13.366435 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:48:13.366350 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-5497d684cc-fpvql" podStartSLOduration=1.883513336 podStartE2EDuration="2.366328115s" totalImagesPullingTime="482.814779ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 15:48:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 15:48:13.363255317 +0000 UTC m=+869.872488505" watchObservedRunningTime="2026-07-06 15:48:13.366328115 +0000 UTC m=+869.875561290" Jul 06 15:48:13.432409 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:48:13.432374 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-676969c7b4-gzp6w"] Jul 06 15:48:13.432812 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:48:13.432784 2563 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-676969c7b4-gzp6w" podUID="3d478fe7-51aa-49aa-8c96-b8ae66640974" containerName="authorino" containerID="cri-o://d73cf6911b423fa8cbe9715c04e44f6e86f4a475958e4fab04777ee80b0b13d4" gracePeriod=30 Jul 06 15:48:13.699910 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:48:13.699877 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-676969c7b4-gzp6w" Jul 06 15:48:13.799487 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:48:13.799452 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/3d478fe7-51aa-49aa-8c96-b8ae66640974-tls-cert\") pod \"3d478fe7-51aa-49aa-8c96-b8ae66640974\" (UID: \"3d478fe7-51aa-49aa-8c96-b8ae66640974\") " Jul 06 15:48:13.799663 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:48:13.799519 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rsg8t\" (UniqueName: \"kubernetes.io/projected/3d478fe7-51aa-49aa-8c96-b8ae66640974-kube-api-access-rsg8t\") pod \"3d478fe7-51aa-49aa-8c96-b8ae66640974\" (UID: \"3d478fe7-51aa-49aa-8c96-b8ae66640974\") " Jul 06 15:48:13.801743 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:48:13.801707 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3d478fe7-51aa-49aa-8c96-b8ae66640974-kube-api-access-rsg8t" (OuterVolumeSpecName: "kube-api-access-rsg8t") pod "3d478fe7-51aa-49aa-8c96-b8ae66640974" (UID: "3d478fe7-51aa-49aa-8c96-b8ae66640974"). InnerVolumeSpecName "kube-api-access-rsg8t". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 15:48:13.810531 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:48:13.810499 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3d478fe7-51aa-49aa-8c96-b8ae66640974-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "3d478fe7-51aa-49aa-8c96-b8ae66640974" (UID: "3d478fe7-51aa-49aa-8c96-b8ae66640974"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 15:48:13.901151 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:48:13.901109 2563 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/3d478fe7-51aa-49aa-8c96-b8ae66640974-tls-cert\") on node \"ip-10-0-129-154.ec2.internal\" DevicePath \"\"" Jul 06 15:48:13.901151 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:48:13.901142 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rsg8t\" (UniqueName: \"kubernetes.io/projected/3d478fe7-51aa-49aa-8c96-b8ae66640974-kube-api-access-rsg8t\") on node \"ip-10-0-129-154.ec2.internal\" DevicePath \"\"" Jul 06 15:48:14.336053 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:48:14.335966 2563 generic.go:356] "Generic (PLEG): container finished" podID="3d478fe7-51aa-49aa-8c96-b8ae66640974" containerID="d73cf6911b423fa8cbe9715c04e44f6e86f4a475958e4fab04777ee80b0b13d4" exitCode=0 Jul 06 15:48:14.336053 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:48:14.336010 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-676969c7b4-gzp6w" Jul 06 15:48:14.336564 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:48:14.336051 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-676969c7b4-gzp6w" event={"ID":"3d478fe7-51aa-49aa-8c96-b8ae66640974","Type":"ContainerDied","Data":"d73cf6911b423fa8cbe9715c04e44f6e86f4a475958e4fab04777ee80b0b13d4"} Jul 06 15:48:14.336564 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:48:14.336092 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-676969c7b4-gzp6w" event={"ID":"3d478fe7-51aa-49aa-8c96-b8ae66640974","Type":"ContainerDied","Data":"f164d322162bbe8fadb2c65bbd8cb22612fe09cc9a1327baaf22d93c1a013c4b"} Jul 06 15:48:14.336564 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:48:14.336115 2563 scope.go:117] "RemoveContainer" containerID="d73cf6911b423fa8cbe9715c04e44f6e86f4a475958e4fab04777ee80b0b13d4" Jul 06 15:48:14.346120 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:48:14.346103 2563 scope.go:117] "RemoveContainer" containerID="d73cf6911b423fa8cbe9715c04e44f6e86f4a475958e4fab04777ee80b0b13d4" Jul 06 15:48:14.346382 ip-10-0-129-154 kubenswrapper[2563]: E0706 15:48:14.346363 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d73cf6911b423fa8cbe9715c04e44f6e86f4a475958e4fab04777ee80b0b13d4\": container with ID starting with d73cf6911b423fa8cbe9715c04e44f6e86f4a475958e4fab04777ee80b0b13d4 not found: ID does not exist" containerID="d73cf6911b423fa8cbe9715c04e44f6e86f4a475958e4fab04777ee80b0b13d4" Jul 06 15:48:14.346500 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:48:14.346392 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d73cf6911b423fa8cbe9715c04e44f6e86f4a475958e4fab04777ee80b0b13d4"} err="failed to get container status \"d73cf6911b423fa8cbe9715c04e44f6e86f4a475958e4fab04777ee80b0b13d4\": rpc error: code = NotFound desc = could not find container \"d73cf6911b423fa8cbe9715c04e44f6e86f4a475958e4fab04777ee80b0b13d4\": container with ID starting with d73cf6911b423fa8cbe9715c04e44f6e86f4a475958e4fab04777ee80b0b13d4 not found: ID does not exist" Jul 06 15:48:14.354772 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:48:14.354746 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-676969c7b4-gzp6w"] Jul 06 15:48:14.359570 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:48:14.359548 2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-676969c7b4-gzp6w"] Jul 06 15:48:15.945680 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:48:15.945644 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3d478fe7-51aa-49aa-8c96-b8ae66640974" path="/var/lib/kubelet/pods/3d478fe7-51aa-49aa-8c96-b8ae66640974/volumes" Jul 06 15:48:43.896123 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:48:43.896044 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-64786d9cbb-trjfs_bf5c30e2-bf88-468b-adf2-2e8f73f136b4/cluster-monitoring-operator/0.log" Jul 06 15:48:43.897406 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:48:43.897381 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-64786d9cbb-trjfs_bf5c30e2-bf88-468b-adf2-2e8f73f136b4/cluster-monitoring-operator/0.log" Jul 06 15:48:43.899901 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:48:43.899883 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-l47bf_9ad7e968-ed2e-4cbe-9b4b-780bc9d01b88/console-operator/2.log" Jul 06 15:48:43.900914 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:48:43.900895 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-l47bf_9ad7e968-ed2e-4cbe-9b4b-780bc9d01b88/console-operator/2.log" Jul 06 15:53:43.940775 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:53:43.940745 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-64786d9cbb-trjfs_bf5c30e2-bf88-468b-adf2-2e8f73f136b4/cluster-monitoring-operator/0.log" Jul 06 15:53:43.945198 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:53:43.945172 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-l47bf_9ad7e968-ed2e-4cbe-9b4b-780bc9d01b88/console-operator/2.log" Jul 06 15:53:43.946234 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:53:43.946212 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-64786d9cbb-trjfs_bf5c30e2-bf88-468b-adf2-2e8f73f136b4/cluster-monitoring-operator/0.log" Jul 06 15:53:43.949585 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:53:43.949566 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-l47bf_9ad7e968-ed2e-4cbe-9b4b-780bc9d01b88/console-operator/2.log" Jul 06 15:58:43.978212 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:58:43.978182 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-64786d9cbb-trjfs_bf5c30e2-bf88-468b-adf2-2e8f73f136b4/cluster-monitoring-operator/0.log" Jul 06 15:58:43.981296 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:58:43.981268 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-l47bf_9ad7e968-ed2e-4cbe-9b4b-780bc9d01b88/console-operator/2.log" Jul 06 15:58:43.983078 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:58:43.983055 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-64786d9cbb-trjfs_bf5c30e2-bf88-468b-adf2-2e8f73f136b4/cluster-monitoring-operator/0.log" Jul 06 15:58:43.986370 ip-10-0-129-154 kubenswrapper[2563]: I0706 15:58:43.986348 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-l47bf_9ad7e968-ed2e-4cbe-9b4b-780bc9d01b88/console-operator/2.log" Jul 06 16:00:00.152916 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:00:00.152871 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-key-cleanup-29722560-85hs7"] Jul 06 16:00:00.153767 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:00:00.153747 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3d478fe7-51aa-49aa-8c96-b8ae66640974" containerName="authorino" Jul 06 16:00:00.153887 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:00:00.153770 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="3d478fe7-51aa-49aa-8c96-b8ae66640974" containerName="authorino" Jul 06 16:00:00.154004 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:00:00.153989 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="3d478fe7-51aa-49aa-8c96-b8ae66640974" containerName="authorino" Jul 06 16:00:00.158167 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:00:00.158135 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29722560-85hs7" Jul 06 16:00:00.161596 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:00:00.161574 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-hz5zw\"" Jul 06 16:00:00.176195 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:00:00.176166 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-29722560-85hs7"] Jul 06 16:00:00.251240 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:00:00.251203 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4wpch\" (UniqueName: \"kubernetes.io/projected/abae7424-7574-4b51-9b0f-4020b0370e15-kube-api-access-4wpch\") pod \"maas-api-key-cleanup-29722560-85hs7\" (UID: \"abae7424-7574-4b51-9b0f-4020b0370e15\") " pod="opendatahub/maas-api-key-cleanup-29722560-85hs7" Jul 06 16:00:00.351751 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:00:00.351709 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4wpch\" (UniqueName: \"kubernetes.io/projected/abae7424-7574-4b51-9b0f-4020b0370e15-kube-api-access-4wpch\") pod \"maas-api-key-cleanup-29722560-85hs7\" (UID: \"abae7424-7574-4b51-9b0f-4020b0370e15\") " pod="opendatahub/maas-api-key-cleanup-29722560-85hs7" Jul 06 16:00:00.361618 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:00:00.361591 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4wpch\" (UniqueName: \"kubernetes.io/projected/abae7424-7574-4b51-9b0f-4020b0370e15-kube-api-access-4wpch\") pod \"maas-api-key-cleanup-29722560-85hs7\" (UID: \"abae7424-7574-4b51-9b0f-4020b0370e15\") " pod="opendatahub/maas-api-key-cleanup-29722560-85hs7" Jul 06 16:00:00.468698 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:00:00.468616 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29722560-85hs7" Jul 06 16:00:00.609870 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:00:00.609847 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-29722560-85hs7"] Jul 06 16:00:00.611786 ip-10-0-129-154 kubenswrapper[2563]: W0706 16:00:00.611760 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podabae7424_7574_4b51_9b0f_4020b0370e15.slice/crio-22eb04f9ebff7338675c579ea2076685f81b99d830a9dfb39644179ed921eb08 WatchSource:0}: Error finding container 22eb04f9ebff7338675c579ea2076685f81b99d830a9dfb39644179ed921eb08: Status 404 returned error can't find the container with id 22eb04f9ebff7338675c579ea2076685f81b99d830a9dfb39644179ed921eb08 Jul 06 16:00:00.613858 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:00:00.613843 2563 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 06 16:00:01.168496 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:00:01.168458 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29722560-85hs7" event={"ID":"abae7424-7574-4b51-9b0f-4020b0370e15","Type":"ContainerStarted","Data":"22eb04f9ebff7338675c579ea2076685f81b99d830a9dfb39644179ed921eb08"} Jul 06 16:00:37.363928 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:00:37.363837 2563 generic.go:356] "Generic (PLEG): container finished" podID="abae7424-7574-4b51-9b0f-4020b0370e15" containerID="bd7e27c772faf5a9a291b05913c7059216a95869e71e228463339ec3ae49bad8" exitCode=0 Jul 06 16:00:37.364323 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:00:37.363922 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29722560-85hs7" event={"ID":"abae7424-7574-4b51-9b0f-4020b0370e15","Type":"ContainerDied","Data":"bd7e27c772faf5a9a291b05913c7059216a95869e71e228463339ec3ae49bad8"} Jul 06 16:00:38.520716 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:00:38.520684 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29722560-85hs7" Jul 06 16:00:38.601855 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:00:38.601814 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4wpch\" (UniqueName: \"kubernetes.io/projected/abae7424-7574-4b51-9b0f-4020b0370e15-kube-api-access-4wpch\") pod \"abae7424-7574-4b51-9b0f-4020b0370e15\" (UID: \"abae7424-7574-4b51-9b0f-4020b0370e15\") " Jul 06 16:00:38.604173 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:00:38.604141 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/abae7424-7574-4b51-9b0f-4020b0370e15-kube-api-access-4wpch" (OuterVolumeSpecName: "kube-api-access-4wpch") pod "abae7424-7574-4b51-9b0f-4020b0370e15" (UID: "abae7424-7574-4b51-9b0f-4020b0370e15"). InnerVolumeSpecName "kube-api-access-4wpch". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 16:00:38.702667 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:00:38.702629 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4wpch\" (UniqueName: \"kubernetes.io/projected/abae7424-7574-4b51-9b0f-4020b0370e15-kube-api-access-4wpch\") on node \"ip-10-0-129-154.ec2.internal\" DevicePath \"\"" Jul 06 16:00:39.376685 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:00:39.376654 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29722560-85hs7" Jul 06 16:00:39.376870 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:00:39.376698 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29722560-85hs7" event={"ID":"abae7424-7574-4b51-9b0f-4020b0370e15","Type":"ContainerDied","Data":"22eb04f9ebff7338675c579ea2076685f81b99d830a9dfb39644179ed921eb08"} Jul 06 16:00:39.376870 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:00:39.376752 2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="22eb04f9ebff7338675c579ea2076685f81b99d830a9dfb39644179ed921eb08" Jul 06 16:03:44.014804 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:03:44.014775 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-64786d9cbb-trjfs_bf5c30e2-bf88-468b-adf2-2e8f73f136b4/cluster-monitoring-operator/0.log" Jul 06 16:03:44.018057 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:03:44.018031 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-l47bf_9ad7e968-ed2e-4cbe-9b4b-780bc9d01b88/console-operator/2.log" Jul 06 16:03:44.026715 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:03:44.026693 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-64786d9cbb-trjfs_bf5c30e2-bf88-468b-adf2-2e8f73f136b4/cluster-monitoring-operator/0.log" Jul 06 16:03:44.029602 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:03:44.029587 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-l47bf_9ad7e968-ed2e-4cbe-9b4b-780bc9d01b88/console-operator/2.log" Jul 06 16:05:33.990461 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:05:33.990399 2563 generic.go:356] "Generic (PLEG): container finished" podID="c2d0ef2f-f717-4854-8a6b-1f09b3e0868e" containerID="404c330fe43821d10501f55b1389c9755b8d43a30446169987028c78aea15cf6" exitCode=1 Jul 06 16:05:33.990874 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:05:33.990474 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-544dccfdb4-rl2tq" event={"ID":"c2d0ef2f-f717-4854-8a6b-1f09b3e0868e","Type":"ContainerDied","Data":"404c330fe43821d10501f55b1389c9755b8d43a30446169987028c78aea15cf6"} Jul 06 16:05:33.990874 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:05:33.990812 2563 scope.go:117] "RemoveContainer" containerID="404c330fe43821d10501f55b1389c9755b8d43a30446169987028c78aea15cf6" Jul 06 16:05:34.997672 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:05:34.997635 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-544dccfdb4-rl2tq" event={"ID":"c2d0ef2f-f717-4854-8a6b-1f09b3e0868e","Type":"ContainerStarted","Data":"8a7ac579b108444713393be316f96cad45bbf8d13e5506b7120990ccd8969ffd"} Jul 06 16:05:34.998046 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:05:34.997836 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-544dccfdb4-rl2tq" Jul 06 16:05:46.005640 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:05:46.005606 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-544dccfdb4-rl2tq" Jul 06 16:07:38.055237 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:07:38.055200 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-785a99-openshift-default-6f845b87c7-rxkzs"] Jul 06 16:07:38.055752 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:07:38.055736 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="abae7424-7574-4b51-9b0f-4020b0370e15" containerName="cleanup" Jul 06 16:07:38.055807 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:07:38.055755 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="abae7424-7574-4b51-9b0f-4020b0370e15" containerName="cleanup" Jul 06 16:07:38.055848 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:07:38.055829 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="abae7424-7574-4b51-9b0f-4020b0370e15" containerName="cleanup" Jul 06 16:07:38.061096 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:07:38.061069 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-785a99-openshift-default-6f845b87c7-rxkzs" Jul 06 16:07:38.065262 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:07:38.065234 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-785a99-openshift-default-dockercfg-2mkz6\"" Jul 06 16:07:38.065379 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:07:38.065287 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 06 16:07:38.067272 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:07:38.067250 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3aefe2ab-e0be-43fb-802d-2544ee3d0876-istiod-ca-cert\") pod \"e2e-inf-b-785a99-openshift-default-6f845b87c7-rxkzs\" (UID: \"3aefe2ab-e0be-43fb-802d-2544ee3d0876\") " pod="openshift-ingress/e2e-inf-b-785a99-openshift-default-6f845b87c7-rxkzs" Jul 06 16:07:38.067489 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:07:38.067287 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3aefe2ab-e0be-43fb-802d-2544ee3d0876-istio-envoy\") pod \"e2e-inf-b-785a99-openshift-default-6f845b87c7-rxkzs\" (UID: \"3aefe2ab-e0be-43fb-802d-2544ee3d0876\") " pod="openshift-ingress/e2e-inf-b-785a99-openshift-default-6f845b87c7-rxkzs" Jul 06 16:07:38.067489 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:07:38.067312 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3aefe2ab-e0be-43fb-802d-2544ee3d0876-workload-socket\") pod \"e2e-inf-b-785a99-openshift-default-6f845b87c7-rxkzs\" (UID: \"3aefe2ab-e0be-43fb-802d-2544ee3d0876\") " pod="openshift-ingress/e2e-inf-b-785a99-openshift-default-6f845b87c7-rxkzs" Jul 06 16:07:38.067489 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:07:38.067366 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3aefe2ab-e0be-43fb-802d-2544ee3d0876-istio-token\") pod \"e2e-inf-b-785a99-openshift-default-6f845b87c7-rxkzs\" (UID: \"3aefe2ab-e0be-43fb-802d-2544ee3d0876\") " pod="openshift-ingress/e2e-inf-b-785a99-openshift-default-6f845b87c7-rxkzs" Jul 06 16:07:38.067489 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:07:38.067411 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3aefe2ab-e0be-43fb-802d-2544ee3d0876-credential-socket\") pod \"e2e-inf-b-785a99-openshift-default-6f845b87c7-rxkzs\" (UID: \"3aefe2ab-e0be-43fb-802d-2544ee3d0876\") " pod="openshift-ingress/e2e-inf-b-785a99-openshift-default-6f845b87c7-rxkzs" Jul 06 16:07:38.067489 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:07:38.067450 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3aefe2ab-e0be-43fb-802d-2544ee3d0876-workload-certs\") pod \"e2e-inf-b-785a99-openshift-default-6f845b87c7-rxkzs\" (UID: \"3aefe2ab-e0be-43fb-802d-2544ee3d0876\") " pod="openshift-ingress/e2e-inf-b-785a99-openshift-default-6f845b87c7-rxkzs" Jul 06 16:07:38.067489 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:07:38.067467 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3aefe2ab-e0be-43fb-802d-2544ee3d0876-istio-data\") pod \"e2e-inf-b-785a99-openshift-default-6f845b87c7-rxkzs\" (UID: \"3aefe2ab-e0be-43fb-802d-2544ee3d0876\") " pod="openshift-ingress/e2e-inf-b-785a99-openshift-default-6f845b87c7-rxkzs" Jul 06 16:07:38.067830 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:07:38.067537 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3aefe2ab-e0be-43fb-802d-2544ee3d0876-istio-podinfo\") pod \"e2e-inf-b-785a99-openshift-default-6f845b87c7-rxkzs\" (UID: \"3aefe2ab-e0be-43fb-802d-2544ee3d0876\") " pod="openshift-ingress/e2e-inf-b-785a99-openshift-default-6f845b87c7-rxkzs" Jul 06 16:07:38.067830 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:07:38.067564 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vmbh4\" (UniqueName: \"kubernetes.io/projected/3aefe2ab-e0be-43fb-802d-2544ee3d0876-kube-api-access-vmbh4\") pod \"e2e-inf-b-785a99-openshift-default-6f845b87c7-rxkzs\" (UID: \"3aefe2ab-e0be-43fb-802d-2544ee3d0876\") " pod="openshift-ingress/e2e-inf-b-785a99-openshift-default-6f845b87c7-rxkzs" Jul 06 16:07:38.075508 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:07:38.075485 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-785a99-openshift-default-6f845b87c7-rxkzs"] Jul 06 16:07:38.168602 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:07:38.168571 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3aefe2ab-e0be-43fb-802d-2544ee3d0876-istiod-ca-cert\") pod \"e2e-inf-b-785a99-openshift-default-6f845b87c7-rxkzs\" (UID: \"3aefe2ab-e0be-43fb-802d-2544ee3d0876\") " pod="openshift-ingress/e2e-inf-b-785a99-openshift-default-6f845b87c7-rxkzs" Jul 06 16:07:38.168783 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:07:38.168614 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3aefe2ab-e0be-43fb-802d-2544ee3d0876-istio-envoy\") pod \"e2e-inf-b-785a99-openshift-default-6f845b87c7-rxkzs\" (UID: \"3aefe2ab-e0be-43fb-802d-2544ee3d0876\") " pod="openshift-ingress/e2e-inf-b-785a99-openshift-default-6f845b87c7-rxkzs" Jul 06 16:07:38.168783 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:07:38.168642 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3aefe2ab-e0be-43fb-802d-2544ee3d0876-workload-socket\") pod \"e2e-inf-b-785a99-openshift-default-6f845b87c7-rxkzs\" (UID: \"3aefe2ab-e0be-43fb-802d-2544ee3d0876\") " pod="openshift-ingress/e2e-inf-b-785a99-openshift-default-6f845b87c7-rxkzs" Jul 06 16:07:38.168783 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:07:38.168663 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3aefe2ab-e0be-43fb-802d-2544ee3d0876-istio-token\") pod \"e2e-inf-b-785a99-openshift-default-6f845b87c7-rxkzs\" (UID: \"3aefe2ab-e0be-43fb-802d-2544ee3d0876\") " pod="openshift-ingress/e2e-inf-b-785a99-openshift-default-6f845b87c7-rxkzs" Jul 06 16:07:38.168783 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:07:38.168685 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3aefe2ab-e0be-43fb-802d-2544ee3d0876-credential-socket\") pod \"e2e-inf-b-785a99-openshift-default-6f845b87c7-rxkzs\" (UID: \"3aefe2ab-e0be-43fb-802d-2544ee3d0876\") " pod="openshift-ingress/e2e-inf-b-785a99-openshift-default-6f845b87c7-rxkzs" Jul 06 16:07:38.168783 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:07:38.168708 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3aefe2ab-e0be-43fb-802d-2544ee3d0876-workload-certs\") pod \"e2e-inf-b-785a99-openshift-default-6f845b87c7-rxkzs\" (UID: \"3aefe2ab-e0be-43fb-802d-2544ee3d0876\") " pod="openshift-ingress/e2e-inf-b-785a99-openshift-default-6f845b87c7-rxkzs" Jul 06 16:07:38.168783 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:07:38.168723 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3aefe2ab-e0be-43fb-802d-2544ee3d0876-istio-data\") pod \"e2e-inf-b-785a99-openshift-default-6f845b87c7-rxkzs\" (UID: \"3aefe2ab-e0be-43fb-802d-2544ee3d0876\") " pod="openshift-ingress/e2e-inf-b-785a99-openshift-default-6f845b87c7-rxkzs" Jul 06 16:07:38.168783 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:07:38.168783 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3aefe2ab-e0be-43fb-802d-2544ee3d0876-istio-podinfo\") pod \"e2e-inf-b-785a99-openshift-default-6f845b87c7-rxkzs\" (UID: \"3aefe2ab-e0be-43fb-802d-2544ee3d0876\") " pod="openshift-ingress/e2e-inf-b-785a99-openshift-default-6f845b87c7-rxkzs" Jul 06 16:07:38.169162 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:07:38.168814 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vmbh4\" (UniqueName: \"kubernetes.io/projected/3aefe2ab-e0be-43fb-802d-2544ee3d0876-kube-api-access-vmbh4\") pod \"e2e-inf-b-785a99-openshift-default-6f845b87c7-rxkzs\" (UID: \"3aefe2ab-e0be-43fb-802d-2544ee3d0876\") " pod="openshift-ingress/e2e-inf-b-785a99-openshift-default-6f845b87c7-rxkzs" Jul 06 16:07:38.169218 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:07:38.169168 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3aefe2ab-e0be-43fb-802d-2544ee3d0876-istio-data\") pod \"e2e-inf-b-785a99-openshift-default-6f845b87c7-rxkzs\" (UID: \"3aefe2ab-e0be-43fb-802d-2544ee3d0876\") " pod="openshift-ingress/e2e-inf-b-785a99-openshift-default-6f845b87c7-rxkzs" Jul 06 16:07:38.169271 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:07:38.169245 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3aefe2ab-e0be-43fb-802d-2544ee3d0876-credential-socket\") pod \"e2e-inf-b-785a99-openshift-default-6f845b87c7-rxkzs\" (UID: \"3aefe2ab-e0be-43fb-802d-2544ee3d0876\") " pod="openshift-ingress/e2e-inf-b-785a99-openshift-default-6f845b87c7-rxkzs" Jul 06 16:07:38.169353 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:07:38.169326 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3aefe2ab-e0be-43fb-802d-2544ee3d0876-workload-socket\") pod \"e2e-inf-b-785a99-openshift-default-6f845b87c7-rxkzs\" (UID: \"3aefe2ab-e0be-43fb-802d-2544ee3d0876\") " pod="openshift-ingress/e2e-inf-b-785a99-openshift-default-6f845b87c7-rxkzs" Jul 06 16:07:38.169353 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:07:38.169342 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3aefe2ab-e0be-43fb-802d-2544ee3d0876-istiod-ca-cert\") pod \"e2e-inf-b-785a99-openshift-default-6f845b87c7-rxkzs\" (UID: \"3aefe2ab-e0be-43fb-802d-2544ee3d0876\") " pod="openshift-ingress/e2e-inf-b-785a99-openshift-default-6f845b87c7-rxkzs" Jul 06 16:07:38.169353 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:07:38.169347 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3aefe2ab-e0be-43fb-802d-2544ee3d0876-workload-certs\") pod \"e2e-inf-b-785a99-openshift-default-6f845b87c7-rxkzs\" (UID: \"3aefe2ab-e0be-43fb-802d-2544ee3d0876\") " pod="openshift-ingress/e2e-inf-b-785a99-openshift-default-6f845b87c7-rxkzs" Jul 06 16:07:38.171243 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:07:38.171219 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3aefe2ab-e0be-43fb-802d-2544ee3d0876-istio-envoy\") pod \"e2e-inf-b-785a99-openshift-default-6f845b87c7-rxkzs\" (UID: \"3aefe2ab-e0be-43fb-802d-2544ee3d0876\") " pod="openshift-ingress/e2e-inf-b-785a99-openshift-default-6f845b87c7-rxkzs" Jul 06 16:07:38.171369 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:07:38.171344 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3aefe2ab-e0be-43fb-802d-2544ee3d0876-istio-podinfo\") pod \"e2e-inf-b-785a99-openshift-default-6f845b87c7-rxkzs\" (UID: \"3aefe2ab-e0be-43fb-802d-2544ee3d0876\") " pod="openshift-ingress/e2e-inf-b-785a99-openshift-default-6f845b87c7-rxkzs" Jul 06 16:07:38.177601 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:07:38.177573 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3aefe2ab-e0be-43fb-802d-2544ee3d0876-istio-token\") pod \"e2e-inf-b-785a99-openshift-default-6f845b87c7-rxkzs\" (UID: \"3aefe2ab-e0be-43fb-802d-2544ee3d0876\") " pod="openshift-ingress/e2e-inf-b-785a99-openshift-default-6f845b87c7-rxkzs" Jul 06 16:07:38.177801 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:07:38.177783 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vmbh4\" (UniqueName: \"kubernetes.io/projected/3aefe2ab-e0be-43fb-802d-2544ee3d0876-kube-api-access-vmbh4\") pod \"e2e-inf-b-785a99-openshift-default-6f845b87c7-rxkzs\" (UID: \"3aefe2ab-e0be-43fb-802d-2544ee3d0876\") " pod="openshift-ingress/e2e-inf-b-785a99-openshift-default-6f845b87c7-rxkzs" Jul 06 16:07:38.375103 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:07:38.375009 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-785a99-openshift-default-6f845b87c7-rxkzs" Jul 06 16:07:38.522823 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:07:38.522800 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-785a99-openshift-default-6f845b87c7-rxkzs"] Jul 06 16:07:38.524633 ip-10-0-129-154 kubenswrapper[2563]: W0706 16:07:38.524597 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3aefe2ab_e0be_43fb_802d_2544ee3d0876.slice/crio-fffbe20164a9ee7e65e7f89bdfa6604fc6cf0f832b216b44811376be12317abb WatchSource:0}: Error finding container fffbe20164a9ee7e65e7f89bdfa6604fc6cf0f832b216b44811376be12317abb: Status 404 returned error can't find the container with id fffbe20164a9ee7e65e7f89bdfa6604fc6cf0f832b216b44811376be12317abb Jul 06 16:07:38.526620 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:07:38.526601 2563 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 06 16:07:38.665238 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:07:38.665197 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-785a99-openshift-default-6f845b87c7-rxkzs" event={"ID":"3aefe2ab-e0be-43fb-802d-2544ee3d0876","Type":"ContainerStarted","Data":"fffbe20164a9ee7e65e7f89bdfa6604fc6cf0f832b216b44811376be12317abb"} Jul 06 16:07:42.417193 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:07:42.417157 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 06 16:07:42.417485 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:07:42.417236 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 06 16:07:42.417485 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:07:42.417267 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 06 16:07:42.690653 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:07:42.690563 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-785a99-openshift-default-6f845b87c7-rxkzs" event={"ID":"3aefe2ab-e0be-43fb-802d-2544ee3d0876","Type":"ContainerStarted","Data":"d24c928ace2dd7724c2765af046cfb0aab116de9e9d52a96cc69f93242fb7bd1"} Jul 06 16:07:42.714774 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:07:42.714727 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-785a99-openshift-default-6f845b87c7-rxkzs" podStartSLOduration=0.824568206 podStartE2EDuration="4.714711062s" totalImagesPullingTime="3.890142856s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 16:07:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 16:07:42.713925437 +0000 UTC m=+2039.223158611" watchObservedRunningTime="2026-07-06 16:07:42.714711062 +0000 UTC m=+2039.223944236" Jul 06 16:07:43.375732 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:07:43.375701 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-785a99-openshift-default-6f845b87c7-rxkzs" Jul 06 16:07:43.380685 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:07:43.380656 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-785a99-openshift-default-6f845b87c7-rxkzs" Jul 06 16:07:43.697825 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:07:43.697787 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-785a99-openshift-default-6f845b87c7-rxkzs" Jul 06 16:07:43.703121 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:07:43.703093 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-785a99-openshift-default-6f845b87c7-rxkzs" Jul 06 16:07:56.769501 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:07:56.769467 2563 generic.go:356] "Generic (PLEG): container finished" podID="3aefe2ab-e0be-43fb-802d-2544ee3d0876" containerID="d24c928ace2dd7724c2765af046cfb0aab116de9e9d52a96cc69f93242fb7bd1" exitCode=0 Jul 06 16:07:56.770021 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:07:56.769542 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-785a99-openshift-default-6f845b87c7-rxkzs" event={"ID":"3aefe2ab-e0be-43fb-802d-2544ee3d0876","Type":"ContainerDied","Data":"d24c928ace2dd7724c2765af046cfb0aab116de9e9d52a96cc69f93242fb7bd1"} Jul 06 16:07:56.770094 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:07:56.770025 2563 scope.go:117] "RemoveContainer" containerID="d24c928ace2dd7724c2765af046cfb0aab116de9e9d52a96cc69f93242fb7bd1" Jul 06 16:07:56.771081 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:07:56.771050 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 06 16:07:56.771179 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:07:56.771134 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 06 16:07:56.771241 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:07:56.771184 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 06 16:07:57.780977 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:07:57.780934 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-785a99-openshift-default-6f845b87c7-rxkzs" event={"ID":"3aefe2ab-e0be-43fb-802d-2544ee3d0876","Type":"ContainerStarted","Data":"bd18c938ba19cfc44a7e8a093f0fc69ed043531fc473dd38a493d384d54f4698"} Jul 06 16:07:58.377628 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:07:58.377578 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-785a99-openshift-default-6f845b87c7-rxkzs" Jul 06 16:07:58.377845 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:07:58.377721 2563 patch_prober.go:28] interesting pod/e2e-inf-b-785a99-openshift-default-6f845b87c7-rxkzs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" start-of-body= Jul 06 16:07:58.377845 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:07:58.377768 2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-785a99-openshift-default-6f845b87c7-rxkzs" podUID="3aefe2ab-e0be-43fb-802d-2544ee3d0876" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" Jul 06 16:07:58.697789 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:07:58.697751 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-785a99-openshift-default-6f845b87c7-rxkzs" Jul 06 16:07:59.375775 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:07:59.375738 2563 patch_prober.go:28] interesting pod/e2e-inf-b-785a99-openshift-default-6f845b87c7-rxkzs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" start-of-body= Jul 06 16:07:59.376347 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:07:59.376313 2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-785a99-openshift-default-6f845b87c7-rxkzs" podUID="3aefe2ab-e0be-43fb-802d-2544ee3d0876" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" Jul 06 16:08:00.376146 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:08:00.376103 2563 patch_prober.go:28] interesting pod/e2e-inf-b-785a99-openshift-default-6f845b87c7-rxkzs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" start-of-body= Jul 06 16:08:00.376775 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:08:00.376740 2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-785a99-openshift-default-6f845b87c7-rxkzs" podUID="3aefe2ab-e0be-43fb-802d-2544ee3d0876" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" Jul 06 16:08:01.375787 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:08:01.375745 2563 patch_prober.go:28] interesting pod/e2e-inf-b-785a99-openshift-default-6f845b87c7-rxkzs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" start-of-body= Jul 06 16:08:01.376104 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:08:01.376075 2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-785a99-openshift-default-6f845b87c7-rxkzs" podUID="3aefe2ab-e0be-43fb-802d-2544ee3d0876" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" Jul 06 16:08:02.380169 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:08:02.380138 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-785a99-openshift-default-6f845b87c7-rxkzs" Jul 06 16:08:02.381275 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:08:02.381250 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-785a99-openshift-default-6f845b87c7-rxkzs" Jul 06 16:08:44.052661 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:08:44.052628 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-64786d9cbb-trjfs_bf5c30e2-bf88-468b-adf2-2e8f73f136b4/cluster-monitoring-operator/0.log" Jul 06 16:08:44.055657 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:08:44.055632 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-l47bf_9ad7e968-ed2e-4cbe-9b4b-780bc9d01b88/console-operator/2.log" Jul 06 16:08:44.064045 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:08:44.064029 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-64786d9cbb-trjfs_bf5c30e2-bf88-468b-adf2-2e8f73f136b4/cluster-monitoring-operator/0.log" Jul 06 16:08:44.066916 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:08:44.066898 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-l47bf_9ad7e968-ed2e-4cbe-9b4b-780bc9d01b88/console-operator/2.log" Jul 06 16:09:30.123749 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:09:30.123706 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-785a99-openshift-default-6f845b87c7-rxkzs"] Jul 06 16:09:30.124348 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:09:30.123981 2563 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-785a99-openshift-default-6f845b87c7-rxkzs" podUID="3aefe2ab-e0be-43fb-802d-2544ee3d0876" containerName="istio-proxy" containerID="cri-o://bd18c938ba19cfc44a7e8a093f0fc69ed043531fc473dd38a493d384d54f4698" gracePeriod=30 Jul 06 16:09:32.382679 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:09:32.382634 2563 patch_prober.go:28] interesting pod/e2e-inf-b-785a99-openshift-default-6f845b87c7-rxkzs container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 06 16:09:32.383085 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:09:32.382712 2563 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-b-785a99-openshift-default-6f845b87c7-rxkzs" podUID="3aefe2ab-e0be-43fb-802d-2544ee3d0876" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 06 16:09:35.322817 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:09:35.322782 2563 generic.go:356] "Generic (PLEG): container finished" podID="3aefe2ab-e0be-43fb-802d-2544ee3d0876" containerID="bd18c938ba19cfc44a7e8a093f0fc69ed043531fc473dd38a493d384d54f4698" exitCode=0 Jul 06 16:09:35.323272 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:09:35.322859 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-785a99-openshift-default-6f845b87c7-rxkzs" event={"ID":"3aefe2ab-e0be-43fb-802d-2544ee3d0876","Type":"ContainerDied","Data":"bd18c938ba19cfc44a7e8a093f0fc69ed043531fc473dd38a493d384d54f4698"} Jul 06 16:09:35.323272 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:09:35.322929 2563 scope.go:117] "RemoveContainer" containerID="d24c928ace2dd7724c2765af046cfb0aab116de9e9d52a96cc69f93242fb7bd1" Jul 06 16:09:35.383957 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:09:35.383887 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-785a99-openshift-default-6f845b87c7-rxkzs" Jul 06 16:09:35.462166 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:09:35.462134 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3aefe2ab-e0be-43fb-802d-2544ee3d0876-istio-envoy\") pod \"3aefe2ab-e0be-43fb-802d-2544ee3d0876\" (UID: \"3aefe2ab-e0be-43fb-802d-2544ee3d0876\") " Jul 06 16:09:35.462353 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:09:35.462195 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3aefe2ab-e0be-43fb-802d-2544ee3d0876-istiod-ca-cert\") pod \"3aefe2ab-e0be-43fb-802d-2544ee3d0876\" (UID: \"3aefe2ab-e0be-43fb-802d-2544ee3d0876\") " Jul 06 16:09:35.462353 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:09:35.462217 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3aefe2ab-e0be-43fb-802d-2544ee3d0876-istio-token\") pod \"3aefe2ab-e0be-43fb-802d-2544ee3d0876\" (UID: \"3aefe2ab-e0be-43fb-802d-2544ee3d0876\") " Jul 06 16:09:35.462353 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:09:35.462247 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3aefe2ab-e0be-43fb-802d-2544ee3d0876-workload-certs\") pod \"3aefe2ab-e0be-43fb-802d-2544ee3d0876\" (UID: \"3aefe2ab-e0be-43fb-802d-2544ee3d0876\") " Jul 06 16:09:35.462353 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:09:35.462303 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3aefe2ab-e0be-43fb-802d-2544ee3d0876-istio-podinfo\") pod \"3aefe2ab-e0be-43fb-802d-2544ee3d0876\" (UID: \"3aefe2ab-e0be-43fb-802d-2544ee3d0876\") " Jul 06 16:09:35.462353 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:09:35.462320 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3aefe2ab-e0be-43fb-802d-2544ee3d0876-credential-socket\") pod \"3aefe2ab-e0be-43fb-802d-2544ee3d0876\" (UID: \"3aefe2ab-e0be-43fb-802d-2544ee3d0876\") " Jul 06 16:09:35.462353 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:09:35.462343 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3aefe2ab-e0be-43fb-802d-2544ee3d0876-istio-data\") pod \"3aefe2ab-e0be-43fb-802d-2544ee3d0876\" (UID: \"3aefe2ab-e0be-43fb-802d-2544ee3d0876\") " Jul 06 16:09:35.462713 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:09:35.462367 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vmbh4\" (UniqueName: \"kubernetes.io/projected/3aefe2ab-e0be-43fb-802d-2544ee3d0876-kube-api-access-vmbh4\") pod \"3aefe2ab-e0be-43fb-802d-2544ee3d0876\" (UID: \"3aefe2ab-e0be-43fb-802d-2544ee3d0876\") " Jul 06 16:09:35.462713 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:09:35.462449 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3aefe2ab-e0be-43fb-802d-2544ee3d0876-workload-socket\") pod \"3aefe2ab-e0be-43fb-802d-2544ee3d0876\" (UID: \"3aefe2ab-e0be-43fb-802d-2544ee3d0876\") " Jul 06 16:09:35.462713 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:09:35.462660 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3aefe2ab-e0be-43fb-802d-2544ee3d0876-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "3aefe2ab-e0be-43fb-802d-2544ee3d0876" (UID: "3aefe2ab-e0be-43fb-802d-2544ee3d0876"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 16:09:35.462881 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:09:35.462725 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3aefe2ab-e0be-43fb-802d-2544ee3d0876-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "3aefe2ab-e0be-43fb-802d-2544ee3d0876" (UID: "3aefe2ab-e0be-43fb-802d-2544ee3d0876"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 16:09:35.462881 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:09:35.462832 2563 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3aefe2ab-e0be-43fb-802d-2544ee3d0876-workload-certs\") on node \"ip-10-0-129-154.ec2.internal\" DevicePath \"\"" Jul 06 16:09:35.462881 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:09:35.462867 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3aefe2ab-e0be-43fb-802d-2544ee3d0876-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "3aefe2ab-e0be-43fb-802d-2544ee3d0876" (UID: "3aefe2ab-e0be-43fb-802d-2544ee3d0876"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 16:09:35.463049 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:09:35.462952 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3aefe2ab-e0be-43fb-802d-2544ee3d0876-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "3aefe2ab-e0be-43fb-802d-2544ee3d0876" (UID: "3aefe2ab-e0be-43fb-802d-2544ee3d0876"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 16:09:35.463720 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:09:35.463688 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3aefe2ab-e0be-43fb-802d-2544ee3d0876-istio-data" (OuterVolumeSpecName: "istio-data") pod "3aefe2ab-e0be-43fb-802d-2544ee3d0876" (UID: "3aefe2ab-e0be-43fb-802d-2544ee3d0876"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 16:09:35.464748 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:09:35.464722 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3aefe2ab-e0be-43fb-802d-2544ee3d0876-istio-token" (OuterVolumeSpecName: "istio-token") pod "3aefe2ab-e0be-43fb-802d-2544ee3d0876" (UID: "3aefe2ab-e0be-43fb-802d-2544ee3d0876"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 16:09:35.464848 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:09:35.464805 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/3aefe2ab-e0be-43fb-802d-2544ee3d0876-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "3aefe2ab-e0be-43fb-802d-2544ee3d0876" (UID: "3aefe2ab-e0be-43fb-802d-2544ee3d0876"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 06 16:09:35.464848 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:09:35.464836 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3aefe2ab-e0be-43fb-802d-2544ee3d0876-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "3aefe2ab-e0be-43fb-802d-2544ee3d0876" (UID: "3aefe2ab-e0be-43fb-802d-2544ee3d0876"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 16:09:35.465019 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:09:35.465002 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3aefe2ab-e0be-43fb-802d-2544ee3d0876-kube-api-access-vmbh4" (OuterVolumeSpecName: "kube-api-access-vmbh4") pod "3aefe2ab-e0be-43fb-802d-2544ee3d0876" (UID: "3aefe2ab-e0be-43fb-802d-2544ee3d0876"). InnerVolumeSpecName "kube-api-access-vmbh4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 16:09:35.563860 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:09:35.563823 2563 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3aefe2ab-e0be-43fb-802d-2544ee3d0876-istio-token\") on node \"ip-10-0-129-154.ec2.internal\" DevicePath \"\"" Jul 06 16:09:35.563860 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:09:35.563856 2563 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3aefe2ab-e0be-43fb-802d-2544ee3d0876-istio-podinfo\") on node \"ip-10-0-129-154.ec2.internal\" DevicePath \"\"" Jul 06 16:09:35.563860 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:09:35.563865 2563 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3aefe2ab-e0be-43fb-802d-2544ee3d0876-credential-socket\") on node \"ip-10-0-129-154.ec2.internal\" DevicePath \"\"" Jul 06 16:09:35.564143 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:09:35.563876 2563 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3aefe2ab-e0be-43fb-802d-2544ee3d0876-istio-data\") on node \"ip-10-0-129-154.ec2.internal\" DevicePath \"\"" Jul 06 16:09:35.564143 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:09:35.563886 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vmbh4\" (UniqueName: \"kubernetes.io/projected/3aefe2ab-e0be-43fb-802d-2544ee3d0876-kube-api-access-vmbh4\") on node \"ip-10-0-129-154.ec2.internal\" DevicePath \"\"" Jul 06 16:09:35.564143 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:09:35.563894 2563 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3aefe2ab-e0be-43fb-802d-2544ee3d0876-workload-socket\") on node \"ip-10-0-129-154.ec2.internal\" DevicePath \"\"" Jul 06 16:09:35.564143 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:09:35.563902 2563 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3aefe2ab-e0be-43fb-802d-2544ee3d0876-istio-envoy\") on node \"ip-10-0-129-154.ec2.internal\" DevicePath \"\"" Jul 06 16:09:35.564143 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:09:35.563910 2563 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3aefe2ab-e0be-43fb-802d-2544ee3d0876-istiod-ca-cert\") on node \"ip-10-0-129-154.ec2.internal\" DevicePath \"\"" Jul 06 16:09:36.329659 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:09:36.329576 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-785a99-openshift-default-6f845b87c7-rxkzs" event={"ID":"3aefe2ab-e0be-43fb-802d-2544ee3d0876","Type":"ContainerDied","Data":"fffbe20164a9ee7e65e7f89bdfa6604fc6cf0f832b216b44811376be12317abb"} Jul 06 16:09:36.329659 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:09:36.329618 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-785a99-openshift-default-6f845b87c7-rxkzs" Jul 06 16:09:36.329659 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:09:36.329637 2563 scope.go:117] "RemoveContainer" containerID="bd18c938ba19cfc44a7e8a093f0fc69ed043531fc473dd38a493d384d54f4698" Jul 06 16:09:36.352661 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:09:36.352631 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-785a99-openshift-default-6f845b87c7-rxkzs"] Jul 06 16:09:36.358573 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:09:36.358548 2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-785a99-openshift-default-6f845b87c7-rxkzs"] Jul 06 16:09:37.945732 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:09:37.945699 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3aefe2ab-e0be-43fb-802d-2544ee3d0876" path="/var/lib/kubelet/pods/3aefe2ab-e0be-43fb-802d-2544ee3d0876/volumes" Jul 06 16:09:39.706253 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:09:39.706222 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-05b5d9-openshift-default-7b8d7d6747-fsddd"] Jul 06 16:09:39.706764 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:09:39.706604 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3aefe2ab-e0be-43fb-802d-2544ee3d0876" containerName="istio-proxy" Jul 06 16:09:39.706764 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:09:39.706615 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="3aefe2ab-e0be-43fb-802d-2544ee3d0876" containerName="istio-proxy" Jul 06 16:09:39.706764 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:09:39.706695 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="3aefe2ab-e0be-43fb-802d-2544ee3d0876" containerName="istio-proxy" Jul 06 16:09:39.706764 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:09:39.706703 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="3aefe2ab-e0be-43fb-802d-2544ee3d0876" containerName="istio-proxy" Jul 06 16:09:39.711235 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:09:39.711216 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-05b5d9-openshift-default-7b8d7d6747-fsddd" Jul 06 16:09:39.714168 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:09:39.714147 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 06 16:09:39.714303 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:09:39.714149 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-05b5d9-openshift-default-dockercfg-cd54f\"" Jul 06 16:09:39.723951 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:09:39.723929 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-05b5d9-openshift-default-7b8d7d6747-fsddd"] Jul 06 16:09:39.800977 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:09:39.800943 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3ef26d78-e0d5-40d5-a9a4-217b5671185b-istio-data\") pod \"e2e-api-b-05b5d9-openshift-default-7b8d7d6747-fsddd\" (UID: \"3ef26d78-e0d5-40d5-a9a4-217b5671185b\") " pod="openshift-ingress/e2e-api-b-05b5d9-openshift-default-7b8d7d6747-fsddd" Jul 06 16:09:39.801145 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:09:39.800987 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3ef26d78-e0d5-40d5-a9a4-217b5671185b-istio-token\") pod \"e2e-api-b-05b5d9-openshift-default-7b8d7d6747-fsddd\" (UID: \"3ef26d78-e0d5-40d5-a9a4-217b5671185b\") " pod="openshift-ingress/e2e-api-b-05b5d9-openshift-default-7b8d7d6747-fsddd" Jul 06 16:09:39.801145 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:09:39.801010 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3ef26d78-e0d5-40d5-a9a4-217b5671185b-workload-socket\") pod \"e2e-api-b-05b5d9-openshift-default-7b8d7d6747-fsddd\" (UID: \"3ef26d78-e0d5-40d5-a9a4-217b5671185b\") " pod="openshift-ingress/e2e-api-b-05b5d9-openshift-default-7b8d7d6747-fsddd" Jul 06 16:09:39.801145 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:09:39.801026 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3ef26d78-e0d5-40d5-a9a4-217b5671185b-istiod-ca-cert\") pod \"e2e-api-b-05b5d9-openshift-default-7b8d7d6747-fsddd\" (UID: \"3ef26d78-e0d5-40d5-a9a4-217b5671185b\") " pod="openshift-ingress/e2e-api-b-05b5d9-openshift-default-7b8d7d6747-fsddd" Jul 06 16:09:39.801145 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:09:39.801045 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3ef26d78-e0d5-40d5-a9a4-217b5671185b-workload-certs\") pod \"e2e-api-b-05b5d9-openshift-default-7b8d7d6747-fsddd\" (UID: \"3ef26d78-e0d5-40d5-a9a4-217b5671185b\") " pod="openshift-ingress/e2e-api-b-05b5d9-openshift-default-7b8d7d6747-fsddd" Jul 06 16:09:39.801145 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:09:39.801136 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3ef26d78-e0d5-40d5-a9a4-217b5671185b-istio-envoy\") pod \"e2e-api-b-05b5d9-openshift-default-7b8d7d6747-fsddd\" (UID: \"3ef26d78-e0d5-40d5-a9a4-217b5671185b\") " pod="openshift-ingress/e2e-api-b-05b5d9-openshift-default-7b8d7d6747-fsddd" Jul 06 16:09:39.801323 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:09:39.801164 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3ef26d78-e0d5-40d5-a9a4-217b5671185b-credential-socket\") pod \"e2e-api-b-05b5d9-openshift-default-7b8d7d6747-fsddd\" (UID: \"3ef26d78-e0d5-40d5-a9a4-217b5671185b\") " pod="openshift-ingress/e2e-api-b-05b5d9-openshift-default-7b8d7d6747-fsddd" Jul 06 16:09:39.801323 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:09:39.801234 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vqlcf\" (UniqueName: \"kubernetes.io/projected/3ef26d78-e0d5-40d5-a9a4-217b5671185b-kube-api-access-vqlcf\") pod \"e2e-api-b-05b5d9-openshift-default-7b8d7d6747-fsddd\" (UID: \"3ef26d78-e0d5-40d5-a9a4-217b5671185b\") " pod="openshift-ingress/e2e-api-b-05b5d9-openshift-default-7b8d7d6747-fsddd" Jul 06 16:09:39.801323 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:09:39.801264 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3ef26d78-e0d5-40d5-a9a4-217b5671185b-istio-podinfo\") pod \"e2e-api-b-05b5d9-openshift-default-7b8d7d6747-fsddd\" (UID: \"3ef26d78-e0d5-40d5-a9a4-217b5671185b\") " pod="openshift-ingress/e2e-api-b-05b5d9-openshift-default-7b8d7d6747-fsddd" Jul 06 16:09:39.902232 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:09:39.902196 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3ef26d78-e0d5-40d5-a9a4-217b5671185b-istio-envoy\") pod \"e2e-api-b-05b5d9-openshift-default-7b8d7d6747-fsddd\" (UID: \"3ef26d78-e0d5-40d5-a9a4-217b5671185b\") " pod="openshift-ingress/e2e-api-b-05b5d9-openshift-default-7b8d7d6747-fsddd" Jul 06 16:09:39.902232 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:09:39.902234 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3ef26d78-e0d5-40d5-a9a4-217b5671185b-credential-socket\") pod \"e2e-api-b-05b5d9-openshift-default-7b8d7d6747-fsddd\" (UID: \"3ef26d78-e0d5-40d5-a9a4-217b5671185b\") " pod="openshift-ingress/e2e-api-b-05b5d9-openshift-default-7b8d7d6747-fsddd" Jul 06 16:09:39.902533 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:09:39.902289 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vqlcf\" (UniqueName: \"kubernetes.io/projected/3ef26d78-e0d5-40d5-a9a4-217b5671185b-kube-api-access-vqlcf\") pod \"e2e-api-b-05b5d9-openshift-default-7b8d7d6747-fsddd\" (UID: \"3ef26d78-e0d5-40d5-a9a4-217b5671185b\") " pod="openshift-ingress/e2e-api-b-05b5d9-openshift-default-7b8d7d6747-fsddd" Jul 06 16:09:39.902533 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:09:39.902328 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3ef26d78-e0d5-40d5-a9a4-217b5671185b-istio-podinfo\") pod \"e2e-api-b-05b5d9-openshift-default-7b8d7d6747-fsddd\" (UID: \"3ef26d78-e0d5-40d5-a9a4-217b5671185b\") " pod="openshift-ingress/e2e-api-b-05b5d9-openshift-default-7b8d7d6747-fsddd" Jul 06 16:09:39.902533 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:09:39.902362 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3ef26d78-e0d5-40d5-a9a4-217b5671185b-istio-data\") pod \"e2e-api-b-05b5d9-openshift-default-7b8d7d6747-fsddd\" (UID: \"3ef26d78-e0d5-40d5-a9a4-217b5671185b\") " pod="openshift-ingress/e2e-api-b-05b5d9-openshift-default-7b8d7d6747-fsddd" Jul 06 16:09:39.902533 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:09:39.902405 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3ef26d78-e0d5-40d5-a9a4-217b5671185b-istio-token\") pod \"e2e-api-b-05b5d9-openshift-default-7b8d7d6747-fsddd\" (UID: \"3ef26d78-e0d5-40d5-a9a4-217b5671185b\") " pod="openshift-ingress/e2e-api-b-05b5d9-openshift-default-7b8d7d6747-fsddd" Jul 06 16:09:39.902533 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:09:39.902464 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3ef26d78-e0d5-40d5-a9a4-217b5671185b-workload-socket\") pod \"e2e-api-b-05b5d9-openshift-default-7b8d7d6747-fsddd\" (UID: \"3ef26d78-e0d5-40d5-a9a4-217b5671185b\") " pod="openshift-ingress/e2e-api-b-05b5d9-openshift-default-7b8d7d6747-fsddd" Jul 06 16:09:39.902812 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:09:39.902547 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3ef26d78-e0d5-40d5-a9a4-217b5671185b-istiod-ca-cert\") pod \"e2e-api-b-05b5d9-openshift-default-7b8d7d6747-fsddd\" (UID: \"3ef26d78-e0d5-40d5-a9a4-217b5671185b\") " pod="openshift-ingress/e2e-api-b-05b5d9-openshift-default-7b8d7d6747-fsddd" Jul 06 16:09:39.902812 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:09:39.902583 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3ef26d78-e0d5-40d5-a9a4-217b5671185b-workload-certs\") pod \"e2e-api-b-05b5d9-openshift-default-7b8d7d6747-fsddd\" (UID: \"3ef26d78-e0d5-40d5-a9a4-217b5671185b\") " pod="openshift-ingress/e2e-api-b-05b5d9-openshift-default-7b8d7d6747-fsddd" Jul 06 16:09:39.902946 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:09:39.902923 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3ef26d78-e0d5-40d5-a9a4-217b5671185b-istio-data\") pod \"e2e-api-b-05b5d9-openshift-default-7b8d7d6747-fsddd\" (UID: \"3ef26d78-e0d5-40d5-a9a4-217b5671185b\") " pod="openshift-ingress/e2e-api-b-05b5d9-openshift-default-7b8d7d6747-fsddd" Jul 06 16:09:39.903038 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:09:39.903012 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3ef26d78-e0d5-40d5-a9a4-217b5671185b-credential-socket\") pod \"e2e-api-b-05b5d9-openshift-default-7b8d7d6747-fsddd\" (UID: \"3ef26d78-e0d5-40d5-a9a4-217b5671185b\") " pod="openshift-ingress/e2e-api-b-05b5d9-openshift-default-7b8d7d6747-fsddd" Jul 06 16:09:39.903038 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:09:39.903038 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3ef26d78-e0d5-40d5-a9a4-217b5671185b-workload-certs\") pod \"e2e-api-b-05b5d9-openshift-default-7b8d7d6747-fsddd\" (UID: \"3ef26d78-e0d5-40d5-a9a4-217b5671185b\") " pod="openshift-ingress/e2e-api-b-05b5d9-openshift-default-7b8d7d6747-fsddd" Jul 06 16:09:39.903286 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:09:39.903265 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3ef26d78-e0d5-40d5-a9a4-217b5671185b-workload-socket\") pod \"e2e-api-b-05b5d9-openshift-default-7b8d7d6747-fsddd\" (UID: \"3ef26d78-e0d5-40d5-a9a4-217b5671185b\") " pod="openshift-ingress/e2e-api-b-05b5d9-openshift-default-7b8d7d6747-fsddd" Jul 06 16:09:39.903398 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:09:39.903382 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3ef26d78-e0d5-40d5-a9a4-217b5671185b-istiod-ca-cert\") pod \"e2e-api-b-05b5d9-openshift-default-7b8d7d6747-fsddd\" (UID: \"3ef26d78-e0d5-40d5-a9a4-217b5671185b\") " pod="openshift-ingress/e2e-api-b-05b5d9-openshift-default-7b8d7d6747-fsddd" Jul 06 16:09:39.904715 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:09:39.904699 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3ef26d78-e0d5-40d5-a9a4-217b5671185b-istio-envoy\") pod \"e2e-api-b-05b5d9-openshift-default-7b8d7d6747-fsddd\" (UID: \"3ef26d78-e0d5-40d5-a9a4-217b5671185b\") " pod="openshift-ingress/e2e-api-b-05b5d9-openshift-default-7b8d7d6747-fsddd" Jul 06 16:09:39.904925 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:09:39.904910 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3ef26d78-e0d5-40d5-a9a4-217b5671185b-istio-podinfo\") pod \"e2e-api-b-05b5d9-openshift-default-7b8d7d6747-fsddd\" (UID: \"3ef26d78-e0d5-40d5-a9a4-217b5671185b\") " pod="openshift-ingress/e2e-api-b-05b5d9-openshift-default-7b8d7d6747-fsddd" Jul 06 16:09:39.912739 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:09:39.912715 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vqlcf\" (UniqueName: \"kubernetes.io/projected/3ef26d78-e0d5-40d5-a9a4-217b5671185b-kube-api-access-vqlcf\") pod \"e2e-api-b-05b5d9-openshift-default-7b8d7d6747-fsddd\" (UID: \"3ef26d78-e0d5-40d5-a9a4-217b5671185b\") " pod="openshift-ingress/e2e-api-b-05b5d9-openshift-default-7b8d7d6747-fsddd" Jul 06 16:09:39.912848 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:09:39.912785 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3ef26d78-e0d5-40d5-a9a4-217b5671185b-istio-token\") pod \"e2e-api-b-05b5d9-openshift-default-7b8d7d6747-fsddd\" (UID: \"3ef26d78-e0d5-40d5-a9a4-217b5671185b\") " pod="openshift-ingress/e2e-api-b-05b5d9-openshift-default-7b8d7d6747-fsddd" Jul 06 16:09:40.021287 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:09:40.021200 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-05b5d9-openshift-default-7b8d7d6747-fsddd" Jul 06 16:09:40.175064 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:09:40.175029 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-05b5d9-openshift-default-7b8d7d6747-fsddd"] Jul 06 16:09:40.177089 ip-10-0-129-154 kubenswrapper[2563]: W0706 16:09:40.177063 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3ef26d78_e0d5_40d5_a9a4_217b5671185b.slice/crio-76e290b2e20a6bc928722d4ab3a91da723c2dfec50da228c8851713726056412 WatchSource:0}: Error finding container 76e290b2e20a6bc928722d4ab3a91da723c2dfec50da228c8851713726056412: Status 404 returned error can't find the container with id 76e290b2e20a6bc928722d4ab3a91da723c2dfec50da228c8851713726056412 Jul 06 16:09:40.179248 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:09:40.179218 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 06 16:09:40.179310 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:09:40.179281 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 06 16:09:40.179356 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:09:40.179319 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 06 16:09:40.354151 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:09:40.354059 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-05b5d9-openshift-default-7b8d7d6747-fsddd" event={"ID":"3ef26d78-e0d5-40d5-a9a4-217b5671185b","Type":"ContainerStarted","Data":"c7d44d346a1cf60cdc8b4768a6beea0d3c1f014c1eb87e702ec145039f5f0afc"} Jul 06 16:09:40.354151 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:09:40.354146 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-05b5d9-openshift-default-7b8d7d6747-fsddd" event={"ID":"3ef26d78-e0d5-40d5-a9a4-217b5671185b","Type":"ContainerStarted","Data":"76e290b2e20a6bc928722d4ab3a91da723c2dfec50da228c8851713726056412"} Jul 06 16:09:40.385775 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:09:40.385719 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-05b5d9-openshift-default-7b8d7d6747-fsddd" podStartSLOduration=1.385700269 podStartE2EDuration="1.385700269s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 16:09:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 16:09:40.383376867 +0000 UTC m=+2156.892610038" watchObservedRunningTime="2026-07-06 16:09:40.385700269 +0000 UTC m=+2156.894933444" Jul 06 16:09:41.021512 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:09:41.021468 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-05b5d9-openshift-default-7b8d7d6747-fsddd" Jul 06 16:09:41.026819 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:09:41.026784 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-05b5d9-openshift-default-7b8d7d6747-fsddd" Jul 06 16:09:41.359875 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:09:41.359796 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-05b5d9-openshift-default-7b8d7d6747-fsddd" Jul 06 16:09:41.360677 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:09:41.360655 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-05b5d9-openshift-default-7b8d7d6747-fsddd" Jul 06 16:09:56.409951 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:09:56.409354 2563 patch_prober.go:28] interesting pod/e2e-api-b-05b5d9-openshift-default-7b8d7d6747-fsddd container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.52:15021/healthz/ready\": read tcp 10.132.0.2:37502->10.132.0.52:15021: read: connection reset by peer" start-of-body= Jul 06 16:09:56.409951 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:09:56.409449 2563 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-b-05b5d9-openshift-default-7b8d7d6747-fsddd" podUID="3ef26d78-e0d5-40d5-a9a4-217b5671185b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.52:15021/healthz/ready\": read tcp 10.132.0.2:37502->10.132.0.52:15021: read: connection reset by peer" Jul 06 16:09:56.443044 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:09:56.443010 2563 generic.go:356] "Generic (PLEG): container finished" podID="3ef26d78-e0d5-40d5-a9a4-217b5671185b" containerID="c7d44d346a1cf60cdc8b4768a6beea0d3c1f014c1eb87e702ec145039f5f0afc" exitCode=0 Jul 06 16:09:56.443226 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:09:56.443073 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-05b5d9-openshift-default-7b8d7d6747-fsddd" event={"ID":"3ef26d78-e0d5-40d5-a9a4-217b5671185b","Type":"ContainerDied","Data":"c7d44d346a1cf60cdc8b4768a6beea0d3c1f014c1eb87e702ec145039f5f0afc"} Jul 06 16:09:56.443579 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:09:56.443562 2563 scope.go:117] "RemoveContainer" containerID="c7d44d346a1cf60cdc8b4768a6beea0d3c1f014c1eb87e702ec145039f5f0afc" Jul 06 16:09:56.444786 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:09:56.444753 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 06 16:09:56.444888 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:09:56.444841 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 06 16:09:56.444953 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:09:56.444893 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 06 16:09:57.450198 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:09:57.450166 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-05b5d9-openshift-default-7b8d7d6747-fsddd" event={"ID":"3ef26d78-e0d5-40d5-a9a4-217b5671185b","Type":"ContainerStarted","Data":"c06861c55f78c7498664add7bcb9ffa0d2dc6cf91e61c434c3ff9f0c715e8369"} Jul 06 16:09:58.022229 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:09:58.022191 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-05b5d9-openshift-default-7b8d7d6747-fsddd" Jul 06 16:09:58.022413 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:09:58.022390 2563 patch_prober.go:28] interesting pod/e2e-api-b-05b5d9-openshift-default-7b8d7d6747-fsddd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" start-of-body= Jul 06 16:09:58.022515 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:09:58.022467 2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-05b5d9-openshift-default-7b8d7d6747-fsddd" podUID="3ef26d78-e0d5-40d5-a9a4-217b5671185b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" Jul 06 16:09:59.022079 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:09:59.022037 2563 patch_prober.go:28] interesting pod/e2e-api-b-05b5d9-openshift-default-7b8d7d6747-fsddd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" start-of-body= Jul 06 16:09:59.022508 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:09:59.022092 2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-05b5d9-openshift-default-7b8d7d6747-fsddd" podUID="3ef26d78-e0d5-40d5-a9a4-217b5671185b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" Jul 06 16:10:00.022383 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:10:00.022338 2563 patch_prober.go:28] interesting pod/e2e-api-b-05b5d9-openshift-default-7b8d7d6747-fsddd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" start-of-body= Jul 06 16:10:00.022794 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:10:00.022447 2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-05b5d9-openshift-default-7b8d7d6747-fsddd" podUID="3ef26d78-e0d5-40d5-a9a4-217b5671185b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" Jul 06 16:10:01.022398 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:10:01.022355 2563 patch_prober.go:28] interesting pod/e2e-api-b-05b5d9-openshift-default-7b8d7d6747-fsddd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" start-of-body= Jul 06 16:10:01.022816 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:10:01.022458 2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-05b5d9-openshift-default-7b8d7d6747-fsddd" podUID="3ef26d78-e0d5-40d5-a9a4-217b5671185b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" Jul 06 16:10:02.025201 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:10:02.025167 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-05b5d9-openshift-default-7b8d7d6747-fsddd" Jul 06 16:10:02.025722 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:10:02.025453 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-05b5d9-openshift-default-7b8d7d6747-fsddd" Jul 06 16:10:02.026303 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:10:02.026288 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-05b5d9-openshift-default-7b8d7d6747-fsddd" Jul 06 16:10:11.461766 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:10:11.461724 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-05b5d9-openshift-default-7b8d7d6747-fsddd"] Jul 06 16:10:11.462162 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:10:11.461933 2563 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-05b5d9-openshift-default-7b8d7d6747-fsddd" podUID="3ef26d78-e0d5-40d5-a9a4-217b5671185b" containerName="istio-proxy" containerID="cri-o://c06861c55f78c7498664add7bcb9ffa0d2dc6cf91e61c434c3ff9f0c715e8369" gracePeriod=30 Jul 06 16:10:16.718838 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:10:16.718806 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-05b5d9-openshift-default-7b8d7d6747-fsddd" Jul 06 16:10:16.824154 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:10:16.824067 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3ef26d78-e0d5-40d5-a9a4-217b5671185b-istio-data\") pod \"3ef26d78-e0d5-40d5-a9a4-217b5671185b\" (UID: \"3ef26d78-e0d5-40d5-a9a4-217b5671185b\") " Jul 06 16:10:16.824154 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:10:16.824118 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3ef26d78-e0d5-40d5-a9a4-217b5671185b-workload-certs\") pod \"3ef26d78-e0d5-40d5-a9a4-217b5671185b\" (UID: \"3ef26d78-e0d5-40d5-a9a4-217b5671185b\") " Jul 06 16:10:16.824373 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:10:16.824169 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3ef26d78-e0d5-40d5-a9a4-217b5671185b-istio-token\") pod \"3ef26d78-e0d5-40d5-a9a4-217b5671185b\" (UID: \"3ef26d78-e0d5-40d5-a9a4-217b5671185b\") " Jul 06 16:10:16.824373 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:10:16.824187 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3ef26d78-e0d5-40d5-a9a4-217b5671185b-istio-envoy\") pod \"3ef26d78-e0d5-40d5-a9a4-217b5671185b\" (UID: \"3ef26d78-e0d5-40d5-a9a4-217b5671185b\") " Jul 06 16:10:16.824373 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:10:16.824236 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3ef26d78-e0d5-40d5-a9a4-217b5671185b-istio-podinfo\") pod \"3ef26d78-e0d5-40d5-a9a4-217b5671185b\" (UID: \"3ef26d78-e0d5-40d5-a9a4-217b5671185b\") " Jul 06 16:10:16.824373 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:10:16.824303 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3ef26d78-e0d5-40d5-a9a4-217b5671185b-istiod-ca-cert\") pod \"3ef26d78-e0d5-40d5-a9a4-217b5671185b\" (UID: \"3ef26d78-e0d5-40d5-a9a4-217b5671185b\") " Jul 06 16:10:16.824373 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:10:16.824331 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3ef26d78-e0d5-40d5-a9a4-217b5671185b-workload-socket\") pod \"3ef26d78-e0d5-40d5-a9a4-217b5671185b\" (UID: \"3ef26d78-e0d5-40d5-a9a4-217b5671185b\") " Jul 06 16:10:16.824373 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:10:16.824372 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3ef26d78-e0d5-40d5-a9a4-217b5671185b-credential-socket\") pod \"3ef26d78-e0d5-40d5-a9a4-217b5671185b\" (UID: \"3ef26d78-e0d5-40d5-a9a4-217b5671185b\") " Jul 06 16:10:16.824797 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:10:16.824403 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vqlcf\" (UniqueName: \"kubernetes.io/projected/3ef26d78-e0d5-40d5-a9a4-217b5671185b-kube-api-access-vqlcf\") pod \"3ef26d78-e0d5-40d5-a9a4-217b5671185b\" (UID: \"3ef26d78-e0d5-40d5-a9a4-217b5671185b\") " Jul 06 16:10:16.824797 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:10:16.824503 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3ef26d78-e0d5-40d5-a9a4-217b5671185b-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "3ef26d78-e0d5-40d5-a9a4-217b5671185b" (UID: "3ef26d78-e0d5-40d5-a9a4-217b5671185b"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 16:10:16.824797 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:10:16.824761 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3ef26d78-e0d5-40d5-a9a4-217b5671185b-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "3ef26d78-e0d5-40d5-a9a4-217b5671185b" (UID: "3ef26d78-e0d5-40d5-a9a4-217b5671185b"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 16:10:16.824797 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:10:16.824781 2563 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3ef26d78-e0d5-40d5-a9a4-217b5671185b-workload-certs\") on node \"ip-10-0-129-154.ec2.internal\" DevicePath \"\"" Jul 06 16:10:16.824797 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:10:16.824788 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3ef26d78-e0d5-40d5-a9a4-217b5671185b-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "3ef26d78-e0d5-40d5-a9a4-217b5671185b" (UID: "3ef26d78-e0d5-40d5-a9a4-217b5671185b"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 16:10:16.825219 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:10:16.825050 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3ef26d78-e0d5-40d5-a9a4-217b5671185b-istio-data" (OuterVolumeSpecName: "istio-data") pod "3ef26d78-e0d5-40d5-a9a4-217b5671185b" (UID: "3ef26d78-e0d5-40d5-a9a4-217b5671185b"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 16:10:16.825219 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:10:16.825078 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3ef26d78-e0d5-40d5-a9a4-217b5671185b-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "3ef26d78-e0d5-40d5-a9a4-217b5671185b" (UID: "3ef26d78-e0d5-40d5-a9a4-217b5671185b"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 16:10:16.826680 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:10:16.826607 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3ef26d78-e0d5-40d5-a9a4-217b5671185b-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "3ef26d78-e0d5-40d5-a9a4-217b5671185b" (UID: "3ef26d78-e0d5-40d5-a9a4-217b5671185b"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 16:10:16.826864 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:10:16.826846 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3ef26d78-e0d5-40d5-a9a4-217b5671185b-istio-token" (OuterVolumeSpecName: "istio-token") pod "3ef26d78-e0d5-40d5-a9a4-217b5671185b" (UID: "3ef26d78-e0d5-40d5-a9a4-217b5671185b"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 16:10:16.826962 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:10:16.826941 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3ef26d78-e0d5-40d5-a9a4-217b5671185b-kube-api-access-vqlcf" (OuterVolumeSpecName: "kube-api-access-vqlcf") pod "3ef26d78-e0d5-40d5-a9a4-217b5671185b" (UID: "3ef26d78-e0d5-40d5-a9a4-217b5671185b"). InnerVolumeSpecName "kube-api-access-vqlcf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 16:10:16.827076 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:10:16.827057 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/3ef26d78-e0d5-40d5-a9a4-217b5671185b-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "3ef26d78-e0d5-40d5-a9a4-217b5671185b" (UID: "3ef26d78-e0d5-40d5-a9a4-217b5671185b"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 06 16:10:16.925271 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:10:16.925240 2563 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3ef26d78-e0d5-40d5-a9a4-217b5671185b-istio-podinfo\") on node \"ip-10-0-129-154.ec2.internal\" DevicePath \"\"" Jul 06 16:10:16.925271 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:10:16.925269 2563 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3ef26d78-e0d5-40d5-a9a4-217b5671185b-istiod-ca-cert\") on node \"ip-10-0-129-154.ec2.internal\" DevicePath \"\"" Jul 06 16:10:16.925519 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:10:16.925281 2563 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3ef26d78-e0d5-40d5-a9a4-217b5671185b-workload-socket\") on node \"ip-10-0-129-154.ec2.internal\" DevicePath \"\"" Jul 06 16:10:16.925519 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:10:16.925292 2563 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3ef26d78-e0d5-40d5-a9a4-217b5671185b-credential-socket\") on node \"ip-10-0-129-154.ec2.internal\" DevicePath \"\"" Jul 06 16:10:16.925519 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:10:16.925306 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vqlcf\" (UniqueName: \"kubernetes.io/projected/3ef26d78-e0d5-40d5-a9a4-217b5671185b-kube-api-access-vqlcf\") on node \"ip-10-0-129-154.ec2.internal\" DevicePath \"\"" Jul 06 16:10:16.925519 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:10:16.925317 2563 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3ef26d78-e0d5-40d5-a9a4-217b5671185b-istio-data\") on node \"ip-10-0-129-154.ec2.internal\" DevicePath \"\"" Jul 06 16:10:16.925519 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:10:16.925327 2563 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3ef26d78-e0d5-40d5-a9a4-217b5671185b-istio-token\") on node \"ip-10-0-129-154.ec2.internal\" DevicePath \"\"" Jul 06 16:10:16.925519 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:10:16.925340 2563 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3ef26d78-e0d5-40d5-a9a4-217b5671185b-istio-envoy\") on node \"ip-10-0-129-154.ec2.internal\" DevicePath \"\"" Jul 06 16:10:17.565208 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:10:17.565174 2563 generic.go:356] "Generic (PLEG): container finished" podID="3ef26d78-e0d5-40d5-a9a4-217b5671185b" containerID="c06861c55f78c7498664add7bcb9ffa0d2dc6cf91e61c434c3ff9f0c715e8369" exitCode=0 Jul 06 16:10:17.565377 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:10:17.565231 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-05b5d9-openshift-default-7b8d7d6747-fsddd" Jul 06 16:10:17.565377 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:10:17.565251 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-05b5d9-openshift-default-7b8d7d6747-fsddd" event={"ID":"3ef26d78-e0d5-40d5-a9a4-217b5671185b","Type":"ContainerDied","Data":"c06861c55f78c7498664add7bcb9ffa0d2dc6cf91e61c434c3ff9f0c715e8369"} Jul 06 16:10:17.565377 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:10:17.565299 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-05b5d9-openshift-default-7b8d7d6747-fsddd" event={"ID":"3ef26d78-e0d5-40d5-a9a4-217b5671185b","Type":"ContainerDied","Data":"76e290b2e20a6bc928722d4ab3a91da723c2dfec50da228c8851713726056412"} Jul 06 16:10:17.565377 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:10:17.565321 2563 scope.go:117] "RemoveContainer" containerID="c06861c55f78c7498664add7bcb9ffa0d2dc6cf91e61c434c3ff9f0c715e8369" Jul 06 16:10:17.575238 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:10:17.575214 2563 scope.go:117] "RemoveContainer" containerID="c7d44d346a1cf60cdc8b4768a6beea0d3c1f014c1eb87e702ec145039f5f0afc" Jul 06 16:10:17.584239 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:10:17.584044 2563 scope.go:117] "RemoveContainer" containerID="c06861c55f78c7498664add7bcb9ffa0d2dc6cf91e61c434c3ff9f0c715e8369" Jul 06 16:10:17.584346 ip-10-0-129-154 kubenswrapper[2563]: E0706 16:10:17.584327 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c06861c55f78c7498664add7bcb9ffa0d2dc6cf91e61c434c3ff9f0c715e8369\": container with ID starting with c06861c55f78c7498664add7bcb9ffa0d2dc6cf91e61c434c3ff9f0c715e8369 not found: ID does not exist" containerID="c06861c55f78c7498664add7bcb9ffa0d2dc6cf91e61c434c3ff9f0c715e8369" Jul 06 16:10:17.584388 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:10:17.584357 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c06861c55f78c7498664add7bcb9ffa0d2dc6cf91e61c434c3ff9f0c715e8369"} err="failed to get container status \"c06861c55f78c7498664add7bcb9ffa0d2dc6cf91e61c434c3ff9f0c715e8369\": rpc error: code = NotFound desc = could not find container \"c06861c55f78c7498664add7bcb9ffa0d2dc6cf91e61c434c3ff9f0c715e8369\": container with ID starting with c06861c55f78c7498664add7bcb9ffa0d2dc6cf91e61c434c3ff9f0c715e8369 not found: ID does not exist" Jul 06 16:10:17.584388 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:10:17.584374 2563 scope.go:117] "RemoveContainer" containerID="c7d44d346a1cf60cdc8b4768a6beea0d3c1f014c1eb87e702ec145039f5f0afc" Jul 06 16:10:17.584625 ip-10-0-129-154 kubenswrapper[2563]: E0706 16:10:17.584608 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c7d44d346a1cf60cdc8b4768a6beea0d3c1f014c1eb87e702ec145039f5f0afc\": container with ID starting with c7d44d346a1cf60cdc8b4768a6beea0d3c1f014c1eb87e702ec145039f5f0afc not found: ID does not exist" containerID="c7d44d346a1cf60cdc8b4768a6beea0d3c1f014c1eb87e702ec145039f5f0afc" Jul 06 16:10:17.584677 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:10:17.584631 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c7d44d346a1cf60cdc8b4768a6beea0d3c1f014c1eb87e702ec145039f5f0afc"} err="failed to get container status \"c7d44d346a1cf60cdc8b4768a6beea0d3c1f014c1eb87e702ec145039f5f0afc\": rpc error: code = NotFound desc = could not find container \"c7d44d346a1cf60cdc8b4768a6beea0d3c1f014c1eb87e702ec145039f5f0afc\": container with ID starting with c7d44d346a1cf60cdc8b4768a6beea0d3c1f014c1eb87e702ec145039f5f0afc not found: ID does not exist" Jul 06 16:10:17.597271 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:10:17.597245 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-05b5d9-openshift-default-7b8d7d6747-fsddd"] Jul 06 16:10:17.614006 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:10:17.613983 2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-05b5d9-openshift-default-7b8d7d6747-fsddd"] Jul 06 16:10:17.946257 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:10:17.946224 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3ef26d78-e0d5-40d5-a9a4-217b5671185b" path="/var/lib/kubelet/pods/3ef26d78-e0d5-40d5-a9a4-217b5671185b/volumes" Jul 06 16:13:44.092330 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:13:44.092297 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-64786d9cbb-trjfs_bf5c30e2-bf88-468b-adf2-2e8f73f136b4/cluster-monitoring-operator/0.log" Jul 06 16:13:44.095683 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:13:44.095662 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-l47bf_9ad7e968-ed2e-4cbe-9b4b-780bc9d01b88/console-operator/2.log" Jul 06 16:13:44.102253 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:13:44.102229 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-64786d9cbb-trjfs_bf5c30e2-bf88-468b-adf2-2e8f73f136b4/cluster-monitoring-operator/0.log" Jul 06 16:13:44.105702 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:13:44.105686 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-l47bf_9ad7e968-ed2e-4cbe-9b4b-780bc9d01b88/console-operator/2.log" Jul 06 16:13:49.629643 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:13:49.629607 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-c8a9d1/sub-test-model-c8a9d1-kserve-6fd688d795-5ptvv"] Jul 06 16:13:49.630015 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:13:49.629960 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3ef26d78-e0d5-40d5-a9a4-217b5671185b" containerName="istio-proxy" Jul 06 16:13:49.630015 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:13:49.629971 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="3ef26d78-e0d5-40d5-a9a4-217b5671185b" containerName="istio-proxy" Jul 06 16:13:49.630015 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:13:49.629982 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3aefe2ab-e0be-43fb-802d-2544ee3d0876" containerName="istio-proxy" Jul 06 16:13:49.630015 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:13:49.629987 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="3aefe2ab-e0be-43fb-802d-2544ee3d0876" containerName="istio-proxy" Jul 06 16:13:49.630146 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:13:49.630053 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="3ef26d78-e0d5-40d5-a9a4-217b5671185b" containerName="istio-proxy" Jul 06 16:13:49.630146 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:13:49.630063 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="3ef26d78-e0d5-40d5-a9a4-217b5671185b" containerName="istio-proxy" Jul 06 16:13:49.633146 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:13:49.633125 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-c8a9d1/sub-test-model-c8a9d1-kserve-6fd688d795-5ptvv" Jul 06 16:13:49.637908 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:13:49.637881 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-c8a9d1\"/\"openshift-service-ca.crt\"" Jul 06 16:13:49.638014 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:13:49.637907 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-c8a9d1\"/\"kube-root-ca.crt\"" Jul 06 16:13:49.638014 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:13:49.637884 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-c8a9d1\"/\"sub-test-model-c8a9d1-kserve-self-signed-certs\"" Jul 06 16:13:49.638014 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:13:49.637880 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-c8a9d1\"/\"default-dockercfg-fkgd7\"" Jul 06 16:13:49.651342 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:13:49.651308 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-c8a9d1/sub-test-model-c8a9d1-kserve-6fd688d795-5ptvv"] Jul 06 16:13:49.684519 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:13:49.684491 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/15378c4d-6744-47bf-b92e-a8558a0fc50f-dshm\") pod \"sub-test-model-c8a9d1-kserve-6fd688d795-5ptvv\" (UID: \"15378c4d-6744-47bf-b92e-a8558a0fc50f\") " pod="ai-tenant-e2e-shared-a-c8a9d1/sub-test-model-c8a9d1-kserve-6fd688d795-5ptvv" Jul 06 16:13:49.684613 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:13:49.684524 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5rhhq\" (UniqueName: \"kubernetes.io/projected/15378c4d-6744-47bf-b92e-a8558a0fc50f-kube-api-access-5rhhq\") pod \"sub-test-model-c8a9d1-kserve-6fd688d795-5ptvv\" (UID: \"15378c4d-6744-47bf-b92e-a8558a0fc50f\") " pod="ai-tenant-e2e-shared-a-c8a9d1/sub-test-model-c8a9d1-kserve-6fd688d795-5ptvv" Jul 06 16:13:49.684613 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:13:49.684544 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/15378c4d-6744-47bf-b92e-a8558a0fc50f-home\") pod \"sub-test-model-c8a9d1-kserve-6fd688d795-5ptvv\" (UID: \"15378c4d-6744-47bf-b92e-a8558a0fc50f\") " pod="ai-tenant-e2e-shared-a-c8a9d1/sub-test-model-c8a9d1-kserve-6fd688d795-5ptvv" Jul 06 16:13:49.684613 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:13:49.684568 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/15378c4d-6744-47bf-b92e-a8558a0fc50f-tmp-dir\") pod \"sub-test-model-c8a9d1-kserve-6fd688d795-5ptvv\" (UID: \"15378c4d-6744-47bf-b92e-a8558a0fc50f\") " pod="ai-tenant-e2e-shared-a-c8a9d1/sub-test-model-c8a9d1-kserve-6fd688d795-5ptvv" Jul 06 16:13:49.684719 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:13:49.684646 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/15378c4d-6744-47bf-b92e-a8558a0fc50f-tls-certs\") pod \"sub-test-model-c8a9d1-kserve-6fd688d795-5ptvv\" (UID: \"15378c4d-6744-47bf-b92e-a8558a0fc50f\") " pod="ai-tenant-e2e-shared-a-c8a9d1/sub-test-model-c8a9d1-kserve-6fd688d795-5ptvv" Jul 06 16:13:49.684719 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:13:49.684687 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/15378c4d-6744-47bf-b92e-a8558a0fc50f-model-cache\") pod \"sub-test-model-c8a9d1-kserve-6fd688d795-5ptvv\" (UID: \"15378c4d-6744-47bf-b92e-a8558a0fc50f\") " pod="ai-tenant-e2e-shared-a-c8a9d1/sub-test-model-c8a9d1-kserve-6fd688d795-5ptvv" Jul 06 16:13:49.785800 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:13:49.785765 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/15378c4d-6744-47bf-b92e-a8558a0fc50f-tls-certs\") pod \"sub-test-model-c8a9d1-kserve-6fd688d795-5ptvv\" (UID: \"15378c4d-6744-47bf-b92e-a8558a0fc50f\") " pod="ai-tenant-e2e-shared-a-c8a9d1/sub-test-model-c8a9d1-kserve-6fd688d795-5ptvv" Jul 06 16:13:49.785992 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:13:49.785813 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/15378c4d-6744-47bf-b92e-a8558a0fc50f-model-cache\") pod \"sub-test-model-c8a9d1-kserve-6fd688d795-5ptvv\" (UID: \"15378c4d-6744-47bf-b92e-a8558a0fc50f\") " pod="ai-tenant-e2e-shared-a-c8a9d1/sub-test-model-c8a9d1-kserve-6fd688d795-5ptvv" Jul 06 16:13:49.785992 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:13:49.785852 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/15378c4d-6744-47bf-b92e-a8558a0fc50f-dshm\") pod \"sub-test-model-c8a9d1-kserve-6fd688d795-5ptvv\" (UID: \"15378c4d-6744-47bf-b92e-a8558a0fc50f\") " pod="ai-tenant-e2e-shared-a-c8a9d1/sub-test-model-c8a9d1-kserve-6fd688d795-5ptvv" Jul 06 16:13:49.785992 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:13:49.785872 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5rhhq\" (UniqueName: \"kubernetes.io/projected/15378c4d-6744-47bf-b92e-a8558a0fc50f-kube-api-access-5rhhq\") pod \"sub-test-model-c8a9d1-kserve-6fd688d795-5ptvv\" (UID: \"15378c4d-6744-47bf-b92e-a8558a0fc50f\") " pod="ai-tenant-e2e-shared-a-c8a9d1/sub-test-model-c8a9d1-kserve-6fd688d795-5ptvv" Jul 06 16:13:49.786157 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:13:49.786029 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/15378c4d-6744-47bf-b92e-a8558a0fc50f-home\") pod \"sub-test-model-c8a9d1-kserve-6fd688d795-5ptvv\" (UID: \"15378c4d-6744-47bf-b92e-a8558a0fc50f\") " pod="ai-tenant-e2e-shared-a-c8a9d1/sub-test-model-c8a9d1-kserve-6fd688d795-5ptvv" Jul 06 16:13:49.786157 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:13:49.786071 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/15378c4d-6744-47bf-b92e-a8558a0fc50f-tmp-dir\") pod \"sub-test-model-c8a9d1-kserve-6fd688d795-5ptvv\" (UID: \"15378c4d-6744-47bf-b92e-a8558a0fc50f\") " pod="ai-tenant-e2e-shared-a-c8a9d1/sub-test-model-c8a9d1-kserve-6fd688d795-5ptvv" Jul 06 16:13:49.786255 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:13:49.786216 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/15378c4d-6744-47bf-b92e-a8558a0fc50f-model-cache\") pod \"sub-test-model-c8a9d1-kserve-6fd688d795-5ptvv\" (UID: \"15378c4d-6744-47bf-b92e-a8558a0fc50f\") " pod="ai-tenant-e2e-shared-a-c8a9d1/sub-test-model-c8a9d1-kserve-6fd688d795-5ptvv" Jul 06 16:13:49.786394 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:13:49.786374 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/15378c4d-6744-47bf-b92e-a8558a0fc50f-tmp-dir\") pod \"sub-test-model-c8a9d1-kserve-6fd688d795-5ptvv\" (UID: \"15378c4d-6744-47bf-b92e-a8558a0fc50f\") " pod="ai-tenant-e2e-shared-a-c8a9d1/sub-test-model-c8a9d1-kserve-6fd688d795-5ptvv" Jul 06 16:13:49.786487 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:13:49.786396 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/15378c4d-6744-47bf-b92e-a8558a0fc50f-home\") pod \"sub-test-model-c8a9d1-kserve-6fd688d795-5ptvv\" (UID: \"15378c4d-6744-47bf-b92e-a8558a0fc50f\") " pod="ai-tenant-e2e-shared-a-c8a9d1/sub-test-model-c8a9d1-kserve-6fd688d795-5ptvv" Jul 06 16:13:49.788307 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:13:49.788284 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/15378c4d-6744-47bf-b92e-a8558a0fc50f-dshm\") pod \"sub-test-model-c8a9d1-kserve-6fd688d795-5ptvv\" (UID: \"15378c4d-6744-47bf-b92e-a8558a0fc50f\") " pod="ai-tenant-e2e-shared-a-c8a9d1/sub-test-model-c8a9d1-kserve-6fd688d795-5ptvv" Jul 06 16:13:49.788579 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:13:49.788562 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/15378c4d-6744-47bf-b92e-a8558a0fc50f-tls-certs\") pod \"sub-test-model-c8a9d1-kserve-6fd688d795-5ptvv\" (UID: \"15378c4d-6744-47bf-b92e-a8558a0fc50f\") " pod="ai-tenant-e2e-shared-a-c8a9d1/sub-test-model-c8a9d1-kserve-6fd688d795-5ptvv" Jul 06 16:13:49.795145 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:13:49.795122 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5rhhq\" (UniqueName: \"kubernetes.io/projected/15378c4d-6744-47bf-b92e-a8558a0fc50f-kube-api-access-5rhhq\") pod \"sub-test-model-c8a9d1-kserve-6fd688d795-5ptvv\" (UID: \"15378c4d-6744-47bf-b92e-a8558a0fc50f\") " pod="ai-tenant-e2e-shared-a-c8a9d1/sub-test-model-c8a9d1-kserve-6fd688d795-5ptvv" Jul 06 16:13:49.946581 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:13:49.946275 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-c8a9d1/sub-test-model-c8a9d1-kserve-6fd688d795-5ptvv" Jul 06 16:13:50.087009 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:13:50.086981 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-c8a9d1/sub-test-model-c8a9d1-kserve-6fd688d795-5ptvv"] Jul 06 16:13:50.088758 ip-10-0-129-154 kubenswrapper[2563]: W0706 16:13:50.088728 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod15378c4d_6744_47bf_b92e_a8558a0fc50f.slice/crio-17c3c206165b2ad0c7d29ba7a4d4bf6e334278e954b4cc27c4854476e37fe395 WatchSource:0}: Error finding container 17c3c206165b2ad0c7d29ba7a4d4bf6e334278e954b4cc27c4854476e37fe395: Status 404 returned error can't find the container with id 17c3c206165b2ad0c7d29ba7a4d4bf6e334278e954b4cc27c4854476e37fe395 Jul 06 16:13:50.725218 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:13:50.725185 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-c8a9d1/sub-test-model-c8a9d1-kserve-6fd688d795-5ptvv" event={"ID":"15378c4d-6744-47bf-b92e-a8558a0fc50f","Type":"ContainerStarted","Data":"9e952fd668b23197b8f66ca7ff6f205f004bfad513df8109564f9ca547c7b98d"} Jul 06 16:13:50.725218 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:13:50.725224 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-c8a9d1/sub-test-model-c8a9d1-kserve-6fd688d795-5ptvv" event={"ID":"15378c4d-6744-47bf-b92e-a8558a0fc50f","Type":"ContainerStarted","Data":"17c3c206165b2ad0c7d29ba7a4d4bf6e334278e954b4cc27c4854476e37fe395"} Jul 06 16:13:50.743206 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:13:50.743158 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-c8a9d1/sub-test-model-c8a9d1-kserve-6fd688d795-5ptvv" podStartSLOduration=1.743145217 podStartE2EDuration="1.743145217s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 16:13:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 16:13:50.74244214 +0000 UTC m=+2407.251675305" watchObservedRunningTime="2026-07-06 16:13:50.743145217 +0000 UTC m=+2407.252378387" Jul 06 16:13:59.947773 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:13:59.947741 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-c8a9d1/sub-test-model-c8a9d1-kserve-6fd688d795-5ptvv" Jul 06 16:13:59.948153 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:13:59.947785 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-c8a9d1/sub-test-model-c8a9d1-kserve-6fd688d795-5ptvv" Jul 06 16:13:59.960386 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:13:59.960361 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-c8a9d1/sub-test-model-c8a9d1-kserve-6fd688d795-5ptvv" Jul 06 16:14:00.792698 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:14:00.792668 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-c8a9d1/sub-test-model-c8a9d1-kserve-6fd688d795-5ptvv" Jul 06 16:15:00.138886 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:15:00.138845 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-key-cleanup-e2e-shared-b-b747bf-29722575-6v8pk"] Jul 06 16:15:00.141387 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:15:00.139216 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3ef26d78-e0d5-40d5-a9a4-217b5671185b" containerName="istio-proxy" Jul 06 16:15:00.141387 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:15:00.139228 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="3ef26d78-e0d5-40d5-a9a4-217b5671185b" containerName="istio-proxy" Jul 06 16:15:00.142356 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:15:00.142340 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-key-cleanup-29722575-nsfs2"] Jul 06 16:15:00.142495 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:15:00.142472 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-e2e-shared-b-b747bf-29722575-6v8pk" Jul 06 16:15:00.145349 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:15:00.145329 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-hz5zw\"" Jul 06 16:15:00.145548 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:15:00.145533 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-key-cleanup-e2e-shared-a-c8a9d1-29722575-kz66l"] Jul 06 16:15:00.145647 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:15:00.145631 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29722575-nsfs2" Jul 06 16:15:00.148817 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:15:00.148798 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-e2e-shared-a-c8a9d1-29722575-kz66l" Jul 06 16:15:00.169293 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:15:00.169257 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-e2e-shared-a-c8a9d1-29722575-kz66l"] Jul 06 16:15:00.173472 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:15:00.173312 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-29722575-nsfs2"] Jul 06 16:15:00.175764 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:15:00.175736 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-e2e-shared-b-b747bf-29722575-6v8pk"] Jul 06 16:15:00.239411 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:15:00.239375 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vx7xj\" (UniqueName: \"kubernetes.io/projected/81ab3838-90d4-4d95-9711-62cde5ce0114-kube-api-access-vx7xj\") pod \"maas-api-key-cleanup-e2e-shared-a-c8a9d1-29722575-kz66l\" (UID: \"81ab3838-90d4-4d95-9711-62cde5ce0114\") " pod="opendatahub/maas-api-key-cleanup-e2e-shared-a-c8a9d1-29722575-kz66l" Jul 06 16:15:00.239625 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:15:00.239446 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tbtkn\" (UniqueName: \"kubernetes.io/projected/561fea79-1ccf-42e6-88a6-8bdfadafffed-kube-api-access-tbtkn\") pod \"maas-api-key-cleanup-e2e-shared-b-b747bf-29722575-6v8pk\" (UID: \"561fea79-1ccf-42e6-88a6-8bdfadafffed\") " pod="opendatahub/maas-api-key-cleanup-e2e-shared-b-b747bf-29722575-6v8pk" Jul 06 16:15:00.239625 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:15:00.239532 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s58c2\" (UniqueName: \"kubernetes.io/projected/d9577510-00ce-444f-8ecb-78f6b6c71b3e-kube-api-access-s58c2\") pod \"maas-api-key-cleanup-29722575-nsfs2\" (UID: \"d9577510-00ce-444f-8ecb-78f6b6c71b3e\") " pod="opendatahub/maas-api-key-cleanup-29722575-nsfs2" Jul 06 16:15:00.340672 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:15:00.340627 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tbtkn\" (UniqueName: \"kubernetes.io/projected/561fea79-1ccf-42e6-88a6-8bdfadafffed-kube-api-access-tbtkn\") pod \"maas-api-key-cleanup-e2e-shared-b-b747bf-29722575-6v8pk\" (UID: \"561fea79-1ccf-42e6-88a6-8bdfadafffed\") " pod="opendatahub/maas-api-key-cleanup-e2e-shared-b-b747bf-29722575-6v8pk" Jul 06 16:15:00.340829 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:15:00.340687 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s58c2\" (UniqueName: \"kubernetes.io/projected/d9577510-00ce-444f-8ecb-78f6b6c71b3e-kube-api-access-s58c2\") pod \"maas-api-key-cleanup-29722575-nsfs2\" (UID: \"d9577510-00ce-444f-8ecb-78f6b6c71b3e\") " pod="opendatahub/maas-api-key-cleanup-29722575-nsfs2" Jul 06 16:15:00.340829 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:15:00.340789 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vx7xj\" (UniqueName: \"kubernetes.io/projected/81ab3838-90d4-4d95-9711-62cde5ce0114-kube-api-access-vx7xj\") pod \"maas-api-key-cleanup-e2e-shared-a-c8a9d1-29722575-kz66l\" (UID: \"81ab3838-90d4-4d95-9711-62cde5ce0114\") " pod="opendatahub/maas-api-key-cleanup-e2e-shared-a-c8a9d1-29722575-kz66l" Jul 06 16:15:00.349868 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:15:00.349833 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vx7xj\" (UniqueName: \"kubernetes.io/projected/81ab3838-90d4-4d95-9711-62cde5ce0114-kube-api-access-vx7xj\") pod \"maas-api-key-cleanup-e2e-shared-a-c8a9d1-29722575-kz66l\" (UID: \"81ab3838-90d4-4d95-9711-62cde5ce0114\") " pod="opendatahub/maas-api-key-cleanup-e2e-shared-a-c8a9d1-29722575-kz66l" Jul 06 16:15:00.349868 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:15:00.349861 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-s58c2\" (UniqueName: \"kubernetes.io/projected/d9577510-00ce-444f-8ecb-78f6b6c71b3e-kube-api-access-s58c2\") pod \"maas-api-key-cleanup-29722575-nsfs2\" (UID: \"d9577510-00ce-444f-8ecb-78f6b6c71b3e\") " pod="opendatahub/maas-api-key-cleanup-29722575-nsfs2" Jul 06 16:15:00.350034 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:15:00.349922 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tbtkn\" (UniqueName: \"kubernetes.io/projected/561fea79-1ccf-42e6-88a6-8bdfadafffed-kube-api-access-tbtkn\") pod \"maas-api-key-cleanup-e2e-shared-b-b747bf-29722575-6v8pk\" (UID: \"561fea79-1ccf-42e6-88a6-8bdfadafffed\") " pod="opendatahub/maas-api-key-cleanup-e2e-shared-b-b747bf-29722575-6v8pk" Jul 06 16:15:00.455072 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:15:00.455039 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-e2e-shared-b-b747bf-29722575-6v8pk" Jul 06 16:15:00.461212 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:15:00.461188 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29722575-nsfs2" Jul 06 16:15:00.466919 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:15:00.466902 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-e2e-shared-a-c8a9d1-29722575-kz66l" Jul 06 16:15:00.624809 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:15:00.624167 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-e2e-shared-b-b747bf-29722575-6v8pk"] Jul 06 16:15:00.653560 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:15:00.653277 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-29722575-nsfs2"] Jul 06 16:15:00.655624 ip-10-0-129-154 kubenswrapper[2563]: W0706 16:15:00.655594 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd9577510_00ce_444f_8ecb_78f6b6c71b3e.slice/crio-0666535e3608a8c7256530cdcae764904b2effc25d89d7b6db35829395b6bfcd WatchSource:0}: Error finding container 0666535e3608a8c7256530cdcae764904b2effc25d89d7b6db35829395b6bfcd: Status 404 returned error can't find the container with id 0666535e3608a8c7256530cdcae764904b2effc25d89d7b6db35829395b6bfcd Jul 06 16:15:00.875842 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:15:00.875779 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-e2e-shared-a-c8a9d1-29722575-kz66l"] Jul 06 16:15:00.879186 ip-10-0-129-154 kubenswrapper[2563]: W0706 16:15:00.878917 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod81ab3838_90d4_4d95_9711_62cde5ce0114.slice/crio-fc74bd663df0e4765acf15e3bab7e883a69fa27c45263e43f1914cfe04695ae3 WatchSource:0}: Error finding container fc74bd663df0e4765acf15e3bab7e883a69fa27c45263e43f1914cfe04695ae3: Status 404 returned error can't find the container with id fc74bd663df0e4765acf15e3bab7e883a69fa27c45263e43f1914cfe04695ae3 Jul 06 16:15:01.116465 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:15:01.116413 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-e2e-shared-a-c8a9d1-29722575-kz66l" event={"ID":"81ab3838-90d4-4d95-9711-62cde5ce0114","Type":"ContainerStarted","Data":"fc74bd663df0e4765acf15e3bab7e883a69fa27c45263e43f1914cfe04695ae3"} Jul 06 16:15:01.117992 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:15:01.117964 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-e2e-shared-b-b747bf-29722575-6v8pk" event={"ID":"561fea79-1ccf-42e6-88a6-8bdfadafffed","Type":"ContainerStarted","Data":"c6f147e3dc15fed1e56190d630b19f8fc37ec9aef8cd97e1eb8fa5b56072e6e3"} Jul 06 16:15:01.118126 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:15:01.118018 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-e2e-shared-b-b747bf-29722575-6v8pk" event={"ID":"561fea79-1ccf-42e6-88a6-8bdfadafffed","Type":"ContainerStarted","Data":"317fe6f2632f0550c5a8bd029f9024f0a4d04f187651df7e54839af8636544e9"} Jul 06 16:15:01.119518 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:15:01.119496 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29722575-nsfs2" event={"ID":"d9577510-00ce-444f-8ecb-78f6b6c71b3e","Type":"ContainerStarted","Data":"bc76d09d29003ae194df13a8b4a84de5da9d37e7111f3358623c46944cc15212"} Jul 06 16:15:01.119600 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:15:01.119535 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29722575-nsfs2" event={"ID":"d9577510-00ce-444f-8ecb-78f6b6c71b3e","Type":"ContainerStarted","Data":"0666535e3608a8c7256530cdcae764904b2effc25d89d7b6db35829395b6bfcd"} Jul 06 16:15:01.138257 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:15:01.138166 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-key-cleanup-e2e-shared-b-b747bf-29722575-6v8pk" podStartSLOduration=1.138152604 podStartE2EDuration="1.138152604s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 16:15:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 16:15:01.13562671 +0000 UTC m=+2477.644859886" watchObservedRunningTime="2026-07-06 16:15:01.138152604 +0000 UTC m=+2477.647385778" Jul 06 16:15:01.154381 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:15:01.154341 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-key-cleanup-29722575-nsfs2" podStartSLOduration=1.154327432 podStartE2EDuration="1.154327432s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 16:15:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 16:15:01.152129431 +0000 UTC m=+2477.661362606" watchObservedRunningTime="2026-07-06 16:15:01.154327432 +0000 UTC m=+2477.663560606" Jul 06 16:15:02.126338 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:15:02.126245 2563 generic.go:356] "Generic (PLEG): container finished" podID="81ab3838-90d4-4d95-9711-62cde5ce0114" containerID="0ebf795b8197198c8885c10ee13a82268b7d1ed8ee8d9c3e010840c00801073c" exitCode=0 Jul 06 16:15:02.126531 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:15:02.126330 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-e2e-shared-a-c8a9d1-29722575-kz66l" event={"ID":"81ab3838-90d4-4d95-9711-62cde5ce0114","Type":"ContainerDied","Data":"0ebf795b8197198c8885c10ee13a82268b7d1ed8ee8d9c3e010840c00801073c"} Jul 06 16:15:02.127728 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:15:02.127706 2563 generic.go:356] "Generic (PLEG): container finished" podID="561fea79-1ccf-42e6-88a6-8bdfadafffed" containerID="c6f147e3dc15fed1e56190d630b19f8fc37ec9aef8cd97e1eb8fa5b56072e6e3" exitCode=0 Jul 06 16:15:02.127826 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:15:02.127781 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-e2e-shared-b-b747bf-29722575-6v8pk" event={"ID":"561fea79-1ccf-42e6-88a6-8bdfadafffed","Type":"ContainerDied","Data":"c6f147e3dc15fed1e56190d630b19f8fc37ec9aef8cd97e1eb8fa5b56072e6e3"} Jul 06 16:15:02.129096 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:15:02.129077 2563 generic.go:356] "Generic (PLEG): container finished" podID="d9577510-00ce-444f-8ecb-78f6b6c71b3e" containerID="bc76d09d29003ae194df13a8b4a84de5da9d37e7111f3358623c46944cc15212" exitCode=0 Jul 06 16:15:02.129186 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:15:02.129153 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29722575-nsfs2" event={"ID":"d9577510-00ce-444f-8ecb-78f6b6c71b3e","Type":"ContainerDied","Data":"bc76d09d29003ae194df13a8b4a84de5da9d37e7111f3358623c46944cc15212"} Jul 06 16:15:03.298084 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:15:03.298058 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-e2e-shared-b-b747bf-29722575-6v8pk" Jul 06 16:15:03.325807 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:15:03.325785 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-e2e-shared-a-c8a9d1-29722575-kz66l" Jul 06 16:15:03.329526 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:15:03.329503 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29722575-nsfs2" Jul 06 16:15:03.369151 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:15:03.369124 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tbtkn\" (UniqueName: \"kubernetes.io/projected/561fea79-1ccf-42e6-88a6-8bdfadafffed-kube-api-access-tbtkn\") pod \"561fea79-1ccf-42e6-88a6-8bdfadafffed\" (UID: \"561fea79-1ccf-42e6-88a6-8bdfadafffed\") " Jul 06 16:15:03.371524 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:15:03.371492 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/561fea79-1ccf-42e6-88a6-8bdfadafffed-kube-api-access-tbtkn" (OuterVolumeSpecName: "kube-api-access-tbtkn") pod "561fea79-1ccf-42e6-88a6-8bdfadafffed" (UID: "561fea79-1ccf-42e6-88a6-8bdfadafffed"). InnerVolumeSpecName "kube-api-access-tbtkn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 16:15:03.469809 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:15:03.469773 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-s58c2\" (UniqueName: \"kubernetes.io/projected/d9577510-00ce-444f-8ecb-78f6b6c71b3e-kube-api-access-s58c2\") pod \"d9577510-00ce-444f-8ecb-78f6b6c71b3e\" (UID: \"d9577510-00ce-444f-8ecb-78f6b6c71b3e\") " Jul 06 16:15:03.470023 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:15:03.469835 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vx7xj\" (UniqueName: \"kubernetes.io/projected/81ab3838-90d4-4d95-9711-62cde5ce0114-kube-api-access-vx7xj\") pod \"81ab3838-90d4-4d95-9711-62cde5ce0114\" (UID: \"81ab3838-90d4-4d95-9711-62cde5ce0114\") " Jul 06 16:15:03.470154 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:15:03.470137 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tbtkn\" (UniqueName: \"kubernetes.io/projected/561fea79-1ccf-42e6-88a6-8bdfadafffed-kube-api-access-tbtkn\") on node \"ip-10-0-129-154.ec2.internal\" DevicePath \"\"" Jul 06 16:15:03.472193 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:15:03.472162 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d9577510-00ce-444f-8ecb-78f6b6c71b3e-kube-api-access-s58c2" (OuterVolumeSpecName: "kube-api-access-s58c2") pod "d9577510-00ce-444f-8ecb-78f6b6c71b3e" (UID: "d9577510-00ce-444f-8ecb-78f6b6c71b3e"). InnerVolumeSpecName "kube-api-access-s58c2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 16:15:03.472302 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:15:03.472189 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/81ab3838-90d4-4d95-9711-62cde5ce0114-kube-api-access-vx7xj" (OuterVolumeSpecName: "kube-api-access-vx7xj") pod "81ab3838-90d4-4d95-9711-62cde5ce0114" (UID: "81ab3838-90d4-4d95-9711-62cde5ce0114"). InnerVolumeSpecName "kube-api-access-vx7xj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 16:15:03.571647 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:15:03.571609 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s58c2\" (UniqueName: \"kubernetes.io/projected/d9577510-00ce-444f-8ecb-78f6b6c71b3e-kube-api-access-s58c2\") on node \"ip-10-0-129-154.ec2.internal\" DevicePath \"\"" Jul 06 16:15:03.571647 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:15:03.571642 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vx7xj\" (UniqueName: \"kubernetes.io/projected/81ab3838-90d4-4d95-9711-62cde5ce0114-kube-api-access-vx7xj\") on node \"ip-10-0-129-154.ec2.internal\" DevicePath \"\"" Jul 06 16:15:04.141363 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:15:04.141326 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-e2e-shared-a-c8a9d1-29722575-kz66l" Jul 06 16:15:04.141583 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:15:04.141335 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-e2e-shared-a-c8a9d1-29722575-kz66l" event={"ID":"81ab3838-90d4-4d95-9711-62cde5ce0114","Type":"ContainerDied","Data":"fc74bd663df0e4765acf15e3bab7e883a69fa27c45263e43f1914cfe04695ae3"} Jul 06 16:15:04.141583 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:15:04.141500 2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="fc74bd663df0e4765acf15e3bab7e883a69fa27c45263e43f1914cfe04695ae3" Jul 06 16:15:04.142826 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:15:04.142809 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-e2e-shared-b-b747bf-29722575-6v8pk" Jul 06 16:15:04.142925 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:15:04.142851 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-e2e-shared-b-b747bf-29722575-6v8pk" event={"ID":"561fea79-1ccf-42e6-88a6-8bdfadafffed","Type":"ContainerDied","Data":"317fe6f2632f0550c5a8bd029f9024f0a4d04f187651df7e54839af8636544e9"} Jul 06 16:15:04.142925 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:15:04.142893 2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="317fe6f2632f0550c5a8bd029f9024f0a4d04f187651df7e54839af8636544e9" Jul 06 16:15:04.144226 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:15:04.144210 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29722575-nsfs2" Jul 06 16:15:04.144316 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:15:04.144233 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29722575-nsfs2" event={"ID":"d9577510-00ce-444f-8ecb-78f6b6c71b3e","Type":"ContainerDied","Data":"0666535e3608a8c7256530cdcae764904b2effc25d89d7b6db35829395b6bfcd"} Jul 06 16:15:04.144316 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:15:04.144265 2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0666535e3608a8c7256530cdcae764904b2effc25d89d7b6db35829395b6bfcd" Jul 06 16:15:34.537070 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:15:34.537033 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-c8a9d1/sub-test-model-c8a9d1-kserve-6fd688d795-5ptvv"] Jul 06 16:15:49.558226 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:15:49.558177 2563 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-c8a9d1/sub-test-model-c8a9d1-kserve-6fd688d795-5ptvv" podUID="15378c4d-6744-47bf-b92e-a8558a0fc50f" containerName="main" containerID="cri-o://9e952fd668b23197b8f66ca7ff6f205f004bfad513df8109564f9ca547c7b98d" gracePeriod=45 Jul 06 16:15:49.816771 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:15:49.816701 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-c8a9d1/sub-test-model-c8a9d1-kserve-6fd688d795-5ptvv" Jul 06 16:15:49.905283 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:15:49.905242 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/15378c4d-6744-47bf-b92e-a8558a0fc50f-dshm\") pod \"15378c4d-6744-47bf-b92e-a8558a0fc50f\" (UID: \"15378c4d-6744-47bf-b92e-a8558a0fc50f\") " Jul 06 16:15:49.905283 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:15:49.905286 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5rhhq\" (UniqueName: \"kubernetes.io/projected/15378c4d-6744-47bf-b92e-a8558a0fc50f-kube-api-access-5rhhq\") pod \"15378c4d-6744-47bf-b92e-a8558a0fc50f\" (UID: \"15378c4d-6744-47bf-b92e-a8558a0fc50f\") " Jul 06 16:15:49.905528 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:15:49.905318 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/15378c4d-6744-47bf-b92e-a8558a0fc50f-tmp-dir\") pod \"15378c4d-6744-47bf-b92e-a8558a0fc50f\" (UID: \"15378c4d-6744-47bf-b92e-a8558a0fc50f\") " Jul 06 16:15:49.905528 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:15:49.905340 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/15378c4d-6744-47bf-b92e-a8558a0fc50f-home\") pod \"15378c4d-6744-47bf-b92e-a8558a0fc50f\" (UID: \"15378c4d-6744-47bf-b92e-a8558a0fc50f\") " Jul 06 16:15:49.905528 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:15:49.905357 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/15378c4d-6744-47bf-b92e-a8558a0fc50f-model-cache\") pod \"15378c4d-6744-47bf-b92e-a8558a0fc50f\" (UID: \"15378c4d-6744-47bf-b92e-a8558a0fc50f\") " Jul 06 16:15:49.905528 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:15:49.905391 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/15378c4d-6744-47bf-b92e-a8558a0fc50f-tls-certs\") pod \"15378c4d-6744-47bf-b92e-a8558a0fc50f\" (UID: \"15378c4d-6744-47bf-b92e-a8558a0fc50f\") " Jul 06 16:15:49.905745 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:15:49.905650 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/15378c4d-6744-47bf-b92e-a8558a0fc50f-home" (OuterVolumeSpecName: "home") pod "15378c4d-6744-47bf-b92e-a8558a0fc50f" (UID: "15378c4d-6744-47bf-b92e-a8558a0fc50f"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 16:15:49.905745 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:15:49.905661 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/15378c4d-6744-47bf-b92e-a8558a0fc50f-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "15378c4d-6744-47bf-b92e-a8558a0fc50f" (UID: "15378c4d-6744-47bf-b92e-a8558a0fc50f"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 16:15:49.905745 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:15:49.905678 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/15378c4d-6744-47bf-b92e-a8558a0fc50f-model-cache" (OuterVolumeSpecName: "model-cache") pod "15378c4d-6744-47bf-b92e-a8558a0fc50f" (UID: "15378c4d-6744-47bf-b92e-a8558a0fc50f"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 16:15:49.905899 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:15:49.905758 2563 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/15378c4d-6744-47bf-b92e-a8558a0fc50f-tmp-dir\") on node \"ip-10-0-129-154.ec2.internal\" DevicePath \"\"" Jul 06 16:15:49.905899 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:15:49.905774 2563 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/15378c4d-6744-47bf-b92e-a8558a0fc50f-home\") on node \"ip-10-0-129-154.ec2.internal\" DevicePath \"\"" Jul 06 16:15:49.905899 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:15:49.905784 2563 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/15378c4d-6744-47bf-b92e-a8558a0fc50f-model-cache\") on node \"ip-10-0-129-154.ec2.internal\" DevicePath \"\"" Jul 06 16:15:49.907823 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:15:49.907793 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/15378c4d-6744-47bf-b92e-a8558a0fc50f-kube-api-access-5rhhq" (OuterVolumeSpecName: "kube-api-access-5rhhq") pod "15378c4d-6744-47bf-b92e-a8558a0fc50f" (UID: "15378c4d-6744-47bf-b92e-a8558a0fc50f"). InnerVolumeSpecName "kube-api-access-5rhhq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 16:15:49.907823 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:15:49.907804 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/15378c4d-6744-47bf-b92e-a8558a0fc50f-dshm" (OuterVolumeSpecName: "dshm") pod "15378c4d-6744-47bf-b92e-a8558a0fc50f" (UID: "15378c4d-6744-47bf-b92e-a8558a0fc50f"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 16:15:49.907823 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:15:49.907817 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/15378c4d-6744-47bf-b92e-a8558a0fc50f-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "15378c4d-6744-47bf-b92e-a8558a0fc50f" (UID: "15378c4d-6744-47bf-b92e-a8558a0fc50f"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 16:15:50.007155 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:15:50.007118 2563 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/15378c4d-6744-47bf-b92e-a8558a0fc50f-tls-certs\") on node \"ip-10-0-129-154.ec2.internal\" DevicePath \"\"" Jul 06 16:15:50.007155 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:15:50.007150 2563 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/15378c4d-6744-47bf-b92e-a8558a0fc50f-dshm\") on node \"ip-10-0-129-154.ec2.internal\" DevicePath \"\"" Jul 06 16:15:50.007155 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:15:50.007162 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5rhhq\" (UniqueName: \"kubernetes.io/projected/15378c4d-6744-47bf-b92e-a8558a0fc50f-kube-api-access-5rhhq\") on node \"ip-10-0-129-154.ec2.internal\" DevicePath \"\"" Jul 06 16:15:50.420269 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:15:50.420230 2563 generic.go:356] "Generic (PLEG): container finished" podID="15378c4d-6744-47bf-b92e-a8558a0fc50f" containerID="9e952fd668b23197b8f66ca7ff6f205f004bfad513df8109564f9ca547c7b98d" exitCode=0 Jul 06 16:15:50.420269 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:15:50.420271 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-c8a9d1/sub-test-model-c8a9d1-kserve-6fd688d795-5ptvv" Jul 06 16:15:50.420551 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:15:50.420310 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-c8a9d1/sub-test-model-c8a9d1-kserve-6fd688d795-5ptvv" event={"ID":"15378c4d-6744-47bf-b92e-a8558a0fc50f","Type":"ContainerDied","Data":"9e952fd668b23197b8f66ca7ff6f205f004bfad513df8109564f9ca547c7b98d"} Jul 06 16:15:50.420551 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:15:50.420354 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-c8a9d1/sub-test-model-c8a9d1-kserve-6fd688d795-5ptvv" event={"ID":"15378c4d-6744-47bf-b92e-a8558a0fc50f","Type":"ContainerDied","Data":"17c3c206165b2ad0c7d29ba7a4d4bf6e334278e954b4cc27c4854476e37fe395"} Jul 06 16:15:50.420551 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:15:50.420372 2563 scope.go:117] "RemoveContainer" containerID="9e952fd668b23197b8f66ca7ff6f205f004bfad513df8109564f9ca547c7b98d" Jul 06 16:15:50.429758 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:15:50.429739 2563 scope.go:117] "RemoveContainer" containerID="9e952fd668b23197b8f66ca7ff6f205f004bfad513df8109564f9ca547c7b98d" Jul 06 16:15:50.430006 ip-10-0-129-154 kubenswrapper[2563]: E0706 16:15:50.429987 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9e952fd668b23197b8f66ca7ff6f205f004bfad513df8109564f9ca547c7b98d\": container with ID starting with 9e952fd668b23197b8f66ca7ff6f205f004bfad513df8109564f9ca547c7b98d not found: ID does not exist" containerID="9e952fd668b23197b8f66ca7ff6f205f004bfad513df8109564f9ca547c7b98d" Jul 06 16:15:50.430051 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:15:50.430015 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9e952fd668b23197b8f66ca7ff6f205f004bfad513df8109564f9ca547c7b98d"} err="failed to get container status \"9e952fd668b23197b8f66ca7ff6f205f004bfad513df8109564f9ca547c7b98d\": rpc error: code = NotFound desc = could not find container \"9e952fd668b23197b8f66ca7ff6f205f004bfad513df8109564f9ca547c7b98d\": container with ID starting with 9e952fd668b23197b8f66ca7ff6f205f004bfad513df8109564f9ca547c7b98d not found: ID does not exist" Jul 06 16:15:50.442193 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:15:50.442167 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-c8a9d1/sub-test-model-c8a9d1-kserve-6fd688d795-5ptvv"] Jul 06 16:15:50.448967 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:15:50.448944 2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-c8a9d1/sub-test-model-c8a9d1-kserve-6fd688d795-5ptvv"] Jul 06 16:15:51.945689 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:15:51.945655 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="15378c4d-6744-47bf-b92e-a8558a0fc50f" path="/var/lib/kubelet/pods/15378c4d-6744-47bf-b92e-a8558a0fc50f/volumes" Jul 06 16:17:14.906658 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:17:14.906629 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-key-cleanup-e2e-shared-a-c8a9d1-29722575-kz66l"] Jul 06 16:17:14.911871 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:17:14.911848 2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-key-cleanup-e2e-shared-a-c8a9d1-29722575-kz66l"] Jul 06 16:17:15.945663 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:17:15.945631 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="81ab3838-90d4-4d95-9711-62cde5ce0114" path="/var/lib/kubelet/pods/81ab3838-90d4-4d95-9711-62cde5ce0114/volumes" Jul 06 16:17:59.296216 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:17:59.296181 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-key-cleanup-e2e-shared-b-b747bf-29722575-6v8pk"] Jul 06 16:17:59.300316 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:17:59.300288 2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-key-cleanup-e2e-shared-b-b747bf-29722575-6v8pk"] Jul 06 16:17:59.946212 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:17:59.946178 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="561fea79-1ccf-42e6-88a6-8bdfadafffed" path="/var/lib/kubelet/pods/561fea79-1ccf-42e6-88a6-8bdfadafffed/volumes" Jul 06 16:18:07.489505 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:18:07.489470 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-5497d684cc-fpvql_3d9bb9b2-66b1-4a36-aa3d-e48975c8f61d/authorino/0.log" Jul 06 16:18:12.695407 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:18:12.695374 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-key-cleanup-29722560-85hs7_abae7424-7574-4b51-9b0f-4020b0370e15/cleanup/0.log" Jul 06 16:18:12.809147 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:18:12.809115 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-key-cleanup-29722575-nsfs2_d9577510-00ce-444f-8ecb-78f6b6c71b3e/cleanup/0.log" Jul 06 16:18:13.057880 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:18:13.057777 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-s2j2x_cd0760e4-311a-4634-9392-f904d4ca5499/server/0.log" Jul 06 16:18:13.578225 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:18:13.578196 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-749df7788b-fdjpk_d44bfd0f-f64b-4f20-b0bd-d3ac355633aa/copy-manifests/0.log" Jul 06 16:18:13.591343 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:18:13.591309 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-749df7788b-fdjpk_d44bfd0f-f64b-4f20-b0bd-d3ac355633aa/manager/0.log" Jul 06 16:18:14.541116 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:18:14.541090 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759vdg26_7d1719d6-6fb3-4557-a9d5-a384fcf5786e/util/0.log" Jul 06 16:18:14.552266 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:18:14.552234 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759vdg26_7d1719d6-6fb3-4557-a9d5-a384fcf5786e/pull/0.log" Jul 06 16:18:14.564262 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:18:14.564244 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759vdg26_7d1719d6-6fb3-4557-a9d5-a384fcf5786e/extract/0.log" Jul 06 16:18:14.682791 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:18:14.682759 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0bc7cp_f38f065d-1c16-4410-9a3b-801bc408a356/util/0.log" Jul 06 16:18:14.693587 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:18:14.693530 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0bc7cp_f38f065d-1c16-4410-9a3b-801bc408a356/pull/0.log" Jul 06 16:18:14.705069 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:18:14.705045 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0bc7cp_f38f065d-1c16-4410-9a3b-801bc408a356/extract/0.log" Jul 06 16:18:14.823911 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:18:14.823795 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73fsrrm_92d77489-516a-4f5a-9b29-72ff30ab12c1/pull/0.log" Jul 06 16:18:14.830990 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:18:14.830968 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73fsrrm_92d77489-516a-4f5a-9b29-72ff30ab12c1/extract/0.log" Jul 06 16:18:14.838994 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:18:14.838963 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73fsrrm_92d77489-516a-4f5a-9b29-72ff30ab12c1/util/0.log" Jul 06 16:18:14.953718 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:18:14.953692 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qgpb2_605191eb-5b8b-4737-a33d-685711524363/util/0.log" Jul 06 16:18:14.960171 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:18:14.960151 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qgpb2_605191eb-5b8b-4737-a33d-685711524363/pull/0.log" Jul 06 16:18:14.966731 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:18:14.966713 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qgpb2_605191eb-5b8b-4737-a33d-685711524363/extract/0.log" Jul 06 16:18:15.084071 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:18:15.083969 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-5497d684cc-fpvql_3d9bb9b2-66b1-4a36-aa3d-e48975c8f61d/authorino/0.log" Jul 06 16:18:16.470727 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:18:16.470695 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-745c454698-6f9d2_a801e02a-0014-4850-bbb6-44986a875432/kube-auth-proxy/0.log" Jul 06 16:18:17.066670 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:18:17.066642 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-5855858db9-884bq_201d3840-f397-4577-b8ba-03b31295ed3f/router/0.log" Jul 06 16:18:17.570396 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:18:17.570365 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-7sgfq_d4c96862-d4b8-4e69-a536-b13867b7f2bf/main/0.log" Jul 06 16:18:17.680778 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:18:17.680754 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-nx96r_6926bf8b-8350-4ce6-aab1-92f7ecf6e970/main/0.log" Jul 06 16:18:22.165294 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:18:22.165261 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-z6r42/must-gather-x5m5l"] Jul 06 16:18:22.165849 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:18:22.165832 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="561fea79-1ccf-42e6-88a6-8bdfadafffed" containerName="cleanup" Jul 06 16:18:22.165892 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:18:22.165853 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="561fea79-1ccf-42e6-88a6-8bdfadafffed" containerName="cleanup" Jul 06 16:18:22.165892 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:18:22.165886 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d9577510-00ce-444f-8ecb-78f6b6c71b3e" containerName="cleanup" Jul 06 16:18:22.165956 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:18:22.165895 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="d9577510-00ce-444f-8ecb-78f6b6c71b3e" containerName="cleanup" Jul 06 16:18:22.165956 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:18:22.165908 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="81ab3838-90d4-4d95-9711-62cde5ce0114" containerName="cleanup" Jul 06 16:18:22.165956 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:18:22.165917 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="81ab3838-90d4-4d95-9711-62cde5ce0114" containerName="cleanup" Jul 06 16:18:22.165956 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:18:22.165929 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="15378c4d-6744-47bf-b92e-a8558a0fc50f" containerName="main" Jul 06 16:18:22.165956 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:18:22.165937 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="15378c4d-6744-47bf-b92e-a8558a0fc50f" containerName="main" Jul 06 16:18:22.166102 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:18:22.166038 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="15378c4d-6744-47bf-b92e-a8558a0fc50f" containerName="main" Jul 06 16:18:22.166102 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:18:22.166055 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="d9577510-00ce-444f-8ecb-78f6b6c71b3e" containerName="cleanup" Jul 06 16:18:22.166102 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:18:22.166065 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="81ab3838-90d4-4d95-9711-62cde5ce0114" containerName="cleanup" Jul 06 16:18:22.166102 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:18:22.166074 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="561fea79-1ccf-42e6-88a6-8bdfadafffed" containerName="cleanup" Jul 06 16:18:22.169650 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:18:22.169625 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-z6r42/must-gather-x5m5l" Jul 06 16:18:22.172559 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:18:22.172535 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-z6r42\"/\"kube-root-ca.crt\"" Jul 06 16:18:22.173638 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:18:22.173615 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-z6r42\"/\"default-dockercfg-mwdkp\"" Jul 06 16:18:22.173780 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:18:22.173654 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-z6r42\"/\"openshift-service-ca.crt\"" Jul 06 16:18:22.176705 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:18:22.176677 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-z6r42/must-gather-x5m5l"] Jul 06 16:18:22.262774 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:18:22.262741 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wplp4\" (UniqueName: \"kubernetes.io/projected/2dccdd0f-3cc7-4a1f-ae9f-a84886bb9396-kube-api-access-wplp4\") pod \"must-gather-x5m5l\" (UID: \"2dccdd0f-3cc7-4a1f-ae9f-a84886bb9396\") " pod="openshift-must-gather-z6r42/must-gather-x5m5l" Jul 06 16:18:22.262946 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:18:22.262844 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/2dccdd0f-3cc7-4a1f-ae9f-a84886bb9396-must-gather-output\") pod \"must-gather-x5m5l\" (UID: \"2dccdd0f-3cc7-4a1f-ae9f-a84886bb9396\") " pod="openshift-must-gather-z6r42/must-gather-x5m5l" Jul 06 16:18:22.363843 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:18:22.363811 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/2dccdd0f-3cc7-4a1f-ae9f-a84886bb9396-must-gather-output\") pod \"must-gather-x5m5l\" (UID: \"2dccdd0f-3cc7-4a1f-ae9f-a84886bb9396\") " pod="openshift-must-gather-z6r42/must-gather-x5m5l" Jul 06 16:18:22.364006 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:18:22.363854 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wplp4\" (UniqueName: \"kubernetes.io/projected/2dccdd0f-3cc7-4a1f-ae9f-a84886bb9396-kube-api-access-wplp4\") pod \"must-gather-x5m5l\" (UID: \"2dccdd0f-3cc7-4a1f-ae9f-a84886bb9396\") " pod="openshift-must-gather-z6r42/must-gather-x5m5l" Jul 06 16:18:22.364149 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:18:22.364121 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/2dccdd0f-3cc7-4a1f-ae9f-a84886bb9396-must-gather-output\") pod \"must-gather-x5m5l\" (UID: \"2dccdd0f-3cc7-4a1f-ae9f-a84886bb9396\") " pod="openshift-must-gather-z6r42/must-gather-x5m5l" Jul 06 16:18:22.374854 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:18:22.374824 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wplp4\" (UniqueName: \"kubernetes.io/projected/2dccdd0f-3cc7-4a1f-ae9f-a84886bb9396-kube-api-access-wplp4\") pod \"must-gather-x5m5l\" (UID: \"2dccdd0f-3cc7-4a1f-ae9f-a84886bb9396\") " pod="openshift-must-gather-z6r42/must-gather-x5m5l" Jul 06 16:18:22.481290 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:18:22.481194 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-z6r42/must-gather-x5m5l" Jul 06 16:18:22.628857 ip-10-0-129-154 kubenswrapper[2563]: W0706 16:18:22.628827 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2dccdd0f_3cc7_4a1f_ae9f_a84886bb9396.slice/crio-24ceafd8e0317a34bf64376ddeee848fda5281709a82c3c1c80efe5044d45002 WatchSource:0}: Error finding container 24ceafd8e0317a34bf64376ddeee848fda5281709a82c3c1c80efe5044d45002: Status 404 returned error can't find the container with id 24ceafd8e0317a34bf64376ddeee848fda5281709a82c3c1c80efe5044d45002 Jul 06 16:18:22.628996 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:18:22.628930 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-z6r42/must-gather-x5m5l"] Jul 06 16:18:22.630834 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:18:22.630813 2563 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 06 16:18:23.290165 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:18:23.290124 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-z6r42/must-gather-x5m5l" event={"ID":"2dccdd0f-3cc7-4a1f-ae9f-a84886bb9396","Type":"ContainerStarted","Data":"24ceafd8e0317a34bf64376ddeee848fda5281709a82c3c1c80efe5044d45002"} Jul 06 16:18:25.309707 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:18:25.309661 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-z6r42/must-gather-x5m5l" event={"ID":"2dccdd0f-3cc7-4a1f-ae9f-a84886bb9396","Type":"ContainerStarted","Data":"67c08d58daab8b1fdeec6ff4318cc8437a25ce18231c523fe067bfa3f5c513d5"} Jul 06 16:18:25.309707 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:18:25.309713 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-z6r42/must-gather-x5m5l" event={"ID":"2dccdd0f-3cc7-4a1f-ae9f-a84886bb9396","Type":"ContainerStarted","Data":"b5a6de727ec48a99a032852bfdec6a6c0ff652274cee942c797ab4cfa60be15c"} Jul 06 16:18:25.337245 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:18:25.337172 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-z6r42/must-gather-x5m5l" podStartSLOduration=1.6220860240000001 podStartE2EDuration="3.337151112s" totalImagesPullingTime="1.715065088s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 16:18:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 16:18:25.334087044 +0000 UTC m=+2681.843320223" watchObservedRunningTime="2026-07-06 16:18:25.337151112 +0000 UTC m=+2681.846384291" Jul 06 16:18:26.140625 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:18:26.140593 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-w9qbt_bce194c9-b363-4021-bb8a-c7cc65facc39/global-pull-secret-syncer/0.log" Jul 06 16:18:26.271044 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:18:26.271011 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-hq8kh_e67f3ab6-afb7-47ff-8219-adc98fe79e74/konnectivity-agent/0.log" Jul 06 16:18:26.298119 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:18:26.298080 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-129-154.ec2.internal_89a933a4336ad4eeb3842cbd7d2c9e02/haproxy/0.log" Jul 06 16:18:30.956226 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:18:30.956153 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759vdg26_7d1719d6-6fb3-4557-a9d5-a384fcf5786e/extract/0.log" Jul 06 16:18:30.979691 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:18:30.979604 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759vdg26_7d1719d6-6fb3-4557-a9d5-a384fcf5786e/util/0.log" Jul 06 16:18:31.003559 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:18:31.003532 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759vdg26_7d1719d6-6fb3-4557-a9d5-a384fcf5786e/pull/0.log" Jul 06 16:18:31.034100 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:18:31.034055 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0bc7cp_f38f065d-1c16-4410-9a3b-801bc408a356/extract/0.log" Jul 06 16:18:31.057134 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:18:31.057086 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0bc7cp_f38f065d-1c16-4410-9a3b-801bc408a356/util/0.log" Jul 06 16:18:31.082359 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:18:31.082330 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0bc7cp_f38f065d-1c16-4410-9a3b-801bc408a356/pull/0.log" Jul 06 16:18:31.117335 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:18:31.117301 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73fsrrm_92d77489-516a-4f5a-9b29-72ff30ab12c1/extract/0.log" Jul 06 16:18:31.142143 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:18:31.142118 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73fsrrm_92d77489-516a-4f5a-9b29-72ff30ab12c1/util/0.log" Jul 06 16:18:31.168177 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:18:31.168148 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73fsrrm_92d77489-516a-4f5a-9b29-72ff30ab12c1/pull/0.log" Jul 06 16:18:31.197897 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:18:31.197841 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qgpb2_605191eb-5b8b-4737-a33d-685711524363/extract/0.log" Jul 06 16:18:31.222552 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:18:31.222478 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qgpb2_605191eb-5b8b-4737-a33d-685711524363/util/0.log" Jul 06 16:18:31.251881 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:18:31.251848 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qgpb2_605191eb-5b8b-4737-a33d-685711524363/pull/0.log" Jul 06 16:18:31.626489 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:18:31.623691 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-5497d684cc-fpvql_3d9bb9b2-66b1-4a36-aa3d-e48975c8f61d/authorino/0.log" Jul 06 16:18:33.993520 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:18:33.993486 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-64786d9cbb-trjfs_bf5c30e2-bf88-468b-adf2-2e8f73f136b4/cluster-monitoring-operator/1.log" Jul 06 16:18:34.088441 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:18:34.087974 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-64786d9cbb-trjfs_bf5c30e2-bf88-468b-adf2-2e8f73f136b4/cluster-monitoring-operator/0.log" Jul 06 16:18:34.213968 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:18:34.213910 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-8465bdc94b-6kxjv_82494c64-e789-4829-8017-c44bd404e188/metrics-server/0.log" Jul 06 16:18:34.551030 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:18:34.551001 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-sjxxv_c55d32ae-8d34-4ca2-8d07-a3a374ecf12d/node-exporter/0.log" Jul 06 16:18:34.594038 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:18:34.594009 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-sjxxv_c55d32ae-8d34-4ca2-8d07-a3a374ecf12d/kube-rbac-proxy/0.log" Jul 06 16:18:34.651206 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:18:34.651174 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-sjxxv_c55d32ae-8d34-4ca2-8d07-a3a374ecf12d/init-textfile/0.log" Jul 06 16:18:35.084432 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:18:35.084377 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-567f7b8b7c-b7s4w_eeb3e868-088a-4b4d-80a1-143a15ad91c8/prometheus-operator/0.log" Jul 06 16:18:35.118647 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:18:35.118618 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-567f7b8b7c-b7s4w_eeb3e868-088a-4b4d-80a1-143a15ad91c8/kube-rbac-proxy/0.log" Jul 06 16:18:35.156301 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:18:35.156269 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-d559d66d5-2bbvr_e5f102a2-585e-4fc2-9579-2c221f542f83/prometheus-operator-admission-webhook/0.log" Jul 06 16:18:35.380377 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:18:35.380298 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-54dc4c49bb-mj926_206bbf71-dc24-4a17-bae2-8d17e7488a47/thanos-query/0.log" Jul 06 16:18:35.415994 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:18:35.415960 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-54dc4c49bb-mj926_206bbf71-dc24-4a17-bae2-8d17e7488a47/kube-rbac-proxy-web/0.log" Jul 06 16:18:35.481768 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:18:35.481738 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-54dc4c49bb-mj926_206bbf71-dc24-4a17-bae2-8d17e7488a47/kube-rbac-proxy/0.log" Jul 06 16:18:35.519278 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:18:35.519248 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-54dc4c49bb-mj926_206bbf71-dc24-4a17-bae2-8d17e7488a47/prom-label-proxy/0.log" Jul 06 16:18:35.550844 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:18:35.550809 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-54dc4c49bb-mj926_206bbf71-dc24-4a17-bae2-8d17e7488a47/kube-rbac-proxy-rules/0.log" Jul 06 16:18:35.611636 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:18:35.611601 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-54dc4c49bb-mj926_206bbf71-dc24-4a17-bae2-8d17e7488a47/kube-rbac-proxy-metrics/0.log" Jul 06 16:18:36.405870 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:18:36.405832 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-z6r42/perf-node-gather-daemonset-jhmv6"] Jul 06 16:18:36.414870 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:18:36.414838 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-z6r42/perf-node-gather-daemonset-jhmv6" Jul 06 16:18:36.435539 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:18:36.435504 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-z6r42/perf-node-gather-daemonset-jhmv6"] Jul 06 16:18:36.514242 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:18:36.513911 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/36f8c262-be9d-44c6-a279-4e8911e9ea2a-proc\") pod \"perf-node-gather-daemonset-jhmv6\" (UID: \"36f8c262-be9d-44c6-a279-4e8911e9ea2a\") " pod="openshift-must-gather-z6r42/perf-node-gather-daemonset-jhmv6" Jul 06 16:18:36.514242 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:18:36.513958 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/36f8c262-be9d-44c6-a279-4e8911e9ea2a-lib-modules\") pod \"perf-node-gather-daemonset-jhmv6\" (UID: \"36f8c262-be9d-44c6-a279-4e8911e9ea2a\") " pod="openshift-must-gather-z6r42/perf-node-gather-daemonset-jhmv6" Jul 06 16:18:36.514242 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:18:36.514053 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/36f8c262-be9d-44c6-a279-4e8911e9ea2a-sys\") pod \"perf-node-gather-daemonset-jhmv6\" (UID: \"36f8c262-be9d-44c6-a279-4e8911e9ea2a\") " pod="openshift-must-gather-z6r42/perf-node-gather-daemonset-jhmv6" Jul 06 16:18:36.514242 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:18:36.514090 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8zv79\" (UniqueName: \"kubernetes.io/projected/36f8c262-be9d-44c6-a279-4e8911e9ea2a-kube-api-access-8zv79\") pod \"perf-node-gather-daemonset-jhmv6\" (UID: \"36f8c262-be9d-44c6-a279-4e8911e9ea2a\") " pod="openshift-must-gather-z6r42/perf-node-gather-daemonset-jhmv6" Jul 06 16:18:36.514242 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:18:36.514163 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/36f8c262-be9d-44c6-a279-4e8911e9ea2a-podres\") pod \"perf-node-gather-daemonset-jhmv6\" (UID: \"36f8c262-be9d-44c6-a279-4e8911e9ea2a\") " pod="openshift-must-gather-z6r42/perf-node-gather-daemonset-jhmv6" Jul 06 16:18:36.615700 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:18:36.615659 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/36f8c262-be9d-44c6-a279-4e8911e9ea2a-sys\") pod \"perf-node-gather-daemonset-jhmv6\" (UID: \"36f8c262-be9d-44c6-a279-4e8911e9ea2a\") " pod="openshift-must-gather-z6r42/perf-node-gather-daemonset-jhmv6" Jul 06 16:18:36.615880 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:18:36.615718 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8zv79\" (UniqueName: \"kubernetes.io/projected/36f8c262-be9d-44c6-a279-4e8911e9ea2a-kube-api-access-8zv79\") pod \"perf-node-gather-daemonset-jhmv6\" (UID: \"36f8c262-be9d-44c6-a279-4e8911e9ea2a\") " pod="openshift-must-gather-z6r42/perf-node-gather-daemonset-jhmv6" Jul 06 16:18:36.615880 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:18:36.615756 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/36f8c262-be9d-44c6-a279-4e8911e9ea2a-podres\") pod \"perf-node-gather-daemonset-jhmv6\" (UID: \"36f8c262-be9d-44c6-a279-4e8911e9ea2a\") " pod="openshift-must-gather-z6r42/perf-node-gather-daemonset-jhmv6" Jul 06 16:18:36.615880 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:18:36.615789 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/36f8c262-be9d-44c6-a279-4e8911e9ea2a-sys\") pod \"perf-node-gather-daemonset-jhmv6\" (UID: \"36f8c262-be9d-44c6-a279-4e8911e9ea2a\") " pod="openshift-must-gather-z6r42/perf-node-gather-daemonset-jhmv6" Jul 06 16:18:36.615880 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:18:36.615827 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/36f8c262-be9d-44c6-a279-4e8911e9ea2a-proc\") pod \"perf-node-gather-daemonset-jhmv6\" (UID: \"36f8c262-be9d-44c6-a279-4e8911e9ea2a\") " pod="openshift-must-gather-z6r42/perf-node-gather-daemonset-jhmv6" Jul 06 16:18:36.615880 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:18:36.615851 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/36f8c262-be9d-44c6-a279-4e8911e9ea2a-lib-modules\") pod \"perf-node-gather-daemonset-jhmv6\" (UID: \"36f8c262-be9d-44c6-a279-4e8911e9ea2a\") " pod="openshift-must-gather-z6r42/perf-node-gather-daemonset-jhmv6" Jul 06 16:18:36.616079 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:18:36.615924 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/36f8c262-be9d-44c6-a279-4e8911e9ea2a-podres\") pod \"perf-node-gather-daemonset-jhmv6\" (UID: \"36f8c262-be9d-44c6-a279-4e8911e9ea2a\") " pod="openshift-must-gather-z6r42/perf-node-gather-daemonset-jhmv6" Jul 06 16:18:36.616079 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:18:36.615938 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/36f8c262-be9d-44c6-a279-4e8911e9ea2a-proc\") pod \"perf-node-gather-daemonset-jhmv6\" (UID: \"36f8c262-be9d-44c6-a279-4e8911e9ea2a\") " pod="openshift-must-gather-z6r42/perf-node-gather-daemonset-jhmv6" Jul 06 16:18:36.616079 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:18:36.616023 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/36f8c262-be9d-44c6-a279-4e8911e9ea2a-lib-modules\") pod \"perf-node-gather-daemonset-jhmv6\" (UID: \"36f8c262-be9d-44c6-a279-4e8911e9ea2a\") " pod="openshift-must-gather-z6r42/perf-node-gather-daemonset-jhmv6" Jul 06 16:18:36.631554 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:18:36.631526 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8zv79\" (UniqueName: \"kubernetes.io/projected/36f8c262-be9d-44c6-a279-4e8911e9ea2a-kube-api-access-8zv79\") pod \"perf-node-gather-daemonset-jhmv6\" (UID: \"36f8c262-be9d-44c6-a279-4e8911e9ea2a\") " pod="openshift-must-gather-z6r42/perf-node-gather-daemonset-jhmv6" Jul 06 16:18:36.727873 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:18:36.727783 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-z6r42/perf-node-gather-daemonset-jhmv6" Jul 06 16:18:36.922973 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:18:36.922944 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-6bffdf9778-fgktw_f955a985-08b6-49d9-b159-8ec3916d8b20/networking-console-plugin/0.log" Jul 06 16:18:36.946113 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:18:36.946083 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-z6r42/perf-node-gather-daemonset-jhmv6"] Jul 06 16:18:37.425998 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:18:37.425966 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-z6r42/perf-node-gather-daemonset-jhmv6" event={"ID":"36f8c262-be9d-44c6-a279-4e8911e9ea2a","Type":"ContainerStarted","Data":"8e7d9895c1a045f7f44c684f47a5fbb06c36b5073135ed2bdc78d068c397dc86"} Jul 06 16:18:37.426377 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:18:37.426024 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-z6r42/perf-node-gather-daemonset-jhmv6" event={"ID":"36f8c262-be9d-44c6-a279-4e8911e9ea2a","Type":"ContainerStarted","Data":"9da3972597aa0d42997e598a7d62cd7bfe075f9554d5fefe6c6fdc1e0a607b34"} Jul 06 16:18:37.426377 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:18:37.426056 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-z6r42/perf-node-gather-daemonset-jhmv6" Jul 06 16:18:37.460438 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:18:37.460360 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-z6r42/perf-node-gather-daemonset-jhmv6" podStartSLOduration=1.460336652 podStartE2EDuration="1.460336652s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 16:18:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 16:18:37.457700727 +0000 UTC m=+2693.966933901" watchObservedRunningTime="2026-07-06 16:18:37.460336652 +0000 UTC m=+2693.969569825" Jul 06 16:18:37.665574 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:18:37.665546 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-l47bf_9ad7e968-ed2e-4cbe-9b4b-780bc9d01b88/console-operator/2.log" Jul 06 16:18:37.676212 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:18:37.676126 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-l47bf_9ad7e968-ed2e-4cbe-9b4b-780bc9d01b88/console-operator/3.log" Jul 06 16:18:38.216684 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:18:38.216654 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-5f6bdbfc9-hfwlk_a3d590d1-bcc0-4c81-abb4-8aa13791bec6/console/0.log" Jul 06 16:18:38.906177 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:18:38.906149 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-5d456fdbcd-4grrz_b7c1005a-9a43-4116-94cb-df36eaf220fb/volume-data-source-validator/0.log" Jul 06 16:18:40.213894 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:18:40.213858 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-h86nn_17c20e5e-6515-48d6-8c51-425a9e86485d/dns/0.log" Jul 06 16:18:40.237442 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:18:40.237401 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-h86nn_17c20e5e-6515-48d6-8c51-425a9e86485d/kube-rbac-proxy/0.log" Jul 06 16:18:40.296062 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:18:40.296038 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-pgkmz_a3b0e3f1-74a1-4f15-9770-1a8ac2a02c4f/dns-node-resolver/0.log" Jul 06 16:18:40.969485 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:18:40.969449 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-rmsk7_0ce018e7-3a1f-43dc-8988-5d8054f04c69/node-ca/0.log" Jul 06 16:18:42.218306 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:18:42.218275 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-745c454698-6f9d2_a801e02a-0014-4850-bbb6-44986a875432/kube-auth-proxy/0.log" Jul 06 16:18:42.410887 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:18:42.410853 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-5855858db9-884bq_201d3840-f397-4577-b8ba-03b31295ed3f/router/0.log" Jul 06 16:18:43.211328 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:18:43.211289 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-qltg7_5967a722-7558-4ab7-98f4-b3a5e367bd7a/serve-healthcheck-canary/0.log" Jul 06 16:18:43.443473 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:18:43.443442 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-z6r42/perf-node-gather-daemonset-jhmv6" Jul 06 16:18:43.754382 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:18:43.754350 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5c847dcfc4-nvj9g_0e2e4871-594a-42f5-abb7-867d61453254/insights-operator/0.log" Jul 06 16:18:43.756710 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:18:43.756689 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5c847dcfc4-nvj9g_0e2e4871-594a-42f5-abb7-867d61453254/insights-operator/1.log" Jul 06 16:18:43.880229 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:18:43.880198 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-pbh67_e6b68ed7-afa1-44b1-acf0-50c8addc4c6c/kube-rbac-proxy/0.log" Jul 06 16:18:43.913636 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:18:43.913603 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-pbh67_e6b68ed7-afa1-44b1-acf0-50c8addc4c6c/exporter/0.log" Jul 06 16:18:43.960113 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:18:43.960077 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-pbh67_e6b68ed7-afa1-44b1-acf0-50c8addc4c6c/extractor/0.log" Jul 06 16:18:44.148526 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:18:44.148497 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-64786d9cbb-trjfs_bf5c30e2-bf88-468b-adf2-2e8f73f136b4/cluster-monitoring-operator/0.log" Jul 06 16:18:44.155303 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:18:44.155282 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-l47bf_9ad7e968-ed2e-4cbe-9b4b-780bc9d01b88/console-operator/2.log" Jul 06 16:18:44.167147 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:18:44.167118 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-64786d9cbb-trjfs_bf5c30e2-bf88-468b-adf2-2e8f73f136b4/cluster-monitoring-operator/0.log" Jul 06 16:18:44.174404 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:18:44.174380 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-l47bf_9ad7e968-ed2e-4cbe-9b4b-780bc9d01b88/console-operator/2.log" Jul 06 16:18:47.053007 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:18:47.052923 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-745c454698-6f9d2_a801e02a-0014-4850-bbb6-44986a875432/kube-auth-proxy/0.log" Jul 06 16:18:47.072963 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:18:47.072940 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-key-cleanup-29722560-85hs7_abae7424-7574-4b51-9b0f-4020b0370e15/cleanup/0.log" Jul 06 16:18:47.104120 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:18:47.104094 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-key-cleanup-29722575-nsfs2_d9577510-00ce-444f-8ecb-78f6b6c71b3e/cleanup/0.log" Jul 06 16:18:47.244708 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:18:47.244681 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-s2j2x_cd0760e4-311a-4634-9392-f904d4ca5499/server/0.log" Jul 06 16:18:47.245812 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:18:47.245798 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-5855858db9-884bq_201d3840-f397-4577-b8ba-03b31295ed3f/router/0.log" Jul 06 16:18:47.638291 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:18:47.638262 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-749df7788b-fdjpk_d44bfd0f-f64b-4f20-b0bd-d3ac355633aa/manager/0.log" Jul 06 16:18:47.664095 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:18:47.664067 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-749df7788b-fdjpk_d44bfd0f-f64b-4f20-b0bd-d3ac355633aa/copy-manifests/0.log" Jul 06 16:18:48.714206 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:18:48.714165 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-745c454698-6f9d2_a801e02a-0014-4850-bbb6-44986a875432/kube-auth-proxy/0.log" Jul 06 16:18:48.878312 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:18:48.878235 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-5855858db9-884bq_201d3840-f397-4577-b8ba-03b31295ed3f/router/0.log" Jul 06 16:18:49.235352 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:18:49.235300 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-544dccfdb4-rl2tq_c2d0ef2f-f717-4854-8a6b-1f09b3e0868e/manager/1.log" Jul 06 16:18:49.235846 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:18:49.235798 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-544dccfdb4-rl2tq_c2d0ef2f-f717-4854-8a6b-1f09b3e0868e/manager/0.log" Jul 06 16:18:49.267029 ip-10-0-129-154 kubenswrapper[2563]: I0706 16:18:49.266994 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-qwbnk_728a2166-19f5-42c2-8ced-023e8720b64a/openshift-lws-operator/0.log"