Jul 05 13:13:19.630962 ip-10-0-130-191 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Jul 05 13:13:19.630972 ip-10-0-130-191 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Jul 05 13:13:19.630981 ip-10-0-130-191 systemd[1]: kubelet.service: Failed with result 'resources'. Jul 05 13:13:19.631265 ip-10-0-130-191 systemd[1]: Failed to start Kubernetes Kubelet. Jul 05 13:13:29.723131 ip-10-0-130-191 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found. Jul 05 13:13:29.723147 ip-10-0-130-191 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot 50de0c6fb92841b18352824db008e16b -- Jul 05 13:15:13.885540 ip-10-0-130-191 systemd[1]: Starting Kubernetes Kubelet... Jul 05 13:15:14.248975 ip-10-0-130-191 kubenswrapper[2578]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 05 13:15:14.248975 ip-10-0-130-191 kubenswrapper[2578]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 05 13:15:14.248975 ip-10-0-130-191 kubenswrapper[2578]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 05 13:15:14.248975 ip-10-0-130-191 kubenswrapper[2578]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 05 13:15:14.248975 ip-10-0-130-191 kubenswrapper[2578]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 05 13:15:14.249687 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:14.249041 2578 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 05 13:15:14.254497 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.254481 2578 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 05 13:15:14.254497 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.254497 2578 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 05 13:15:14.254574 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.254501 2578 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 05 13:15:14.254574 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.254504 2578 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 05 13:15:14.254574 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.254508 2578 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 05 13:15:14.254574 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.254511 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 05 13:15:14.254574 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.254513 2578 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 05 13:15:14.254574 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.254516 2578 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 05 13:15:14.254574 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.254519 2578 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 05 13:15:14.254574 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.254522 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 05 13:15:14.254574 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.254524 2578 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 05 13:15:14.254574 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.254527 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 05 13:15:14.254574 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.254529 2578 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 05 13:15:14.254574 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.254532 2578 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 05 13:15:14.254574 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.254534 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 05 13:15:14.254574 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.254537 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 05 13:15:14.254574 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.254539 2578 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 05 13:15:14.254574 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.254542 2578 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 05 13:15:14.254574 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.254546 2578 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 05 13:15:14.254574 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.254550 2578 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 05 13:15:14.254574 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.254553 2578 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 05 13:15:14.255040 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.254556 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 05 13:15:14.255040 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.254559 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 05 13:15:14.255040 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.254561 2578 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 05 13:15:14.255040 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.254564 2578 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 05 13:15:14.255040 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.254566 2578 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 05 13:15:14.255040 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.254568 2578 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 05 13:15:14.255040 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.254571 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 05 13:15:14.255040 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.254576 2578 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 05 13:15:14.255040 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.254582 2578 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 05 13:15:14.255040 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.254586 2578 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 05 13:15:14.255040 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.254589 2578 feature_gate.go:328] unrecognized feature gate: Example2 Jul 05 13:15:14.255040 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.254591 2578 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 05 13:15:14.255040 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.254594 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 05 13:15:14.255040 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.254597 2578 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 05 13:15:14.255040 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.254600 2578 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 05 13:15:14.255040 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.254603 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 05 13:15:14.255040 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.254606 2578 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 05 13:15:14.255040 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.254608 2578 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 05 13:15:14.255040 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.254611 2578 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 05 13:15:14.255492 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.254614 2578 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 05 13:15:14.255492 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.254616 2578 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 05 13:15:14.255492 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.254618 2578 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 05 13:15:14.255492 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.254621 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 05 13:15:14.255492 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.254623 2578 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 05 13:15:14.255492 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.254625 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 05 13:15:14.255492 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.254628 2578 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 05 13:15:14.255492 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.254630 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 05 13:15:14.255492 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.254633 2578 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 05 13:15:14.255492 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.254635 2578 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 05 13:15:14.255492 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.254637 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 05 13:15:14.255492 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.254640 2578 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 05 13:15:14.255492 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.254642 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 05 13:15:14.255492 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.254644 2578 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 05 13:15:14.255492 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.254647 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 05 13:15:14.255492 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.254650 2578 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 05 13:15:14.255492 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.254652 2578 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 05 13:15:14.255492 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.254657 2578 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 05 13:15:14.255492 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.254659 2578 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 05 13:15:14.255492 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.254662 2578 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 05 13:15:14.255965 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.254664 2578 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 05 13:15:14.255965 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.254667 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 05 13:15:14.255965 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.254669 2578 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 05 13:15:14.255965 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.254672 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 05 13:15:14.255965 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.254674 2578 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 05 13:15:14.255965 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.254676 2578 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 05 13:15:14.255965 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.254678 2578 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 05 13:15:14.255965 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.254681 2578 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 05 13:15:14.255965 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.254683 2578 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 05 13:15:14.255965 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.254685 2578 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 05 13:15:14.255965 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.254688 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 05 13:15:14.255965 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.254690 2578 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 05 13:15:14.255965 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.254692 2578 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 05 13:15:14.255965 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.254695 2578 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 05 13:15:14.255965 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.254697 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 05 13:15:14.255965 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.254700 2578 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 05 13:15:14.255965 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.254702 2578 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 05 13:15:14.255965 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.254704 2578 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 05 13:15:14.255965 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.254707 2578 feature_gate.go:328] unrecognized feature gate: Example Jul 05 13:15:14.255965 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.254709 2578 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 05 13:15:14.256439 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.254713 2578 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 05 13:15:14.256439 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.254715 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 05 13:15:14.256439 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.254717 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 05 13:15:14.256439 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.254719 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 05 13:15:14.256439 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.254722 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 05 13:15:14.256439 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.254724 2578 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 05 13:15:14.256439 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.254727 2578 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 05 13:15:14.256439 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.254729 2578 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 05 13:15:14.256439 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.254732 2578 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 05 13:15:14.256439 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.254734 2578 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 05 13:15:14.256439 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.254736 2578 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 05 13:15:14.256439 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.254739 2578 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 05 13:15:14.256439 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.254741 2578 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 05 13:15:14.256439 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.254743 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 05 13:15:14.256439 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.254745 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 05 13:15:14.256439 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.254748 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 05 13:15:14.256439 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.254750 2578 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 05 13:15:14.256439 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.254753 2578 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 05 13:15:14.256439 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.254755 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 05 13:15:14.256884 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.255447 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 05 13:15:14.256884 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.255454 2578 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 05 13:15:14.256884 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.255457 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 05 13:15:14.256884 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.255459 2578 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 05 13:15:14.256884 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.255462 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 05 13:15:14.256884 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.255465 2578 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 05 13:15:14.256884 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.255467 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 05 13:15:14.256884 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.255470 2578 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 05 13:15:14.256884 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.255472 2578 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 05 13:15:14.256884 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.255475 2578 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 05 13:15:14.256884 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.255478 2578 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 05 13:15:14.256884 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.255481 2578 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 05 13:15:14.256884 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.255483 2578 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 05 13:15:14.256884 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.255486 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 05 13:15:14.256884 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.255490 2578 feature_gate.go:328] unrecognized feature gate: Example Jul 05 13:15:14.256884 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.255493 2578 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 05 13:15:14.256884 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.255495 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 05 13:15:14.256884 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.255498 2578 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 05 13:15:14.256884 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.255502 2578 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 05 13:15:14.256884 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.255507 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 05 13:15:14.257386 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.255510 2578 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 05 13:15:14.257386 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.255513 2578 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 05 13:15:14.257386 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.255516 2578 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 05 13:15:14.257386 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.255519 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 05 13:15:14.257386 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.255522 2578 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 05 13:15:14.257386 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.255524 2578 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 05 13:15:14.257386 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.255527 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 05 13:15:14.257386 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.255529 2578 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 05 13:15:14.257386 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.255532 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 05 13:15:14.257386 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.255534 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 05 13:15:14.257386 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.255537 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 05 13:15:14.257386 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.255539 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 05 13:15:14.257386 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.255541 2578 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 05 13:15:14.257386 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.255544 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 05 13:15:14.257386 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.255546 2578 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 05 13:15:14.257386 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.255549 2578 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 05 13:15:14.257386 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.255552 2578 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 05 13:15:14.257386 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.255556 2578 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 05 13:15:14.257386 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.255559 2578 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 05 13:15:14.257836 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.255561 2578 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 05 13:15:14.257836 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.255564 2578 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 05 13:15:14.257836 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.255567 2578 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 05 13:15:14.257836 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.255569 2578 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 05 13:15:14.257836 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.255572 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 05 13:15:14.257836 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.255574 2578 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 05 13:15:14.257836 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.255576 2578 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 05 13:15:14.257836 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.255579 2578 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 05 13:15:14.257836 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.255582 2578 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 05 13:15:14.257836 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.255584 2578 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 05 13:15:14.257836 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.255587 2578 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 05 13:15:14.257836 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.255589 2578 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 05 13:15:14.257836 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.255592 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 05 13:15:14.257836 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.255595 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 05 13:15:14.257836 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.255598 2578 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 05 13:15:14.257836 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.255602 2578 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 05 13:15:14.257836 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.255605 2578 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 05 13:15:14.257836 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.255607 2578 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 05 13:15:14.257836 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.255611 2578 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 05 13:15:14.258329 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.255614 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 05 13:15:14.258329 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.255616 2578 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 05 13:15:14.258329 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.255619 2578 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 05 13:15:14.258329 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.255621 2578 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 05 13:15:14.258329 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.255624 2578 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 05 13:15:14.258329 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.255626 2578 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 05 13:15:14.258329 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.255628 2578 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 05 13:15:14.258329 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.255631 2578 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 05 13:15:14.258329 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.255633 2578 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 05 13:15:14.258329 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.255636 2578 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 05 13:15:14.258329 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.255638 2578 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 05 13:15:14.258329 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.255641 2578 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 05 13:15:14.258329 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.255644 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 05 13:15:14.258329 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.255646 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 05 13:15:14.258329 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.255648 2578 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 05 13:15:14.258329 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.255650 2578 feature_gate.go:328] unrecognized feature gate: Example2 Jul 05 13:15:14.258329 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.255653 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 05 13:15:14.258329 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.255656 2578 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 05 13:15:14.258329 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.255659 2578 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 05 13:15:14.258329 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.255661 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 05 13:15:14.258805 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.255663 2578 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 05 13:15:14.258805 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.255666 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 05 13:15:14.258805 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.255669 2578 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 05 13:15:14.258805 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.255671 2578 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 05 13:15:14.258805 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.255673 2578 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 05 13:15:14.258805 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.255675 2578 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 05 13:15:14.258805 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.255678 2578 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 05 13:15:14.258805 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.255680 2578 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 05 13:15:14.258805 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.255682 2578 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 05 13:15:14.258805 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.255684 2578 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 05 13:15:14.258805 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.255687 2578 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 05 13:15:14.258805 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.255689 2578 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 05 13:15:14.258805 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.255691 2578 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 05 13:15:14.258805 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.255693 2578 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 05 13:15:14.258805 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.255696 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 05 13:15:14.258805 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.255698 2578 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 05 13:15:14.258805 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.255700 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 05 13:15:14.258805 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.255702 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 05 13:15:14.258805 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.255705 2578 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 05 13:15:14.258805 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.255707 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 05 13:15:14.259294 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.255710 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 05 13:15:14.259294 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.256094 2578 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 05 13:15:14.259294 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.256099 2578 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 05 13:15:14.259294 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.256102 2578 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 05 13:15:14.259294 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.256104 2578 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 05 13:15:14.259294 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.256107 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 05 13:15:14.259294 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.256110 2578 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 05 13:15:14.259294 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.256113 2578 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 05 13:15:14.259294 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.256116 2578 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 05 13:15:14.259294 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.256118 2578 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 05 13:15:14.259294 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.256121 2578 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 05 13:15:14.259294 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.256124 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 05 13:15:14.259294 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.256126 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 05 13:15:14.259294 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.256128 2578 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 05 13:15:14.259294 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.256132 2578 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 05 13:15:14.259294 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.256134 2578 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 05 13:15:14.259294 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.256136 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 05 13:15:14.259294 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.256139 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 05 13:15:14.259294 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.256141 2578 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 05 13:15:14.259294 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.256144 2578 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 05 13:15:14.259767 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.256146 2578 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 05 13:15:14.259767 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.256148 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 05 13:15:14.259767 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.256151 2578 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 05 13:15:14.259767 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.256154 2578 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 05 13:15:14.259767 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.256156 2578 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 05 13:15:14.259767 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.256158 2578 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 05 13:15:14.259767 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.256161 2578 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 05 13:15:14.259767 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.256163 2578 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 05 13:15:14.259767 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.256165 2578 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 05 13:15:14.259767 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.256169 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 05 13:15:14.259767 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.256171 2578 feature_gate.go:328] unrecognized feature gate: Example2 Jul 05 13:15:14.259767 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.256174 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 05 13:15:14.259767 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.256178 2578 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 05 13:15:14.259767 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.256182 2578 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 05 13:15:14.259767 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.256185 2578 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 05 13:15:14.259767 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.256188 2578 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 05 13:15:14.259767 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.256190 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 05 13:15:14.259767 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.256193 2578 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 05 13:15:14.259767 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.256195 2578 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 05 13:15:14.260347 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.256199 2578 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 05 13:15:14.260347 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.256202 2578 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 05 13:15:14.260347 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.256204 2578 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 05 13:15:14.260347 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.256207 2578 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 05 13:15:14.260347 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.256209 2578 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 05 13:15:14.260347 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.256211 2578 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 05 13:15:14.260347 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.256214 2578 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 05 13:15:14.260347 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.256216 2578 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 05 13:15:14.260347 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.256222 2578 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 05 13:15:14.260347 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.256225 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 05 13:15:14.260347 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.256227 2578 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 05 13:15:14.260347 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.256229 2578 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 05 13:15:14.260347 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.256232 2578 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 05 13:15:14.260347 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.256234 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 05 13:15:14.260347 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.256236 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 05 13:15:14.260347 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.256239 2578 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 05 13:15:14.260347 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.256241 2578 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 05 13:15:14.260347 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.256243 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 05 13:15:14.260347 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.256246 2578 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 05 13:15:14.260347 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.256248 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 05 13:15:14.260848 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.256250 2578 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 05 13:15:14.260848 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.256253 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 05 13:15:14.260848 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.256256 2578 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 05 13:15:14.260848 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.256258 2578 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 05 13:15:14.260848 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.256261 2578 feature_gate.go:328] unrecognized feature gate: Example Jul 05 13:15:14.260848 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.256264 2578 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 05 13:15:14.260848 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.256267 2578 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 05 13:15:14.260848 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.256269 2578 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 05 13:15:14.260848 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.256272 2578 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 05 13:15:14.260848 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.256274 2578 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 05 13:15:14.260848 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.256292 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 05 13:15:14.260848 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.256294 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 05 13:15:14.260848 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.256297 2578 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 05 13:15:14.260848 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.256299 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 05 13:15:14.260848 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.256302 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 05 13:15:14.260848 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.256304 2578 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 05 13:15:14.260848 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.256307 2578 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 05 13:15:14.260848 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.256309 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 05 13:15:14.260848 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.256312 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 05 13:15:14.260848 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.256314 2578 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 05 13:15:14.261347 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.256316 2578 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 05 13:15:14.261347 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.256325 2578 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 05 13:15:14.261347 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.256328 2578 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 05 13:15:14.261347 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.256330 2578 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 05 13:15:14.261347 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.256332 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 05 13:15:14.261347 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.256335 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 05 13:15:14.261347 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.256337 2578 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 05 13:15:14.261347 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.256340 2578 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 05 13:15:14.261347 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.256342 2578 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 05 13:15:14.261347 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.256344 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 05 13:15:14.261347 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.256346 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 05 13:15:14.261347 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.256349 2578 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 05 13:15:14.261347 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.256351 2578 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 05 13:15:14.261347 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.256355 2578 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 05 13:15:14.261347 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.256358 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 05 13:15:14.261347 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.256361 2578 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 05 13:15:14.261347 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.256364 2578 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 05 13:15:14.261347 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.256367 2578 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 05 13:15:14.261347 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.256372 2578 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 05 13:15:14.261347 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.256374 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 05 13:15:14.261840 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.256377 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 05 13:15:14.261840 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:14.256475 2578 flags.go:64] FLAG: --address="0.0.0.0" Jul 05 13:15:14.261840 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:14.256483 2578 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 05 13:15:14.261840 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:14.256491 2578 flags.go:64] FLAG: --anonymous-auth="true" Jul 05 13:15:14.261840 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:14.256495 2578 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 05 13:15:14.261840 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:14.256499 2578 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 05 13:15:14.261840 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:14.256502 2578 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 05 13:15:14.261840 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:14.256506 2578 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 05 13:15:14.261840 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:14.256510 2578 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 05 13:15:14.261840 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:14.256513 2578 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 05 13:15:14.261840 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:14.256516 2578 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 05 13:15:14.261840 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:14.256519 2578 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 05 13:15:14.261840 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:14.256521 2578 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 05 13:15:14.261840 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:14.256524 2578 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 05 13:15:14.261840 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:14.256528 2578 flags.go:64] FLAG: --cgroup-root="" Jul 05 13:15:14.261840 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:14.256531 2578 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 05 13:15:14.261840 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:14.256534 2578 flags.go:64] FLAG: --client-ca-file="" Jul 05 13:15:14.261840 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:14.256536 2578 flags.go:64] FLAG: --cloud-provider="external" Jul 05 13:15:14.261840 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:14.256539 2578 flags.go:64] FLAG: --cluster-dns="[]" Jul 05 13:15:14.261840 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:14.256543 2578 flags.go:64] FLAG: --cluster-domain="" Jul 05 13:15:14.261840 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:14.256545 2578 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 05 13:15:14.261840 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:14.256548 2578 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 05 13:15:14.261840 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:14.256550 2578 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 05 13:15:14.262397 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:14.256554 2578 flags.go:64] FLAG: --container-log-max-files="5" Jul 05 13:15:14.262397 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:14.256558 2578 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 05 13:15:14.262397 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:14.256560 2578 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 05 13:15:14.262397 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:14.256563 2578 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 05 13:15:14.262397 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:14.256566 2578 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 05 13:15:14.262397 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:14.256569 2578 flags.go:64] FLAG: --contention-profiling="false" Jul 05 13:15:14.262397 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:14.256571 2578 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 05 13:15:14.262397 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:14.256575 2578 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 05 13:15:14.262397 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:14.256578 2578 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 05 13:15:14.262397 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:14.256580 2578 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 05 13:15:14.262397 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:14.256584 2578 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 05 13:15:14.262397 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:14.256587 2578 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 05 13:15:14.262397 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:14.256590 2578 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 05 13:15:14.262397 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:14.256592 2578 flags.go:64] FLAG: --enable-load-reader="false" Jul 05 13:15:14.262397 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:14.256595 2578 flags.go:64] FLAG: --enable-server="true" Jul 05 13:15:14.262397 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:14.256597 2578 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 05 13:15:14.262397 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:14.256602 2578 flags.go:64] FLAG: --event-burst="100" Jul 05 13:15:14.262397 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:14.256605 2578 flags.go:64] FLAG: --event-qps="50" Jul 05 13:15:14.262397 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:14.256607 2578 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 05 13:15:14.262397 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:14.256610 2578 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 05 13:15:14.262397 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:14.256613 2578 flags.go:64] FLAG: --eviction-hard="" Jul 05 13:15:14.262397 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:14.256617 2578 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 05 13:15:14.262397 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:14.256619 2578 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 05 13:15:14.262397 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:14.256622 2578 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 05 13:15:14.262397 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:14.256626 2578 flags.go:64] FLAG: --eviction-soft="" Jul 05 13:15:14.263177 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:14.256628 2578 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 05 13:15:14.263177 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:14.256631 2578 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 05 13:15:14.263177 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:14.256634 2578 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 05 13:15:14.263177 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:14.256637 2578 flags.go:64] FLAG: --experimental-mounter-path="" Jul 05 13:15:14.263177 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:14.256640 2578 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 05 13:15:14.263177 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:14.256643 2578 flags.go:64] FLAG: --fail-swap-on="true" Jul 05 13:15:14.263177 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:14.256645 2578 flags.go:64] FLAG: --feature-gates="" Jul 05 13:15:14.263177 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:14.256649 2578 flags.go:64] FLAG: --file-check-frequency="20s" Jul 05 13:15:14.263177 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:14.256651 2578 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 05 13:15:14.263177 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:14.256654 2578 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 05 13:15:14.263177 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:14.256657 2578 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 05 13:15:14.263177 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:14.256660 2578 flags.go:64] FLAG: --healthz-port="10248" Jul 05 13:15:14.263177 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:14.256662 2578 flags.go:64] FLAG: --help="false" Jul 05 13:15:14.263177 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:14.256665 2578 flags.go:64] FLAG: --hostname-override="ip-10-0-130-191.ec2.internal" Jul 05 13:15:14.263177 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:14.256668 2578 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 05 13:15:14.263177 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:14.256671 2578 flags.go:64] FLAG: --http-check-frequency="20s" Jul 05 13:15:14.263177 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:14.256674 2578 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 05 13:15:14.263177 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:14.256677 2578 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 05 13:15:14.263177 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:14.256680 2578 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 05 13:15:14.263177 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:14.256683 2578 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 05 13:15:14.263177 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:14.256685 2578 flags.go:64] FLAG: --image-service-endpoint="" Jul 05 13:15:14.263177 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:14.256688 2578 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 05 13:15:14.263177 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:14.256691 2578 flags.go:64] FLAG: --kube-api-burst="100" Jul 05 13:15:14.264186 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:14.256693 2578 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 05 13:15:14.264186 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:14.256696 2578 flags.go:64] FLAG: --kube-api-qps="50" Jul 05 13:15:14.264186 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:14.256699 2578 flags.go:64] FLAG: --kube-reserved="" Jul 05 13:15:14.264186 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:14.256702 2578 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 05 13:15:14.264186 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:14.256704 2578 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 05 13:15:14.264186 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:14.256707 2578 flags.go:64] FLAG: --kubelet-cgroups="" Jul 05 13:15:14.264186 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:14.256709 2578 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 05 13:15:14.264186 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:14.256712 2578 flags.go:64] FLAG: --lock-file="" Jul 05 13:15:14.264186 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:14.256715 2578 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 05 13:15:14.264186 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:14.256718 2578 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 05 13:15:14.264186 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:14.256721 2578 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 05 13:15:14.264186 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:14.256728 2578 flags.go:64] FLAG: --log-json-split-stream="false" Jul 05 13:15:14.264186 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:14.256731 2578 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 05 13:15:14.264186 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:14.256733 2578 flags.go:64] FLAG: --log-text-split-stream="false" Jul 05 13:15:14.264186 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:14.256736 2578 flags.go:64] FLAG: --logging-format="text" Jul 05 13:15:14.264186 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:14.256738 2578 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 05 13:15:14.264186 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:14.256742 2578 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 05 13:15:14.264186 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:14.256745 2578 flags.go:64] FLAG: --manifest-url="" Jul 05 13:15:14.264186 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:14.256747 2578 flags.go:64] FLAG: --manifest-url-header="" Jul 05 13:15:14.264186 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:14.256752 2578 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 05 13:15:14.264186 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:14.256754 2578 flags.go:64] FLAG: --max-open-files="1000000" Jul 05 13:15:14.264186 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:14.256758 2578 flags.go:64] FLAG: --max-pods="110" Jul 05 13:15:14.264186 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:14.256761 2578 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 05 13:15:14.264186 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:14.256764 2578 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 05 13:15:14.264186 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:14.256767 2578 flags.go:64] FLAG: --memory-manager-policy="None" Jul 05 13:15:14.265239 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:14.256769 2578 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 05 13:15:14.265239 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:14.256772 2578 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 05 13:15:14.265239 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:14.256775 2578 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 05 13:15:14.265239 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:14.256777 2578 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 05 13:15:14.265239 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:14.256784 2578 flags.go:64] FLAG: --node-status-max-images="50" Jul 05 13:15:14.265239 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:14.256787 2578 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 05 13:15:14.265239 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:14.256790 2578 flags.go:64] FLAG: --oom-score-adj="-999" Jul 05 13:15:14.265239 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:14.256792 2578 flags.go:64] FLAG: --pod-cidr="" Jul 05 13:15:14.265239 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:14.256795 2578 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ce93f6b4197b9c62a26a3131f8057ffe9c9379c5079a4f62e5919b75151394e5" Jul 05 13:15:14.265239 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:14.256801 2578 flags.go:64] FLAG: --pod-manifest-path="" Jul 05 13:15:14.265239 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:14.256803 2578 flags.go:64] FLAG: --pod-max-pids="-1" Jul 05 13:15:14.265239 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:14.256806 2578 flags.go:64] FLAG: --pods-per-core="0" Jul 05 13:15:14.265239 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:14.256809 2578 flags.go:64] FLAG: --port="10250" Jul 05 13:15:14.265239 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:14.256812 2578 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 05 13:15:14.265239 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:14.256815 2578 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0e119ccf99238cb98" Jul 05 13:15:14.265239 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:14.256817 2578 flags.go:64] FLAG: --qos-reserved="" Jul 05 13:15:14.265239 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:14.256820 2578 flags.go:64] FLAG: --read-only-port="10255" Jul 05 13:15:14.265239 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:14.256824 2578 flags.go:64] FLAG: --register-node="true" Jul 05 13:15:14.265239 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:14.256826 2578 flags.go:64] FLAG: --register-with-taints="" Jul 05 13:15:14.265239 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:14.256831 2578 flags.go:64] FLAG: --registry-burst="10" Jul 05 13:15:14.265239 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:14.256834 2578 flags.go:64] FLAG: --registry-qps="5" Jul 05 13:15:14.265239 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:14.256836 2578 flags.go:64] FLAG: --reserved-cpus="" Jul 05 13:15:14.265239 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:14.256839 2578 flags.go:64] FLAG: --reserved-memory="" Jul 05 13:15:14.265239 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:14.256843 2578 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 05 13:15:14.265239 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:14.256846 2578 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 05 13:15:14.266348 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:14.256849 2578 flags.go:64] FLAG: --rotate-certificates="false" Jul 05 13:15:14.266348 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:14.256851 2578 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 05 13:15:14.266348 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:14.256854 2578 flags.go:64] FLAG: --runonce="false" Jul 05 13:15:14.266348 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:14.256857 2578 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 05 13:15:14.266348 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:14.256860 2578 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 05 13:15:14.266348 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:14.256862 2578 flags.go:64] FLAG: --seccomp-default="false" Jul 05 13:15:14.266348 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:14.256865 2578 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 05 13:15:14.266348 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:14.256868 2578 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 05 13:15:14.266348 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:14.256870 2578 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 05 13:15:14.266348 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:14.256873 2578 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 05 13:15:14.266348 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:14.256876 2578 flags.go:64] FLAG: --storage-driver-password="root" Jul 05 13:15:14.266348 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:14.256878 2578 flags.go:64] FLAG: --storage-driver-secure="false" Jul 05 13:15:14.266348 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:14.256881 2578 flags.go:64] FLAG: --storage-driver-table="stats" Jul 05 13:15:14.266348 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:14.256884 2578 flags.go:64] FLAG: --storage-driver-user="root" Jul 05 13:15:14.266348 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:14.256886 2578 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 05 13:15:14.266348 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:14.256889 2578 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 05 13:15:14.266348 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:14.256892 2578 flags.go:64] FLAG: --system-cgroups="" Jul 05 13:15:14.266348 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:14.256895 2578 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 05 13:15:14.266348 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:14.256900 2578 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 05 13:15:14.266348 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:14.256902 2578 flags.go:64] FLAG: --tls-cert-file="" Jul 05 13:15:14.266348 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:14.256905 2578 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 05 13:15:14.266348 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:14.256908 2578 flags.go:64] FLAG: --tls-min-version="" Jul 05 13:15:14.266348 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:14.256911 2578 flags.go:64] FLAG: --tls-private-key-file="" Jul 05 13:15:14.266348 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:14.256914 2578 flags.go:64] FLAG: --topology-manager-policy="none" Jul 05 13:15:14.266348 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:14.256916 2578 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 05 13:15:14.267139 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:14.256919 2578 flags.go:64] FLAG: --topology-manager-scope="container" Jul 05 13:15:14.267139 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:14.256923 2578 flags.go:64] FLAG: --v="2" Jul 05 13:15:14.267139 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:14.256927 2578 flags.go:64] FLAG: --version="false" Jul 05 13:15:14.267139 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:14.256930 2578 flags.go:64] FLAG: --vmodule="" Jul 05 13:15:14.267139 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:14.256934 2578 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 05 13:15:14.267139 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:14.256937 2578 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 05 13:15:14.267139 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.257065 2578 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 05 13:15:14.267139 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.257070 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 05 13:15:14.267139 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.257073 2578 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 05 13:15:14.267139 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.257075 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 05 13:15:14.267139 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.257078 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 05 13:15:14.267139 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.257081 2578 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 05 13:15:14.267139 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.257084 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 05 13:15:14.267139 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.257086 2578 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 05 13:15:14.267139 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.257089 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 05 13:15:14.267139 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.257091 2578 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 05 13:15:14.267139 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.257093 2578 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 05 13:15:14.267139 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.257096 2578 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 05 13:15:14.267139 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.257098 2578 feature_gate.go:328] unrecognized feature gate: Example Jul 05 13:15:14.267139 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.257100 2578 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 05 13:15:14.267139 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.257103 2578 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 05 13:15:14.267139 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.257105 2578 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 05 13:15:14.268035 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.257107 2578 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 05 13:15:14.268035 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.257110 2578 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 05 13:15:14.268035 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.257112 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 05 13:15:14.268035 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.257114 2578 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 05 13:15:14.268035 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.257117 2578 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 05 13:15:14.268035 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.257119 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 05 13:15:14.268035 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.257122 2578 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 05 13:15:14.268035 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.257124 2578 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 05 13:15:14.268035 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.257126 2578 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 05 13:15:14.268035 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.257130 2578 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 05 13:15:14.268035 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.257133 2578 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 05 13:15:14.268035 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.257138 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 05 13:15:14.268035 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.257140 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 05 13:15:14.268035 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.257143 2578 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 05 13:15:14.268035 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.257146 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 05 13:15:14.268035 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.257148 2578 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 05 13:15:14.268035 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.257151 2578 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 05 13:15:14.268035 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.257153 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 05 13:15:14.268035 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.257155 2578 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 05 13:15:14.268831 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.257158 2578 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 05 13:15:14.268831 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.257160 2578 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 05 13:15:14.268831 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.257164 2578 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 05 13:15:14.268831 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.257166 2578 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 05 13:15:14.268831 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.257169 2578 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 05 13:15:14.268831 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.257171 2578 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 05 13:15:14.268831 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.257173 2578 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 05 13:15:14.268831 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.257177 2578 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 05 13:15:14.268831 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.257181 2578 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 05 13:15:14.268831 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.257184 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 05 13:15:14.268831 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.257187 2578 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 05 13:15:14.268831 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.257189 2578 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 05 13:15:14.268831 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.257192 2578 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 05 13:15:14.268831 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.257194 2578 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 05 13:15:14.268831 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.257196 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 05 13:15:14.268831 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.257199 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 05 13:15:14.268831 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.257201 2578 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 05 13:15:14.268831 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.257203 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 05 13:15:14.268831 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.257206 2578 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 05 13:15:14.269411 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.257208 2578 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 05 13:15:14.269411 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.257210 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 05 13:15:14.269411 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.257213 2578 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 05 13:15:14.269411 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.257215 2578 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 05 13:15:14.269411 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.257217 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 05 13:15:14.269411 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.257221 2578 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 05 13:15:14.269411 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.257224 2578 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 05 13:15:14.269411 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.257227 2578 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 05 13:15:14.269411 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.257229 2578 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 05 13:15:14.269411 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.257231 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 05 13:15:14.269411 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.257234 2578 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 05 13:15:14.269411 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.257236 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 05 13:15:14.269411 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.257238 2578 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 05 13:15:14.269411 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.257241 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 05 13:15:14.269411 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.257243 2578 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 05 13:15:14.269411 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.257247 2578 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 05 13:15:14.269411 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.257249 2578 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 05 13:15:14.269411 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.257252 2578 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 05 13:15:14.269411 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.257254 2578 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 05 13:15:14.269411 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.257256 2578 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 05 13:15:14.269973 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.257259 2578 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 05 13:15:14.269973 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.257261 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 05 13:15:14.269973 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.257264 2578 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 05 13:15:14.269973 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.257266 2578 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 05 13:15:14.269973 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.257269 2578 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 05 13:15:14.269973 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.257271 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 05 13:15:14.269973 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.257273 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 05 13:15:14.269973 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.257291 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 05 13:15:14.269973 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.257293 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 05 13:15:14.269973 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.257295 2578 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 05 13:15:14.269973 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.257298 2578 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 05 13:15:14.269973 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.257300 2578 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 05 13:15:14.269973 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.257302 2578 feature_gate.go:328] unrecognized feature gate: Example2 Jul 05 13:15:14.269973 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.257305 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 05 13:15:14.269973 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.257307 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 05 13:15:14.269973 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.257310 2578 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 05 13:15:14.269973 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.257312 2578 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 05 13:15:14.269973 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.257317 2578 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 05 13:15:14.269973 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.257319 2578 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 05 13:15:14.269973 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.257322 2578 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 05 13:15:14.270481 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.257325 2578 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 05 13:15:14.270481 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.257327 2578 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 05 13:15:14.270481 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.257330 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 05 13:15:14.270481 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.257332 2578 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 05 13:15:14.270481 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.257335 2578 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 05 13:15:14.270481 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:14.257855 2578 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 05 13:15:14.270481 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:14.265157 2578 server.go:529] "Kubelet version" kubeletVersion="v1.34.8" Jul 05 13:15:14.270481 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:14.265176 2578 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 05 13:15:14.270481 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:14.265200 2578 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 05 13:15:14.270481 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:14.265213 2578 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 05 13:15:14.270481 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.265326 2578 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 05 13:15:14.270481 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.265334 2578 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 05 13:15:14.270481 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.265339 2578 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 05 13:15:14.270481 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.265343 2578 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 05 13:15:14.270890 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.265347 2578 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 05 13:15:14.270890 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.265352 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 05 13:15:14.270890 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.265356 2578 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 05 13:15:14.270890 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.265360 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 05 13:15:14.270890 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.265364 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 05 13:15:14.270890 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.265368 2578 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 05 13:15:14.270890 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.265373 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 05 13:15:14.270890 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.265377 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 05 13:15:14.270890 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.265381 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 05 13:15:14.270890 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.265385 2578 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 05 13:15:14.270890 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.265389 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 05 13:15:14.270890 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.265393 2578 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 05 13:15:14.270890 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.265397 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 05 13:15:14.270890 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.265401 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 05 13:15:14.270890 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.265407 2578 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 05 13:15:14.270890 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.265414 2578 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 05 13:15:14.270890 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.265418 2578 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 05 13:15:14.270890 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.265431 2578 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 05 13:15:14.270890 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.265436 2578 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 05 13:15:14.270890 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.265440 2578 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 05 13:15:14.271451 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.265443 2578 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 05 13:15:14.271451 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.265447 2578 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 05 13:15:14.271451 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.265451 2578 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 05 13:15:14.271451 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.265456 2578 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 05 13:15:14.271451 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.265460 2578 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 05 13:15:14.271451 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.265464 2578 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 05 13:15:14.271451 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.265468 2578 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 05 13:15:14.271451 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.265473 2578 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 05 13:15:14.271451 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.265477 2578 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 05 13:15:14.271451 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.265489 2578 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 05 13:15:14.271451 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.265494 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 05 13:15:14.271451 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.265498 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 05 13:15:14.271451 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.265502 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 05 13:15:14.271451 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.265506 2578 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 05 13:15:14.271451 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.265510 2578 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 05 13:15:14.271451 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.265514 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 05 13:15:14.271451 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.265518 2578 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 05 13:15:14.271451 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.265522 2578 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 05 13:15:14.271451 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.265526 2578 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 05 13:15:14.271451 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.265530 2578 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 05 13:15:14.271965 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.265534 2578 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 05 13:15:14.271965 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.265538 2578 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 05 13:15:14.271965 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.265542 2578 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 05 13:15:14.271965 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.265546 2578 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 05 13:15:14.271965 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.265550 2578 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 05 13:15:14.271965 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.265554 2578 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 05 13:15:14.271965 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.265558 2578 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 05 13:15:14.271965 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.265562 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 05 13:15:14.271965 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.265566 2578 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 05 13:15:14.271965 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.265570 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 05 13:15:14.271965 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.265581 2578 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 05 13:15:14.271965 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.265585 2578 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 05 13:15:14.271965 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.265589 2578 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 05 13:15:14.271965 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.265593 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 05 13:15:14.271965 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.265597 2578 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 05 13:15:14.271965 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.265602 2578 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 05 13:15:14.271965 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.265607 2578 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 05 13:15:14.271965 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.265611 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 05 13:15:14.271965 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.265615 2578 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 05 13:15:14.271965 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.265619 2578 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 05 13:15:14.272502 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.265623 2578 feature_gate.go:328] unrecognized feature gate: Example2 Jul 05 13:15:14.272502 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.265629 2578 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 05 13:15:14.272502 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.265635 2578 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 05 13:15:14.272502 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.265639 2578 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 05 13:15:14.272502 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.265643 2578 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 05 13:15:14.272502 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.265648 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 05 13:15:14.272502 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.265652 2578 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 05 13:15:14.272502 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.265656 2578 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 05 13:15:14.272502 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.265660 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 05 13:15:14.272502 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.265664 2578 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 05 13:15:14.272502 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.265668 2578 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 05 13:15:14.272502 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.265672 2578 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 05 13:15:14.272502 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.265676 2578 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 05 13:15:14.272502 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.265680 2578 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 05 13:15:14.272502 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.265684 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 05 13:15:14.272502 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.265688 2578 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 05 13:15:14.272502 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.265694 2578 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 05 13:15:14.272502 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.265699 2578 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 05 13:15:14.272502 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.265709 2578 feature_gate.go:328] unrecognized feature gate: Example Jul 05 13:15:14.273174 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.265713 2578 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 05 13:15:14.273174 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.265718 2578 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 05 13:15:14.273174 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.265721 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 05 13:15:14.273174 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.265726 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 05 13:15:14.273174 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.265737 2578 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 05 13:15:14.273174 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.265741 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 05 13:15:14.273174 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.265745 2578 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 05 13:15:14.273174 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.265749 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 05 13:15:14.273174 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.265753 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 05 13:15:14.273174 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.265757 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 05 13:15:14.273174 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.265762 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 05 13:15:14.273174 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.265766 2578 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 05 13:15:14.273174 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.265770 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 05 13:15:14.273174 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.265774 2578 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 05 13:15:14.273174 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.265778 2578 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 05 13:15:14.273174 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.265782 2578 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 05 13:15:14.273735 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:14.265790 2578 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 05 13:15:14.273735 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.266002 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 05 13:15:14.273735 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.266010 2578 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 05 13:15:14.273735 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.266015 2578 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 05 13:15:14.273735 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.266020 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 05 13:15:14.273735 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.266024 2578 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 05 13:15:14.273735 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.266028 2578 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 05 13:15:14.273735 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.266032 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 05 13:15:14.273735 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.266036 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 05 13:15:14.273735 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.266040 2578 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 05 13:15:14.273735 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.266044 2578 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 05 13:15:14.273735 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.266048 2578 feature_gate.go:328] unrecognized feature gate: Example Jul 05 13:15:14.273735 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.266052 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 05 13:15:14.273735 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.266056 2578 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 05 13:15:14.273735 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.266059 2578 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 05 13:15:14.274103 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.266063 2578 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 05 13:15:14.274103 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.266068 2578 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 05 13:15:14.274103 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.266072 2578 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 05 13:15:14.274103 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.266075 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 05 13:15:14.274103 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.266079 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 05 13:15:14.274103 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.266083 2578 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 05 13:15:14.274103 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.266095 2578 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 05 13:15:14.274103 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.266099 2578 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 05 13:15:14.274103 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.266103 2578 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 05 13:15:14.274103 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.266107 2578 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 05 13:15:14.274103 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.266111 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 05 13:15:14.274103 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.266115 2578 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 05 13:15:14.274103 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.266119 2578 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 05 13:15:14.274103 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.266123 2578 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 05 13:15:14.274103 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.266129 2578 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 05 13:15:14.274103 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.266134 2578 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 05 13:15:14.274103 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.266138 2578 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 05 13:15:14.274103 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.266142 2578 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 05 13:15:14.274103 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.266147 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 05 13:15:14.274640 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.266153 2578 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 05 13:15:14.274640 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.266159 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 05 13:15:14.274640 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.266164 2578 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 05 13:15:14.274640 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.266168 2578 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 05 13:15:14.274640 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.266172 2578 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 05 13:15:14.274640 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.266177 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 05 13:15:14.274640 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.266180 2578 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 05 13:15:14.274640 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.266185 2578 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 05 13:15:14.274640 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.266188 2578 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 05 13:15:14.274640 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.266192 2578 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 05 13:15:14.274640 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.266196 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 05 13:15:14.274640 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.266200 2578 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 05 13:15:14.274640 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.266204 2578 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 05 13:15:14.274640 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.266208 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 05 13:15:14.274640 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.266213 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 05 13:15:14.274640 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.266218 2578 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 05 13:15:14.274640 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.266221 2578 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 05 13:15:14.274640 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.266225 2578 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 05 13:15:14.274640 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.266229 2578 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 05 13:15:14.274640 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.266233 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 05 13:15:14.275161 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.266245 2578 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 05 13:15:14.275161 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.266250 2578 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 05 13:15:14.275161 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.266254 2578 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 05 13:15:14.275161 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.266258 2578 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 05 13:15:14.275161 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.266262 2578 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 05 13:15:14.275161 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.266266 2578 feature_gate.go:328] unrecognized feature gate: Example2 Jul 05 13:15:14.275161 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.266270 2578 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 05 13:15:14.275161 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.266296 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 05 13:15:14.275161 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.266300 2578 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 05 13:15:14.275161 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.266304 2578 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 05 13:15:14.275161 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.266308 2578 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 05 13:15:14.275161 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.266312 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 05 13:15:14.275161 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.266316 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 05 13:15:14.275161 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.266319 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 05 13:15:14.275161 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.266324 2578 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 05 13:15:14.275161 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.266327 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 05 13:15:14.275161 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.266331 2578 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 05 13:15:14.275161 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.266335 2578 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 05 13:15:14.275161 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.266339 2578 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 05 13:15:14.275161 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.266343 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 05 13:15:14.275954 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.266347 2578 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 05 13:15:14.275954 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.266351 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 05 13:15:14.275954 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.266355 2578 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 05 13:15:14.275954 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.266359 2578 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 05 13:15:14.275954 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.266363 2578 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 05 13:15:14.275954 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.266366 2578 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 05 13:15:14.275954 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.266370 2578 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 05 13:15:14.275954 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.266374 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 05 13:15:14.275954 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.266378 2578 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 05 13:15:14.275954 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.266382 2578 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 05 13:15:14.275954 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.266388 2578 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 05 13:15:14.275954 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.266393 2578 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 05 13:15:14.275954 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.266398 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 05 13:15:14.275954 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.266410 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 05 13:15:14.275954 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.266415 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 05 13:15:14.275954 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.266419 2578 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 05 13:15:14.275954 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.266423 2578 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 05 13:15:14.275954 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.266427 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 05 13:15:14.275954 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.266431 2578 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 05 13:15:14.276686 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.266435 2578 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 05 13:15:14.276686 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.266439 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 05 13:15:14.276686 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.266442 2578 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 05 13:15:14.276686 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.266446 2578 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 05 13:15:14.276686 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.266450 2578 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 05 13:15:14.276686 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.266454 2578 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 05 13:15:14.276686 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:14.266458 2578 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 05 13:15:14.276686 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:14.266466 2578 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 05 13:15:14.276686 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:14.266639 2578 server.go:962] "Client rotation is on, will bootstrap in background" Jul 05 13:15:14.276686 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:14.269348 2578 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 05 13:15:14.276686 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:14.270554 2578 server.go:1019] "Starting client certificate rotation" Jul 05 13:15:14.276686 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:14.270666 2578 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 05 13:15:14.276686 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:14.271359 2578 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 05 13:15:14.277050 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:14.274315 2578 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 05 13:15:14.278402 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:14.278383 2578 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 05 13:15:14.293042 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:14.293023 2578 log.go:25] "Validated CRI v1 runtime API" Jul 05 13:15:14.294960 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:14.294947 2578 log.go:25] "Validated CRI v1 image API" Jul 05 13:15:14.295909 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:14.295892 2578 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 05 13:15:14.298673 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:14.298655 2578 fs.go:135] Filesystem UUIDs: map[5f9ab1c1-9800-4335-8b84-3828d9a7c30e:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 a0d1deab-b88a-4d08-9294-88979550188b:/dev/nvme0n1p3] Jul 05 13:15:14.298749 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:14.298674 2578 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 05 13:15:14.303810 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:14.303791 2578 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 05 13:15:14.303991 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:14.303880 2578 manager.go:216] Machine: {Timestamp:2026-07-05 13:15:14.302625092 +0000 UTC m=+0.301489321 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3163264 MemoryCapacity:32812179456 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec28f7cb4a1bbe20a4d7367a04527381 SystemUUID:ec28f7cb-4a1b-be20-a4d7-367a04527381 BootID:50de0c6f-b928-41b1-8352-824db008e16b Filesystems:[{Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406089728 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406089728 Type:vfs Inodes:4005393 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562439168 Type:vfs Inodes:819200 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:d3:ee:03:a6:91 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:d3:ee:03:a6:91 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:6e:f7:e3:7d:c0:ca Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812179456 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 05 13:15:14.303991 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:14.303988 2578 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 05 13:15:14.304088 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:14.304074 2578 manager.go:232] Version: {KernelVersion:5.14.0-570.125.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260630-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 05 13:15:14.307498 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:14.307475 2578 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 05 13:15:14.307638 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:14.307501 2578 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-130-191.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 05 13:15:14.308230 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:14.308220 2578 topology_manager.go:138] "Creating topology manager with none policy" Jul 05 13:15:14.308269 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:14.308232 2578 container_manager_linux.go:309] "Creating device plugin manager" Jul 05 13:15:14.308269 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:14.308240 2578 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 05 13:15:14.308821 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:14.308811 2578 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 05 13:15:14.309956 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:14.309947 2578 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 05 13:15:14.315291 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:14.315267 2578 state_mem.go:36] "Initialized new in-memory state store" Jul 05 13:15:14.315409 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:14.315400 2578 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 05 13:15:14.315441 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:14.315430 2578 kubelet.go:486] "Attempting to sync node with API server" Jul 05 13:15:14.315441 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:14.315440 2578 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 05 13:15:14.315496 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:14.315455 2578 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 05 13:15:14.315496 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:14.315464 2578 kubelet.go:398] "Adding apiserver pod source" Jul 05 13:15:14.315496 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:14.315473 2578 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 05 13:15:14.315876 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:14.315862 2578 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 05 13:15:14.315909 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:14.315891 2578 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 05 13:15:14.318019 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:14.317997 2578 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.8-3.rhaos4.21.git91b4b0b.el9" apiVersion="v1" Jul 05 13:15:14.319992 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:14.319977 2578 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 05 13:15:14.320067 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:14.320008 2578 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 05 13:15:14.321014 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:14.321003 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 05 13:15:14.321059 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:14.321020 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 05 13:15:14.321059 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:14.321026 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 05 13:15:14.321059 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:14.321031 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 05 13:15:14.321059 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:14.321039 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 05 13:15:14.321059 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:14.321045 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 05 13:15:14.321059 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:14.321054 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 05 13:15:14.321059 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:14.321058 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 05 13:15:14.321330 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:14.321065 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 05 13:15:14.321330 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:14.321070 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 05 13:15:14.321330 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:14.321079 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 05 13:15:14.321330 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:14.321084 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 05 13:15:14.322910 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:14.322897 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 05 13:15:14.322976 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:14.322916 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 05 13:15:14.326881 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:14.326864 2578 server.go:1268] "Started kubelet" Jul 05 13:15:14.327047 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:14.327013 2578 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 05 13:15:14.327109 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:14.327022 2578 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 05 13:15:14.327109 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:14.327101 2578 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 05 13:15:14.327775 ip-10-0-130-191 kubenswrapper[2578]: E0705 13:15:14.327748 2578 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-130-191.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 05 13:15:14.327871 ip-10-0-130-191 kubenswrapper[2578]: E0705 13:15:14.327807 2578 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 05 13:15:14.327842 ip-10-0-130-191 systemd[1]: Started Kubernetes Kubelet. Jul 05 13:15:14.328008 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:14.327975 2578 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-130-191.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 05 13:15:14.328775 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:14.328711 2578 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 05 13:15:14.329157 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:14.329143 2578 server.go:310] "Adding debug handlers to kubelet server" Jul 05 13:15:14.333502 ip-10-0-130-191 kubenswrapper[2578]: E0705 13:15:14.333466 2578 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 05 13:15:14.334937 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:14.334923 2578 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 05 13:15:14.335006 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:14.334958 2578 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 05 13:15:14.335160 ip-10-0-130-191 kubenswrapper[2578]: E0705 13:15:14.335138 2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-191.ec2.internal\" not found" Jul 05 13:15:14.335478 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:14.335459 2578 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 05 13:15:14.335478 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:14.335462 2578 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 05 13:15:14.335615 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:14.335486 2578 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 05 13:15:14.335615 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:14.335588 2578 reconstruct.go:97] "Volume reconstruction finished" Jul 05 13:15:14.335615 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:14.335598 2578 reconciler.go:29] "Reconciler: start to sync state" Jul 05 13:15:14.336326 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:14.336308 2578 factory.go:55] Registering systemd factory Jul 05 13:15:14.336326 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:14.336325 2578 factory.go:223] Registration of the systemd container factory successfully Jul 05 13:15:14.336466 ip-10-0-130-191 kubenswrapper[2578]: E0705 13:15:14.335422 2578 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-130-191.ec2.internal.18bf67144b916ea9 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-130-191.ec2.internal,UID:ip-10-0-130-191.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-130-191.ec2.internal,},FirstTimestamp:2026-07-05 13:15:14.326834857 +0000 UTC m=+0.325699085,LastTimestamp:2026-07-05 13:15:14.326834857 +0000 UTC m=+0.325699085,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-130-191.ec2.internal,}" Jul 05 13:15:14.336635 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:14.336621 2578 factory.go:153] Registering CRI-O factory Jul 05 13:15:14.336696 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:14.336638 2578 factory.go:223] Registration of the crio container factory successfully Jul 05 13:15:14.336696 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:14.336689 2578 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 05 13:15:14.336793 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:14.336714 2578 factory.go:103] Registering Raw factory Jul 05 13:15:14.336793 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:14.336732 2578 manager.go:1196] Started watching for new ooms in manager Jul 05 13:15:14.337607 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:14.337558 2578 manager.go:349] Starting recovery of all containers Jul 05 13:15:14.340321 ip-10-0-130-191 kubenswrapper[2578]: E0705 13:15:14.340297 2578 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 05 13:15:14.340437 ip-10-0-130-191 kubenswrapper[2578]: E0705 13:15:14.340414 2578 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-130-191.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 05 13:15:14.346946 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:14.346800 2578 manager.go:354] Recovery completed Jul 05 13:15:14.349395 ip-10-0-130-191 kubenswrapper[2578]: E0705 13:15:14.349372 2578 watcher.go:152] Failed to watch directory "/sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service": inotify_add_watch /sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service: no such file or directory Jul 05 13:15:14.352258 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:14.352246 2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 05 13:15:14.354030 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:14.354009 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-191.ec2.internal" event="NodeHasSufficientMemory" Jul 05 13:15:14.354105 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:14.354046 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-191.ec2.internal" event="NodeHasNoDiskPressure" Jul 05 13:15:14.354105 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:14.354060 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-191.ec2.internal" event="NodeHasSufficientPID" Jul 05 13:15:14.354555 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:14.354543 2578 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 05 13:15:14.354608 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:14.354556 2578 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 05 13:15:14.354608 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:14.354560 2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 05 13:15:14.354608 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:14.354572 2578 state_mem.go:36] "Initialized new in-memory state store" Jul 05 13:15:14.355262 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:14.355248 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-191.ec2.internal" event="NodeHasSufficientMemory" Jul 05 13:15:14.355358 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:14.355289 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-191.ec2.internal" event="NodeHasNoDiskPressure" Jul 05 13:15:14.355358 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:14.355304 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-191.ec2.internal" event="NodeHasSufficientPID" Jul 05 13:15:14.356319 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:14.356306 2578 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-csvm5" Jul 05 13:15:14.356811 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:14.356800 2578 policy_none.go:49] "None policy: Start" Jul 05 13:15:14.356859 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:14.356815 2578 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 05 13:15:14.356859 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:14.356825 2578 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 05 13:15:14.356921 ip-10-0-130-191 kubenswrapper[2578]: E0705 13:15:14.356812 2578 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-130-191.ec2.internal.18bf67144d30654a default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-130-191.ec2.internal,UID:ip-10-0-130-191.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-130-191.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-130-191.ec2.internal,},FirstTimestamp:2026-07-05 13:15:14.354029898 +0000 UTC m=+0.352894131,LastTimestamp:2026-07-05 13:15:14.354029898 +0000 UTC m=+0.352894131,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-130-191.ec2.internal,}" Jul 05 13:15:14.358819 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:14.358808 2578 policy_none.go:47] "Start" Jul 05 13:15:14.363756 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:14.363738 2578 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-csvm5" Jul 05 13:15:14.365536 ip-10-0-130-191 kubenswrapper[2578]: E0705 13:15:14.365478 2578 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-130-191.ec2.internal.18bf67144d30c405 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-130-191.ec2.internal,UID:ip-10-0-130-191.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-130-191.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-130-191.ec2.internal,},FirstTimestamp:2026-07-05 13:15:14.354054149 +0000 UTC m=+0.352918386,LastTimestamp:2026-07-05 13:15:14.354054149 +0000 UTC m=+0.352918386,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-130-191.ec2.internal,}" Jul 05 13:15:14.395875 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:14.395860 2578 manager.go:337] "Starting Device Plugin manager" Jul 05 13:15:14.395946 ip-10-0-130-191 kubenswrapper[2578]: E0705 13:15:14.395892 2578 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 05 13:15:14.395946 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:14.395901 2578 server.go:87] "Starting device plugin registration server" Jul 05 13:15:14.396142 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:14.396125 2578 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 05 13:15:14.396237 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:14.396141 2578 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 05 13:15:14.396315 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:14.396268 2578 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 05 13:15:14.396370 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:14.396358 2578 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 05 13:15:14.396595 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:14.396370 2578 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 05 13:15:14.396829 ip-10-0-130-191 kubenswrapper[2578]: E0705 13:15:14.396808 2578 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 05 13:15:14.396917 ip-10-0-130-191 kubenswrapper[2578]: E0705 13:15:14.396855 2578 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-130-191.ec2.internal\" not found" Jul 05 13:15:14.460889 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:14.460866 2578 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 05 13:15:14.468719 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:14.462142 2578 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 05 13:15:14.468719 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:14.462185 2578 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 05 13:15:14.468719 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:14.462209 2578 kubelet.go:2448] "Starting kubelet main sync loop" Jul 05 13:15:14.468719 ip-10-0-130-191 kubenswrapper[2578]: E0705 13:15:14.462246 2578 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 05 13:15:14.468719 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:14.464489 2578 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 05 13:15:14.496240 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:14.496202 2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 05 13:15:14.497066 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:14.497041 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-191.ec2.internal" event="NodeHasSufficientMemory" Jul 05 13:15:14.497169 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:14.497079 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-191.ec2.internal" event="NodeHasNoDiskPressure" Jul 05 13:15:14.497169 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:14.497094 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-191.ec2.internal" event="NodeHasSufficientPID" Jul 05 13:15:14.497169 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:14.497126 2578 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-130-191.ec2.internal" Jul 05 13:15:14.505614 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:14.505576 2578 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-130-191.ec2.internal" Jul 05 13:15:14.505614 ip-10-0-130-191 kubenswrapper[2578]: E0705 13:15:14.505596 2578 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-130-191.ec2.internal\": node \"ip-10-0-130-191.ec2.internal\" not found" Jul 05 13:15:14.539294 ip-10-0-130-191 kubenswrapper[2578]: E0705 13:15:14.539258 2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-191.ec2.internal\" not found" Jul 05 13:15:14.563268 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:14.563250 2578 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-191.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-130-191.ec2.internal"] Jul 05 13:15:14.563340 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:14.563314 2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 05 13:15:14.564667 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:14.564647 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-191.ec2.internal" event="NodeHasSufficientMemory" Jul 05 13:15:14.564756 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:14.564675 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-191.ec2.internal" event="NodeHasNoDiskPressure" Jul 05 13:15:14.564756 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:14.564688 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-191.ec2.internal" event="NodeHasSufficientPID" Jul 05 13:15:14.566974 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:14.566962 2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 05 13:15:14.567098 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:14.567086 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-191.ec2.internal" Jul 05 13:15:14.567145 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:14.567111 2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 05 13:15:14.567644 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:14.567618 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-191.ec2.internal" event="NodeHasSufficientMemory" Jul 05 13:15:14.567735 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:14.567660 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-191.ec2.internal" event="NodeHasNoDiskPressure" Jul 05 13:15:14.567735 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:14.567676 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-191.ec2.internal" event="NodeHasSufficientPID" Jul 05 13:15:14.567735 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:14.567622 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-191.ec2.internal" event="NodeHasSufficientMemory" Jul 05 13:15:14.567735 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:14.567727 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-191.ec2.internal" event="NodeHasNoDiskPressure" Jul 05 13:15:14.567735 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:14.567737 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-191.ec2.internal" event="NodeHasSufficientPID" Jul 05 13:15:14.569818 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:14.569801 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-191.ec2.internal" Jul 05 13:15:14.569898 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:14.569829 2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 05 13:15:14.570494 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:14.570480 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-191.ec2.internal" event="NodeHasSufficientMemory" Jul 05 13:15:14.570570 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:14.570500 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-191.ec2.internal" event="NodeHasNoDiskPressure" Jul 05 13:15:14.570570 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:14.570509 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-191.ec2.internal" event="NodeHasSufficientPID" Jul 05 13:15:14.590909 ip-10-0-130-191 kubenswrapper[2578]: E0705 13:15:14.590887 2578 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-130-191.ec2.internal\" not found" node="ip-10-0-130-191.ec2.internal" Jul 05 13:15:14.594349 ip-10-0-130-191 kubenswrapper[2578]: E0705 13:15:14.594334 2578 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-130-191.ec2.internal\" not found" node="ip-10-0-130-191.ec2.internal" Jul 05 13:15:14.639551 ip-10-0-130-191 kubenswrapper[2578]: E0705 13:15:14.639532 2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-191.ec2.internal\" not found" Jul 05 13:15:14.736756 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:14.736724 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/8e4a5e8e2cce016390634f883dc8c308-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-130-191.ec2.internal\" (UID: \"8e4a5e8e2cce016390634f883dc8c308\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-191.ec2.internal" Jul 05 13:15:14.736756 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:14.736759 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8e4a5e8e2cce016390634f883dc8c308-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-130-191.ec2.internal\" (UID: \"8e4a5e8e2cce016390634f883dc8c308\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-191.ec2.internal" Jul 05 13:15:14.736907 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:14.736774 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/d82e074a4be1637d97b91534bd80eb21-config\") pod \"kube-apiserver-proxy-ip-10-0-130-191.ec2.internal\" (UID: \"d82e074a4be1637d97b91534bd80eb21\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-130-191.ec2.internal" Jul 05 13:15:14.739795 ip-10-0-130-191 kubenswrapper[2578]: E0705 13:15:14.739778 2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-191.ec2.internal\" not found" Jul 05 13:15:14.837486 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:14.837415 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/8e4a5e8e2cce016390634f883dc8c308-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-130-191.ec2.internal\" (UID: \"8e4a5e8e2cce016390634f883dc8c308\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-191.ec2.internal" Jul 05 13:15:14.837486 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:14.837483 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8e4a5e8e2cce016390634f883dc8c308-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-130-191.ec2.internal\" (UID: \"8e4a5e8e2cce016390634f883dc8c308\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-191.ec2.internal" Jul 05 13:15:14.837586 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:14.837454 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8e4a5e8e2cce016390634f883dc8c308-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-130-191.ec2.internal\" (UID: \"8e4a5e8e2cce016390634f883dc8c308\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-191.ec2.internal" Jul 05 13:15:14.837586 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:14.837507 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/8e4a5e8e2cce016390634f883dc8c308-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-130-191.ec2.internal\" (UID: \"8e4a5e8e2cce016390634f883dc8c308\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-191.ec2.internal" Jul 05 13:15:14.837586 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:14.837539 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/d82e074a4be1637d97b91534bd80eb21-config\") pod \"kube-apiserver-proxy-ip-10-0-130-191.ec2.internal\" (UID: \"d82e074a4be1637d97b91534bd80eb21\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-130-191.ec2.internal" Jul 05 13:15:14.837676 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:14.837585 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/d82e074a4be1637d97b91534bd80eb21-config\") pod \"kube-apiserver-proxy-ip-10-0-130-191.ec2.internal\" (UID: \"d82e074a4be1637d97b91534bd80eb21\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-130-191.ec2.internal" Jul 05 13:15:14.840517 ip-10-0-130-191 kubenswrapper[2578]: E0705 13:15:14.840503 2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-191.ec2.internal\" not found" Jul 05 13:15:14.893676 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:14.893658 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-191.ec2.internal" Jul 05 13:15:14.897159 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:14.897139 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-191.ec2.internal" Jul 05 13:15:14.941111 ip-10-0-130-191 kubenswrapper[2578]: E0705 13:15:14.941085 2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-191.ec2.internal\" not found" Jul 05 13:15:15.041680 ip-10-0-130-191 kubenswrapper[2578]: E0705 13:15:15.041656 2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-191.ec2.internal\" not found" Jul 05 13:15:15.142243 ip-10-0-130-191 kubenswrapper[2578]: E0705 13:15:15.142180 2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-191.ec2.internal\" not found" Jul 05 13:15:15.242759 ip-10-0-130-191 kubenswrapper[2578]: E0705 13:15:15.242725 2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-191.ec2.internal\" not found" Jul 05 13:15:15.269901 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:15.269880 2578 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 05 13:15:15.270337 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:15.270004 2578 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 05 13:15:15.343430 ip-10-0-130-191 kubenswrapper[2578]: E0705 13:15:15.343399 2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-191.ec2.internal\" not found" Jul 05 13:15:15.366542 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:15.366508 2578 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-04 13:10:15 +0000 UTC" deadline="2028-01-14 10:56:08.385797985 +0000 UTC" Jul 05 13:15:15.366542 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:15.366536 2578 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13389h40m53.019265151s" Jul 05 13:15:15.438037 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:15.437981 2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 05 13:15:15.439514 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:15.439493 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-191.ec2.internal" event="NodeHasSufficientMemory" Jul 05 13:15:15.439572 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:15.439529 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-191.ec2.internal" event="NodeHasNoDiskPressure" Jul 05 13:15:15.439572 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:15.439540 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-191.ec2.internal" event="NodeHasSufficientPID" Jul 05 13:15:15.443722 ip-10-0-130-191 kubenswrapper[2578]: E0705 13:15:15.443707 2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-191.ec2.internal\" not found" Jul 05 13:15:17.085007 ip-10-0-130-191 kubenswrapper[2578]: E0705 13:15:17.084987 2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-191.ec2.internal\" not found" Jul 05 13:15:17.124726 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:17.124694 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8e4a5e8e2cce016390634f883dc8c308.slice/crio-6e4acd89fc2f1a3d4235cecd785fe34a878b37f94b6c651e7a0758f4d64bc7f6 WatchSource:0}: Error finding container 6e4acd89fc2f1a3d4235cecd785fe34a878b37f94b6c651e7a0758f4d64bc7f6: Status 404 returned error can't find the container with id 6e4acd89fc2f1a3d4235cecd785fe34a878b37f94b6c651e7a0758f4d64bc7f6 Jul 05 13:15:17.125126 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:17.125106 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd82e074a4be1637d97b91534bd80eb21.slice/crio-059eb809d8f4e8a37b74e61056a536a80825062b0ce477b7520dcb464c338aff WatchSource:0}: Error finding container 059eb809d8f4e8a37b74e61056a536a80825062b0ce477b7520dcb464c338aff: Status 404 returned error can't find the container with id 059eb809d8f4e8a37b74e61056a536a80825062b0ce477b7520dcb464c338aff Jul 05 13:15:17.130990 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:17.130977 2578 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 05 13:15:17.145084 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:17.145068 2578 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 05 13:15:17.165884 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:17.165866 2578 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 05 13:15:17.176257 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:17.176225 2578 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-191.ec2.internal" Jul 05 13:15:17.184200 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:17.184183 2578 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 05 13:15:17.185236 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:17.185224 2578 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-191.ec2.internal" Jul 05 13:15:17.198321 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:17.198297 2578 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 05 13:15:17.208440 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:17.208423 2578 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 05 13:15:17.798250 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:17.798222 2578 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 05 13:15:17.856696 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:17.856650 2578 apiserver.go:52] "Watching apiserver" Jul 05 13:15:17.866154 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:17.866129 2578 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 05 13:15:17.867877 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:17.867849 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/kube-apiserver-proxy-ip-10-0-130-191.ec2.internal","openshift-dns/node-resolver-btxlt","openshift-multus/multus-lq7qp","openshift-network-operator/iptables-alerter-b2c6n","openshift-ovn-kubernetes/ovnkube-node-656m6","kube-system/global-pull-secret-syncer-72444","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mkr9r","openshift-cluster-node-tuning-operator/tuned-fds9k","openshift-image-registry/node-ca-g7nxb","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-191.ec2.internal","openshift-multus/multus-additional-cni-plugins-7n8f2","openshift-multus/network-metrics-daemon-dxwj6","openshift-network-diagnostics/network-check-target-jldjr","kube-system/konnectivity-agent-kpmmd"] Jul 05 13:15:17.870871 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:17.870477 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mkr9r" Jul 05 13:15:17.872708 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:17.872684 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jldjr" Jul 05 13:15:17.872795 ip-10-0-130-191 kubenswrapper[2578]: E0705 13:15:17.872774 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jldjr" podUID="50dc9dcd-a00c-4f85-b3dc-dbac06316c3e" Jul 05 13:15:17.874446 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:17.873393 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 05 13:15:17.874446 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:17.873407 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-qxvhb\"" Jul 05 13:15:17.874446 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:17.873433 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 05 13:15:17.874446 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:17.873755 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 05 13:15:17.875818 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:17.875801 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-lq7qp" Jul 05 13:15:17.876427 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:17.876412 2578 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 05 13:15:17.877953 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:17.877905 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-b2c6n" Jul 05 13:15:17.879073 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:17.878884 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 05 13:15:17.879073 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:17.879056 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 05 13:15:17.879256 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:17.879241 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 05 13:15:17.879454 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:17.879433 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-6xzc8\"" Jul 05 13:15:17.879693 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:17.879678 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 05 13:15:17.880908 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:17.880889 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 05 13:15:17.881122 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:17.881106 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 05 13:15:17.881314 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:17.881299 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-2h8nt\"" Jul 05 13:15:17.882218 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:17.881681 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 05 13:15:17.882218 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:17.881698 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-656m6" Jul 05 13:15:17.883762 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:17.883743 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-72444" Jul 05 13:15:17.883827 ip-10-0-130-191 kubenswrapper[2578]: E0705 13:15:17.883808 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-72444" podUID="4ff28158-61b2-4cb9-8b92-2ac0793c23bb" Jul 05 13:15:17.884138 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:17.884119 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 05 13:15:17.884449 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:17.884427 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/c3c5ffe7-eb78-4b28-abb6-106dd0bbb008-host-slash\") pod \"iptables-alerter-b2c6n\" (UID: \"c3c5ffe7-eb78-4b28-abb6-106dd0bbb008\") " pod="openshift-network-operator/iptables-alerter-b2c6n" Jul 05 13:15:17.884518 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:17.884464 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/eda62bac-3d03-4f52-b055-a77147e9ee97-host-run-netns\") pod \"multus-lq7qp\" (UID: \"eda62bac-3d03-4f52-b055-a77147e9ee97\") " pod="openshift-multus/multus-lq7qp" Jul 05 13:15:17.884518 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:17.884472 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 05 13:15:17.884518 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:17.884487 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xk9dk\" (UniqueName: \"kubernetes.io/projected/eda62bac-3d03-4f52-b055-a77147e9ee97-kube-api-access-xk9dk\") pod \"multus-lq7qp\" (UID: \"eda62bac-3d03-4f52-b055-a77147e9ee97\") " pod="openshift-multus/multus-lq7qp" Jul 05 13:15:17.884518 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:17.884509 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/eda62bac-3d03-4f52-b055-a77147e9ee97-system-cni-dir\") pod \"multus-lq7qp\" (UID: \"eda62bac-3d03-4f52-b055-a77147e9ee97\") " pod="openshift-multus/multus-lq7qp" Jul 05 13:15:17.884756 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:17.884535 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/5386321b-a286-4e02-9b76-b81ed7a920ea-device-dir\") pod \"aws-ebs-csi-driver-node-mkr9r\" (UID: \"5386321b-a286-4e02-9b76-b81ed7a920ea\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mkr9r" Jul 05 13:15:17.884756 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:17.884557 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/5386321b-a286-4e02-9b76-b81ed7a920ea-sys-fs\") pod \"aws-ebs-csi-driver-node-mkr9r\" (UID: \"5386321b-a286-4e02-9b76-b81ed7a920ea\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mkr9r" Jul 05 13:15:17.884756 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:17.884597 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/5386321b-a286-4e02-9b76-b81ed7a920ea-registration-dir\") pod \"aws-ebs-csi-driver-node-mkr9r\" (UID: \"5386321b-a286-4e02-9b76-b81ed7a920ea\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mkr9r" Jul 05 13:15:17.884756 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:17.884632 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/eda62bac-3d03-4f52-b055-a77147e9ee97-cni-binary-copy\") pod \"multus-lq7qp\" (UID: \"eda62bac-3d03-4f52-b055-a77147e9ee97\") " pod="openshift-multus/multus-lq7qp" Jul 05 13:15:17.884756 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:17.884655 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/eda62bac-3d03-4f52-b055-a77147e9ee97-host-run-k8s-cni-cncf-io\") pod \"multus-lq7qp\" (UID: \"eda62bac-3d03-4f52-b055-a77147e9ee97\") " pod="openshift-multus/multus-lq7qp" Jul 05 13:15:17.884756 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:17.884678 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 05 13:15:17.885032 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:17.884677 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/eda62bac-3d03-4f52-b055-a77147e9ee97-host-var-lib-cni-multus\") pod \"multus-lq7qp\" (UID: \"eda62bac-3d03-4f52-b055-a77147e9ee97\") " pod="openshift-multus/multus-lq7qp" Jul 05 13:15:17.885032 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:17.884832 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/eda62bac-3d03-4f52-b055-a77147e9ee97-multus-conf-dir\") pod \"multus-lq7qp\" (UID: \"eda62bac-3d03-4f52-b055-a77147e9ee97\") " pod="openshift-multus/multus-lq7qp" Jul 05 13:15:17.885032 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:17.884844 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 05 13:15:17.885032 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:17.884855 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/eda62bac-3d03-4f52-b055-a77147e9ee97-host-run-multus-certs\") pod \"multus-lq7qp\" (UID: \"eda62bac-3d03-4f52-b055-a77147e9ee97\") " pod="openshift-multus/multus-lq7qp" Jul 05 13:15:17.885032 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:17.884892 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/5386321b-a286-4e02-9b76-b81ed7a920ea-kubelet-dir\") pod \"aws-ebs-csi-driver-node-mkr9r\" (UID: \"5386321b-a286-4e02-9b76-b81ed7a920ea\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mkr9r" Jul 05 13:15:17.885032 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:17.884914 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/eda62bac-3d03-4f52-b055-a77147e9ee97-hostroot\") pod \"multus-lq7qp\" (UID: \"eda62bac-3d03-4f52-b055-a77147e9ee97\") " pod="openshift-multus/multus-lq7qp" Jul 05 13:15:17.885032 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:17.884934 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/eda62bac-3d03-4f52-b055-a77147e9ee97-multus-daemon-config\") pod \"multus-lq7qp\" (UID: \"eda62bac-3d03-4f52-b055-a77147e9ee97\") " pod="openshift-multus/multus-lq7qp" Jul 05 13:15:17.885032 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:17.884962 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-czl8p\" (UniqueName: \"kubernetes.io/projected/50dc9dcd-a00c-4f85-b3dc-dbac06316c3e-kube-api-access-czl8p\") pod \"network-check-target-jldjr\" (UID: \"50dc9dcd-a00c-4f85-b3dc-dbac06316c3e\") " pod="openshift-network-diagnostics/network-check-target-jldjr" Jul 05 13:15:17.885032 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:17.884985 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/eda62bac-3d03-4f52-b055-a77147e9ee97-cnibin\") pod \"multus-lq7qp\" (UID: \"eda62bac-3d03-4f52-b055-a77147e9ee97\") " pod="openshift-multus/multus-lq7qp" Jul 05 13:15:17.885032 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:17.885006 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/eda62bac-3d03-4f52-b055-a77147e9ee97-multus-socket-dir-parent\") pod \"multus-lq7qp\" (UID: \"eda62bac-3d03-4f52-b055-a77147e9ee97\") " pod="openshift-multus/multus-lq7qp" Jul 05 13:15:17.885032 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:17.885026 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/eda62bac-3d03-4f52-b055-a77147e9ee97-host-var-lib-kubelet\") pod \"multus-lq7qp\" (UID: \"eda62bac-3d03-4f52-b055-a77147e9ee97\") " pod="openshift-multus/multus-lq7qp" Jul 05 13:15:17.885469 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:17.885048 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/c3c5ffe7-eb78-4b28-abb6-106dd0bbb008-iptables-alerter-script\") pod \"iptables-alerter-b2c6n\" (UID: \"c3c5ffe7-eb78-4b28-abb6-106dd0bbb008\") " pod="openshift-network-operator/iptables-alerter-b2c6n" Jul 05 13:15:17.885469 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:17.885079 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/eda62bac-3d03-4f52-b055-a77147e9ee97-host-var-lib-cni-bin\") pod \"multus-lq7qp\" (UID: \"eda62bac-3d03-4f52-b055-a77147e9ee97\") " pod="openshift-multus/multus-lq7qp" Jul 05 13:15:17.885469 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:17.885108 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/5386321b-a286-4e02-9b76-b81ed7a920ea-socket-dir\") pod \"aws-ebs-csi-driver-node-mkr9r\" (UID: \"5386321b-a286-4e02-9b76-b81ed7a920ea\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mkr9r" Jul 05 13:15:17.885469 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:17.885129 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/5386321b-a286-4e02-9b76-b81ed7a920ea-etc-selinux\") pod \"aws-ebs-csi-driver-node-mkr9r\" (UID: \"5386321b-a286-4e02-9b76-b81ed7a920ea\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mkr9r" Jul 05 13:15:17.885469 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:17.885166 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/eda62bac-3d03-4f52-b055-a77147e9ee97-os-release\") pod \"multus-lq7qp\" (UID: \"eda62bac-3d03-4f52-b055-a77147e9ee97\") " pod="openshift-multus/multus-lq7qp" Jul 05 13:15:17.885469 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:17.885202 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5g68t\" (UniqueName: \"kubernetes.io/projected/c3c5ffe7-eb78-4b28-abb6-106dd0bbb008-kube-api-access-5g68t\") pod \"iptables-alerter-b2c6n\" (UID: \"c3c5ffe7-eb78-4b28-abb6-106dd0bbb008\") " pod="openshift-network-operator/iptables-alerter-b2c6n" Jul 05 13:15:17.885469 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:17.885253 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kdsgp\" (UniqueName: \"kubernetes.io/projected/5386321b-a286-4e02-9b76-b81ed7a920ea-kube-api-access-kdsgp\") pod \"aws-ebs-csi-driver-node-mkr9r\" (UID: \"5386321b-a286-4e02-9b76-b81ed7a920ea\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mkr9r" Jul 05 13:15:17.885469 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:17.885281 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/eda62bac-3d03-4f52-b055-a77147e9ee97-multus-cni-dir\") pod \"multus-lq7qp\" (UID: \"eda62bac-3d03-4f52-b055-a77147e9ee97\") " pod="openshift-multus/multus-lq7qp" Jul 05 13:15:17.885469 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:17.885305 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/eda62bac-3d03-4f52-b055-a77147e9ee97-etc-kubernetes\") pod \"multus-lq7qp\" (UID: \"eda62bac-3d03-4f52-b055-a77147e9ee97\") " pod="openshift-multus/multus-lq7qp" Jul 05 13:15:17.885891 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:17.885760 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 05 13:15:17.886265 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:17.886225 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 05 13:15:17.887266 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:17.886858 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-g7nxb" Jul 05 13:15:17.887266 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:17.886974 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-2jx2x\"" Jul 05 13:15:17.888105 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:17.888078 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-kpmmd" Jul 05 13:15:17.888904 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:17.888882 2578 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 05 13:15:17.889450 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:17.889433 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 05 13:15:17.889658 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:17.889634 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 05 13:15:17.889749 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:17.889691 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 05 13:15:17.894035 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:17.891851 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-k9dsn\"" Jul 05 13:15:17.894035 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:17.892304 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-btxlt" Jul 05 13:15:17.894035 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:17.893442 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-4pwzs\"" Jul 05 13:15:17.894035 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:17.893467 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 05 13:15:17.894035 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:17.894031 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 05 13:15:17.896723 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:17.896025 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 05 13:15:17.896876 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:17.896851 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-gz97x\"" Jul 05 13:15:17.897671 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:17.897657 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 05 13:15:17.899403 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:17.899383 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-fds9k" Jul 05 13:15:17.900821 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:17.900806 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-7n8f2" Jul 05 13:15:17.902429 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:17.902394 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 05 13:15:17.902867 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:17.902844 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 05 13:15:17.903499 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:17.903484 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-qcbg2\"" Jul 05 13:15:17.903705 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:17.903682 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 05 13:15:17.903773 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:17.903689 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dxwj6" Jul 05 13:15:17.903861 ip-10-0-130-191 kubenswrapper[2578]: E0705 13:15:17.903837 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dxwj6" podUID="5ad92660-5823-4f14-adcc-763d06385f45" Jul 05 13:15:17.905188 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:17.905122 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 05 13:15:17.909149 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:17.909133 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-pnqq4\"" Jul 05 13:15:17.914306 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:17.914289 2578 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-6dsdd" Jul 05 13:15:17.919763 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:17.919743 2578 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-6dsdd" Jul 05 13:15:17.976941 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:17.976878 2578 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 05 13:15:17.986131 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:17.986106 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/7bdbd204-0fb7-4ebf-af05-432c370a814b-host-slash\") pod \"ovnkube-node-656m6\" (UID: \"7bdbd204-0fb7-4ebf-af05-432c370a814b\") " pod="openshift-ovn-kubernetes/ovnkube-node-656m6" Jul 05 13:15:17.986236 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:17.986134 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/7bdbd204-0fb7-4ebf-af05-432c370a814b-run-systemd\") pod \"ovnkube-node-656m6\" (UID: \"7bdbd204-0fb7-4ebf-af05-432c370a814b\") " pod="openshift-ovn-kubernetes/ovnkube-node-656m6" Jul 05 13:15:17.986236 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:17.986151 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b9962dff-a80f-4ece-bb92-b9761bcf6334-lib-modules\") pod \"tuned-fds9k\" (UID: \"b9962dff-a80f-4ece-bb92-b9761bcf6334\") " pod="openshift-cluster-node-tuning-operator/tuned-fds9k" Jul 05 13:15:17.986236 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:17.986174 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/b14dc74b-da46-4d14-8fd6-fd9357838fe2-serviceca\") pod \"node-ca-g7nxb\" (UID: \"b14dc74b-da46-4d14-8fd6-fd9357838fe2\") " pod="openshift-image-registry/node-ca-g7nxb" Jul 05 13:15:17.986236 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:17.986225 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/eda62bac-3d03-4f52-b055-a77147e9ee97-cnibin\") pod \"multus-lq7qp\" (UID: \"eda62bac-3d03-4f52-b055-a77147e9ee97\") " pod="openshift-multus/multus-lq7qp" Jul 05 13:15:17.986430 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:17.986263 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/c3c5ffe7-eb78-4b28-abb6-106dd0bbb008-iptables-alerter-script\") pod \"iptables-alerter-b2c6n\" (UID: \"c3c5ffe7-eb78-4b28-abb6-106dd0bbb008\") " pod="openshift-network-operator/iptables-alerter-b2c6n" Jul 05 13:15:17.986430 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:17.986293 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/7bdbd204-0fb7-4ebf-af05-432c370a814b-ovnkube-script-lib\") pod \"ovnkube-node-656m6\" (UID: \"7bdbd204-0fb7-4ebf-af05-432c370a814b\") " pod="openshift-ovn-kubernetes/ovnkube-node-656m6" Jul 05 13:15:17.986430 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:17.986318 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b9962dff-a80f-4ece-bb92-b9761bcf6334-tmp\") pod \"tuned-fds9k\" (UID: \"b9962dff-a80f-4ece-bb92-b9761bcf6334\") " pod="openshift-cluster-node-tuning-operator/tuned-fds9k" Jul 05 13:15:17.986430 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:17.986337 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/eda62bac-3d03-4f52-b055-a77147e9ee97-cnibin\") pod \"multus-lq7qp\" (UID: \"eda62bac-3d03-4f52-b055-a77147e9ee97\") " pod="openshift-multus/multus-lq7qp" Jul 05 13:15:17.986430 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:17.986342 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b14dc74b-da46-4d14-8fd6-fd9357838fe2-host\") pod \"node-ca-g7nxb\" (UID: \"b14dc74b-da46-4d14-8fd6-fd9357838fe2\") " pod="openshift-image-registry/node-ca-g7nxb" Jul 05 13:15:17.986430 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:17.986369 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/eda62bac-3d03-4f52-b055-a77147e9ee97-host-var-lib-cni-bin\") pod \"multus-lq7qp\" (UID: \"eda62bac-3d03-4f52-b055-a77147e9ee97\") " pod="openshift-multus/multus-lq7qp" Jul 05 13:15:17.986430 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:17.986392 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/5386321b-a286-4e02-9b76-b81ed7a920ea-socket-dir\") pod \"aws-ebs-csi-driver-node-mkr9r\" (UID: \"5386321b-a286-4e02-9b76-b81ed7a920ea\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mkr9r" Jul 05 13:15:17.986430 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:17.986416 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/5386321b-a286-4e02-9b76-b81ed7a920ea-etc-selinux\") pod \"aws-ebs-csi-driver-node-mkr9r\" (UID: \"5386321b-a286-4e02-9b76-b81ed7a920ea\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mkr9r" Jul 05 13:15:17.986876 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:17.986438 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/eda62bac-3d03-4f52-b055-a77147e9ee97-os-release\") pod \"multus-lq7qp\" (UID: \"eda62bac-3d03-4f52-b055-a77147e9ee97\") " pod="openshift-multus/multus-lq7qp" Jul 05 13:15:17.986876 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:17.986478 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5g68t\" (UniqueName: \"kubernetes.io/projected/c3c5ffe7-eb78-4b28-abb6-106dd0bbb008-kube-api-access-5g68t\") pod \"iptables-alerter-b2c6n\" (UID: \"c3c5ffe7-eb78-4b28-abb6-106dd0bbb008\") " pod="openshift-network-operator/iptables-alerter-b2c6n" Jul 05 13:15:17.986876 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:17.986503 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/545d8134-4ad2-4c62-b302-64b8e10bc493-system-cni-dir\") pod \"multus-additional-cni-plugins-7n8f2\" (UID: \"545d8134-4ad2-4c62-b302-64b8e10bc493\") " pod="openshift-multus/multus-additional-cni-plugins-7n8f2" Jul 05 13:15:17.986876 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:17.986526 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/545d8134-4ad2-4c62-b302-64b8e10bc493-cnibin\") pod \"multus-additional-cni-plugins-7n8f2\" (UID: \"545d8134-4ad2-4c62-b302-64b8e10bc493\") " pod="openshift-multus/multus-additional-cni-plugins-7n8f2" Jul 05 13:15:17.986876 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:17.986549 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/545d8134-4ad2-4c62-b302-64b8e10bc493-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-7n8f2\" (UID: \"545d8134-4ad2-4c62-b302-64b8e10bc493\") " pod="openshift-multus/multus-additional-cni-plugins-7n8f2" Jul 05 13:15:17.986876 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:17.986595 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kdsgp\" (UniqueName: \"kubernetes.io/projected/5386321b-a286-4e02-9b76-b81ed7a920ea-kube-api-access-kdsgp\") pod \"aws-ebs-csi-driver-node-mkr9r\" (UID: \"5386321b-a286-4e02-9b76-b81ed7a920ea\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mkr9r" Jul 05 13:15:17.986876 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:17.986632 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/eda62bac-3d03-4f52-b055-a77147e9ee97-multus-cni-dir\") pod \"multus-lq7qp\" (UID: \"eda62bac-3d03-4f52-b055-a77147e9ee97\") " pod="openshift-multus/multus-lq7qp" Jul 05 13:15:17.986876 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:17.986653 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/eda62bac-3d03-4f52-b055-a77147e9ee97-etc-kubernetes\") pod \"multus-lq7qp\" (UID: \"eda62bac-3d03-4f52-b055-a77147e9ee97\") " pod="openshift-multus/multus-lq7qp" Jul 05 13:15:17.986876 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:17.986676 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gs6jp\" (UniqueName: \"kubernetes.io/projected/5ad92660-5823-4f14-adcc-763d06385f45-kube-api-access-gs6jp\") pod \"network-metrics-daemon-dxwj6\" (UID: \"5ad92660-5823-4f14-adcc-763d06385f45\") " pod="openshift-multus/network-metrics-daemon-dxwj6" Jul 05 13:15:17.986876 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:17.986698 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/7bdbd204-0fb7-4ebf-af05-432c370a814b-host-run-netns\") pod \"ovnkube-node-656m6\" (UID: \"7bdbd204-0fb7-4ebf-af05-432c370a814b\") " pod="openshift-ovn-kubernetes/ovnkube-node-656m6" Jul 05 13:15:17.986876 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:17.986720 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c01b0489-2ef4-4fbe-9f9e-1870bc580ee9-tmp-dir\") pod \"node-resolver-btxlt\" (UID: \"c01b0489-2ef4-4fbe-9f9e-1870bc580ee9\") " pod="openshift-dns/node-resolver-btxlt" Jul 05 13:15:17.986876 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:17.986744 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5ad92660-5823-4f14-adcc-763d06385f45-metrics-certs\") pod \"network-metrics-daemon-dxwj6\" (UID: \"5ad92660-5823-4f14-adcc-763d06385f45\") " pod="openshift-multus/network-metrics-daemon-dxwj6" Jul 05 13:15:17.986876 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:17.986774 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/b9962dff-a80f-4ece-bb92-b9761bcf6334-etc-modprobe-d\") pod \"tuned-fds9k\" (UID: \"b9962dff-a80f-4ece-bb92-b9761bcf6334\") " pod="openshift-cluster-node-tuning-operator/tuned-fds9k" Jul 05 13:15:17.986876 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:17.986804 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/b9962dff-a80f-4ece-bb92-b9761bcf6334-etc-systemd\") pod \"tuned-fds9k\" (UID: \"b9962dff-a80f-4ece-bb92-b9761bcf6334\") " pod="openshift-cluster-node-tuning-operator/tuned-fds9k" Jul 05 13:15:17.986876 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:17.986828 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-skvcw\" (UniqueName: \"kubernetes.io/projected/b14dc74b-da46-4d14-8fd6-fd9357838fe2-kube-api-access-skvcw\") pod \"node-ca-g7nxb\" (UID: \"b14dc74b-da46-4d14-8fd6-fd9357838fe2\") " pod="openshift-image-registry/node-ca-g7nxb" Jul 05 13:15:17.986876 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:17.986863 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/eda62bac-3d03-4f52-b055-a77147e9ee97-host-run-netns\") pod \"multus-lq7qp\" (UID: \"eda62bac-3d03-4f52-b055-a77147e9ee97\") " pod="openshift-multus/multus-lq7qp" Jul 05 13:15:17.987644 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:17.986885 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xk9dk\" (UniqueName: \"kubernetes.io/projected/eda62bac-3d03-4f52-b055-a77147e9ee97-kube-api-access-xk9dk\") pod \"multus-lq7qp\" (UID: \"eda62bac-3d03-4f52-b055-a77147e9ee97\") " pod="openshift-multus/multus-lq7qp" Jul 05 13:15:17.987644 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:17.986908 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7bdbd204-0fb7-4ebf-af05-432c370a814b-host-run-ovn-kubernetes\") pod \"ovnkube-node-656m6\" (UID: \"7bdbd204-0fb7-4ebf-af05-432c370a814b\") " pod="openshift-ovn-kubernetes/ovnkube-node-656m6" Jul 05 13:15:17.987644 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:17.986936 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/b9962dff-a80f-4ece-bb92-b9761bcf6334-etc-sysctl-d\") pod \"tuned-fds9k\" (UID: \"b9962dff-a80f-4ece-bb92-b9761bcf6334\") " pod="openshift-cluster-node-tuning-operator/tuned-fds9k" Jul 05 13:15:17.987644 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:17.987021 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/eda62bac-3d03-4f52-b055-a77147e9ee97-multus-cni-dir\") pod \"multus-lq7qp\" (UID: \"eda62bac-3d03-4f52-b055-a77147e9ee97\") " pod="openshift-multus/multus-lq7qp" Jul 05 13:15:17.987644 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:17.987067 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/eda62bac-3d03-4f52-b055-a77147e9ee97-etc-kubernetes\") pod \"multus-lq7qp\" (UID: \"eda62bac-3d03-4f52-b055-a77147e9ee97\") " pod="openshift-multus/multus-lq7qp" Jul 05 13:15:17.987644 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:17.987231 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/4ff28158-61b2-4cb9-8b92-2ac0793c23bb-dbus\") pod \"global-pull-secret-syncer-72444\" (UID: \"4ff28158-61b2-4cb9-8b92-2ac0793c23bb\") " pod="kube-system/global-pull-secret-syncer-72444" Jul 05 13:15:17.987644 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:17.986906 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/c3c5ffe7-eb78-4b28-abb6-106dd0bbb008-iptables-alerter-script\") pod \"iptables-alerter-b2c6n\" (UID: \"c3c5ffe7-eb78-4b28-abb6-106dd0bbb008\") " pod="openshift-network-operator/iptables-alerter-b2c6n" Jul 05 13:15:17.987644 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:17.987317 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/eda62bac-3d03-4f52-b055-a77147e9ee97-host-var-lib-cni-bin\") pod \"multus-lq7qp\" (UID: \"eda62bac-3d03-4f52-b055-a77147e9ee97\") " pod="openshift-multus/multus-lq7qp" Jul 05 13:15:17.987644 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:17.987411 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/5386321b-a286-4e02-9b76-b81ed7a920ea-socket-dir\") pod \"aws-ebs-csi-driver-node-mkr9r\" (UID: \"5386321b-a286-4e02-9b76-b81ed7a920ea\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mkr9r" Jul 05 13:15:17.987644 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:17.987423 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/5386321b-a286-4e02-9b76-b81ed7a920ea-etc-selinux\") pod \"aws-ebs-csi-driver-node-mkr9r\" (UID: \"5386321b-a286-4e02-9b76-b81ed7a920ea\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mkr9r" Jul 05 13:15:17.987644 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:17.987459 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/eda62bac-3d03-4f52-b055-a77147e9ee97-system-cni-dir\") pod \"multus-lq7qp\" (UID: \"eda62bac-3d03-4f52-b055-a77147e9ee97\") " pod="openshift-multus/multus-lq7qp" Jul 05 13:15:17.987644 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:17.987488 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/5386321b-a286-4e02-9b76-b81ed7a920ea-device-dir\") pod \"aws-ebs-csi-driver-node-mkr9r\" (UID: \"5386321b-a286-4e02-9b76-b81ed7a920ea\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mkr9r" Jul 05 13:15:17.987644 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:17.987516 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8hghn\" (UniqueName: \"kubernetes.io/projected/545d8134-4ad2-4c62-b302-64b8e10bc493-kube-api-access-8hghn\") pod \"multus-additional-cni-plugins-7n8f2\" (UID: \"545d8134-4ad2-4c62-b302-64b8e10bc493\") " pod="openshift-multus/multus-additional-cni-plugins-7n8f2" Jul 05 13:15:17.987644 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:17.987536 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/b9962dff-a80f-4ece-bb92-b9761bcf6334-etc-sysconfig\") pod \"tuned-fds9k\" (UID: \"b9962dff-a80f-4ece-bb92-b9761bcf6334\") " pod="openshift-cluster-node-tuning-operator/tuned-fds9k" Jul 05 13:15:17.987644 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:17.987552 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/eda62bac-3d03-4f52-b055-a77147e9ee97-cni-binary-copy\") pod \"multus-lq7qp\" (UID: \"eda62bac-3d03-4f52-b055-a77147e9ee97\") " pod="openshift-multus/multus-lq7qp" Jul 05 13:15:17.987644 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:17.987637 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/eda62bac-3d03-4f52-b055-a77147e9ee97-os-release\") pod \"multus-lq7qp\" (UID: \"eda62bac-3d03-4f52-b055-a77147e9ee97\") " pod="openshift-multus/multus-lq7qp" Jul 05 13:15:17.988376 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:17.987673 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/eda62bac-3d03-4f52-b055-a77147e9ee97-host-run-k8s-cni-cncf-io\") pod \"multus-lq7qp\" (UID: \"eda62bac-3d03-4f52-b055-a77147e9ee97\") " pod="openshift-multus/multus-lq7qp" Jul 05 13:15:17.988376 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:17.987704 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/eda62bac-3d03-4f52-b055-a77147e9ee97-host-var-lib-cni-multus\") pod \"multus-lq7qp\" (UID: \"eda62bac-3d03-4f52-b055-a77147e9ee97\") " pod="openshift-multus/multus-lq7qp" Jul 05 13:15:17.988376 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:17.987747 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/eda62bac-3d03-4f52-b055-a77147e9ee97-host-run-multus-certs\") pod \"multus-lq7qp\" (UID: \"eda62bac-3d03-4f52-b055-a77147e9ee97\") " pod="openshift-multus/multus-lq7qp" Jul 05 13:15:17.988376 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:17.987774 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/545d8134-4ad2-4c62-b302-64b8e10bc493-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-7n8f2\" (UID: \"545d8134-4ad2-4c62-b302-64b8e10bc493\") " pod="openshift-multus/multus-additional-cni-plugins-7n8f2" Jul 05 13:15:17.988376 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:17.987799 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/7bdbd204-0fb7-4ebf-af05-432c370a814b-env-overrides\") pod \"ovnkube-node-656m6\" (UID: \"7bdbd204-0fb7-4ebf-af05-432c370a814b\") " pod="openshift-ovn-kubernetes/ovnkube-node-656m6" Jul 05 13:15:17.988376 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:17.987843 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/b9962dff-a80f-4ece-bb92-b9761bcf6334-etc-sysctl-conf\") pod \"tuned-fds9k\" (UID: \"b9962dff-a80f-4ece-bb92-b9761bcf6334\") " pod="openshift-cluster-node-tuning-operator/tuned-fds9k" Jul 05 13:15:17.988376 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:17.987866 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wlgrd\" (UniqueName: \"kubernetes.io/projected/b9962dff-a80f-4ece-bb92-b9761bcf6334-kube-api-access-wlgrd\") pod \"tuned-fds9k\" (UID: \"b9962dff-a80f-4ece-bb92-b9761bcf6334\") " pod="openshift-cluster-node-tuning-operator/tuned-fds9k" Jul 05 13:15:17.988376 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:17.987901 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/eda62bac-3d03-4f52-b055-a77147e9ee97-hostroot\") pod \"multus-lq7qp\" (UID: \"eda62bac-3d03-4f52-b055-a77147e9ee97\") " pod="openshift-multus/multus-lq7qp" Jul 05 13:15:17.988376 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:17.987956 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/e8e7535a-8f4e-4fd6-95d2-4cb8888b284b-agent-certs\") pod \"konnectivity-agent-kpmmd\" (UID: \"e8e7535a-8f4e-4fd6-95d2-4cb8888b284b\") " pod="kube-system/konnectivity-agent-kpmmd" Jul 05 13:15:17.988376 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:17.987980 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/545d8134-4ad2-4c62-b302-64b8e10bc493-os-release\") pod \"multus-additional-cni-plugins-7n8f2\" (UID: \"545d8134-4ad2-4c62-b302-64b8e10bc493\") " pod="openshift-multus/multus-additional-cni-plugins-7n8f2" Jul 05 13:15:17.988376 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:17.988003 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/7bdbd204-0fb7-4ebf-af05-432c370a814b-run-ovn\") pod \"ovnkube-node-656m6\" (UID: \"7bdbd204-0fb7-4ebf-af05-432c370a814b\") " pod="openshift-ovn-kubernetes/ovnkube-node-656m6" Jul 05 13:15:17.988376 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:17.988025 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wdf5k\" (UniqueName: \"kubernetes.io/projected/7bdbd204-0fb7-4ebf-af05-432c370a814b-kube-api-access-wdf5k\") pod \"ovnkube-node-656m6\" (UID: \"7bdbd204-0fb7-4ebf-af05-432c370a814b\") " pod="openshift-ovn-kubernetes/ovnkube-node-656m6" Jul 05 13:15:17.988376 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:17.988047 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/b9962dff-a80f-4ece-bb92-b9761bcf6334-etc-tuned\") pod \"tuned-fds9k\" (UID: \"b9962dff-a80f-4ece-bb92-b9761bcf6334\") " pod="openshift-cluster-node-tuning-operator/tuned-fds9k" Jul 05 13:15:17.988376 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:17.988073 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-czl8p\" (UniqueName: \"kubernetes.io/projected/50dc9dcd-a00c-4f85-b3dc-dbac06316c3e-kube-api-access-czl8p\") pod \"network-check-target-jldjr\" (UID: \"50dc9dcd-a00c-4f85-b3dc-dbac06316c3e\") " pod="openshift-network-diagnostics/network-check-target-jldjr" Jul 05 13:15:17.988376 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:17.988096 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/eda62bac-3d03-4f52-b055-a77147e9ee97-multus-socket-dir-parent\") pod \"multus-lq7qp\" (UID: \"eda62bac-3d03-4f52-b055-a77147e9ee97\") " pod="openshift-multus/multus-lq7qp" Jul 05 13:15:17.988376 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:17.988121 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/eda62bac-3d03-4f52-b055-a77147e9ee97-host-var-lib-kubelet\") pod \"multus-lq7qp\" (UID: \"eda62bac-3d03-4f52-b055-a77147e9ee97\") " pod="openshift-multus/multus-lq7qp" Jul 05 13:15:17.988376 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:17.988152 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/eda62bac-3d03-4f52-b055-a77147e9ee97-host-run-netns\") pod \"multus-lq7qp\" (UID: \"eda62bac-3d03-4f52-b055-a77147e9ee97\") " pod="openshift-multus/multus-lq7qp" Jul 05 13:15:17.989246 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:17.988178 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7bdbd204-0fb7-4ebf-af05-432c370a814b-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-656m6\" (UID: \"7bdbd204-0fb7-4ebf-af05-432c370a814b\") " pod="openshift-ovn-kubernetes/ovnkube-node-656m6" Jul 05 13:15:17.989246 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:17.988185 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/eda62bac-3d03-4f52-b055-a77147e9ee97-hostroot\") pod \"multus-lq7qp\" (UID: \"eda62bac-3d03-4f52-b055-a77147e9ee97\") " pod="openshift-multus/multus-lq7qp" Jul 05 13:15:17.989246 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:17.988213 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/eda62bac-3d03-4f52-b055-a77147e9ee97-system-cni-dir\") pod \"multus-lq7qp\" (UID: \"eda62bac-3d03-4f52-b055-a77147e9ee97\") " pod="openshift-multus/multus-lq7qp" Jul 05 13:15:17.989246 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:17.988112 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/eda62bac-3d03-4f52-b055-a77147e9ee97-cni-binary-copy\") pod \"multus-lq7qp\" (UID: \"eda62bac-3d03-4f52-b055-a77147e9ee97\") " pod="openshift-multus/multus-lq7qp" Jul 05 13:15:17.989246 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:17.988258 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/eda62bac-3d03-4f52-b055-a77147e9ee97-host-var-lib-kubelet\") pod \"multus-lq7qp\" (UID: \"eda62bac-3d03-4f52-b055-a77147e9ee97\") " pod="openshift-multus/multus-lq7qp" Jul 05 13:15:17.989246 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:17.988277 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/eda62bac-3d03-4f52-b055-a77147e9ee97-multus-socket-dir-parent\") pod \"multus-lq7qp\" (UID: \"eda62bac-3d03-4f52-b055-a77147e9ee97\") " pod="openshift-multus/multus-lq7qp" Jul 05 13:15:17.989246 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:17.988281 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-985mk\" (UniqueName: \"kubernetes.io/projected/c01b0489-2ef4-4fbe-9f9e-1870bc580ee9-kube-api-access-985mk\") pod \"node-resolver-btxlt\" (UID: \"c01b0489-2ef4-4fbe-9f9e-1870bc580ee9\") " pod="openshift-dns/node-resolver-btxlt" Jul 05 13:15:17.989246 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:17.988315 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b9962dff-a80f-4ece-bb92-b9761bcf6334-etc-kubernetes\") pod \"tuned-fds9k\" (UID: \"b9962dff-a80f-4ece-bb92-b9761bcf6334\") " pod="openshift-cluster-node-tuning-operator/tuned-fds9k" Jul 05 13:15:17.989246 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:17.988326 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/5386321b-a286-4e02-9b76-b81ed7a920ea-device-dir\") pod \"aws-ebs-csi-driver-node-mkr9r\" (UID: \"5386321b-a286-4e02-9b76-b81ed7a920ea\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mkr9r" Jul 05 13:15:17.989246 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:17.988340 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/4ff28158-61b2-4cb9-8b92-2ac0793c23bb-kubelet-config\") pod \"global-pull-secret-syncer-72444\" (UID: \"4ff28158-61b2-4cb9-8b92-2ac0793c23bb\") " pod="kube-system/global-pull-secret-syncer-72444" Jul 05 13:15:17.989246 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:17.988363 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/eda62bac-3d03-4f52-b055-a77147e9ee97-host-run-k8s-cni-cncf-io\") pod \"multus-lq7qp\" (UID: \"eda62bac-3d03-4f52-b055-a77147e9ee97\") " pod="openshift-multus/multus-lq7qp" Jul 05 13:15:17.989246 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:17.988408 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/eda62bac-3d03-4f52-b055-a77147e9ee97-host-var-lib-cni-multus\") pod \"multus-lq7qp\" (UID: \"eda62bac-3d03-4f52-b055-a77147e9ee97\") " pod="openshift-multus/multus-lq7qp" Jul 05 13:15:17.989246 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:17.988448 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/545d8134-4ad2-4c62-b302-64b8e10bc493-cni-binary-copy\") pod \"multus-additional-cni-plugins-7n8f2\" (UID: \"545d8134-4ad2-4c62-b302-64b8e10bc493\") " pod="openshift-multus/multus-additional-cni-plugins-7n8f2" Jul 05 13:15:17.989246 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:17.988447 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/eda62bac-3d03-4f52-b055-a77147e9ee97-host-run-multus-certs\") pod \"multus-lq7qp\" (UID: \"eda62bac-3d03-4f52-b055-a77147e9ee97\") " pod="openshift-multus/multus-lq7qp" Jul 05 13:15:17.989246 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:17.988496 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/7bdbd204-0fb7-4ebf-af05-432c370a814b-systemd-units\") pod \"ovnkube-node-656m6\" (UID: \"7bdbd204-0fb7-4ebf-af05-432c370a814b\") " pod="openshift-ovn-kubernetes/ovnkube-node-656m6" Jul 05 13:15:17.989246 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:17.988529 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7bdbd204-0fb7-4ebf-af05-432c370a814b-etc-openvswitch\") pod \"ovnkube-node-656m6\" (UID: \"7bdbd204-0fb7-4ebf-af05-432c370a814b\") " pod="openshift-ovn-kubernetes/ovnkube-node-656m6" Jul 05 13:15:17.989246 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:17.988554 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7bdbd204-0fb7-4ebf-af05-432c370a814b-run-openvswitch\") pod \"ovnkube-node-656m6\" (UID: \"7bdbd204-0fb7-4ebf-af05-432c370a814b\") " pod="openshift-ovn-kubernetes/ovnkube-node-656m6" Jul 05 13:15:17.989939 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:17.988607 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b9962dff-a80f-4ece-bb92-b9761bcf6334-sys\") pod \"tuned-fds9k\" (UID: \"b9962dff-a80f-4ece-bb92-b9761bcf6334\") " pod="openshift-cluster-node-tuning-operator/tuned-fds9k" Jul 05 13:15:17.989939 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:17.988635 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/c3c5ffe7-eb78-4b28-abb6-106dd0bbb008-host-slash\") pod \"iptables-alerter-b2c6n\" (UID: \"c3c5ffe7-eb78-4b28-abb6-106dd0bbb008\") " pod="openshift-network-operator/iptables-alerter-b2c6n" Jul 05 13:15:17.989939 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:17.988661 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7bdbd204-0fb7-4ebf-af05-432c370a814b-var-lib-openvswitch\") pod \"ovnkube-node-656m6\" (UID: \"7bdbd204-0fb7-4ebf-af05-432c370a814b\") " pod="openshift-ovn-kubernetes/ovnkube-node-656m6" Jul 05 13:15:17.989939 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:17.988707 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/7bdbd204-0fb7-4ebf-af05-432c370a814b-node-log\") pod \"ovnkube-node-656m6\" (UID: \"7bdbd204-0fb7-4ebf-af05-432c370a814b\") " pod="openshift-ovn-kubernetes/ovnkube-node-656m6" Jul 05 13:15:17.989939 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:17.988727 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/c3c5ffe7-eb78-4b28-abb6-106dd0bbb008-host-slash\") pod \"iptables-alerter-b2c6n\" (UID: \"c3c5ffe7-eb78-4b28-abb6-106dd0bbb008\") " pod="openshift-network-operator/iptables-alerter-b2c6n" Jul 05 13:15:17.989939 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:17.988769 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/7bdbd204-0fb7-4ebf-af05-432c370a814b-host-cni-bin\") pod \"ovnkube-node-656m6\" (UID: \"7bdbd204-0fb7-4ebf-af05-432c370a814b\") " pod="openshift-ovn-kubernetes/ovnkube-node-656m6" Jul 05 13:15:17.989939 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:17.988799 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b9962dff-a80f-4ece-bb92-b9761bcf6334-var-lib-kubelet\") pod \"tuned-fds9k\" (UID: \"b9962dff-a80f-4ece-bb92-b9761bcf6334\") " pod="openshift-cluster-node-tuning-operator/tuned-fds9k" Jul 05 13:15:17.989939 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:17.988823 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/545d8134-4ad2-4c62-b302-64b8e10bc493-tuning-conf-dir\") pod \"multus-additional-cni-plugins-7n8f2\" (UID: \"545d8134-4ad2-4c62-b302-64b8e10bc493\") " pod="openshift-multus/multus-additional-cni-plugins-7n8f2" Jul 05 13:15:17.989939 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:17.988887 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/7bdbd204-0fb7-4ebf-af05-432c370a814b-log-socket\") pod \"ovnkube-node-656m6\" (UID: \"7bdbd204-0fb7-4ebf-af05-432c370a814b\") " pod="openshift-ovn-kubernetes/ovnkube-node-656m6" Jul 05 13:15:17.989939 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:17.988924 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/7bdbd204-0fb7-4ebf-af05-432c370a814b-host-cni-netd\") pod \"ovnkube-node-656m6\" (UID: \"7bdbd204-0fb7-4ebf-af05-432c370a814b\") " pod="openshift-ovn-kubernetes/ovnkube-node-656m6" Jul 05 13:15:17.989939 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:17.988956 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b9962dff-a80f-4ece-bb92-b9761bcf6334-host\") pod \"tuned-fds9k\" (UID: \"b9962dff-a80f-4ece-bb92-b9761bcf6334\") " pod="openshift-cluster-node-tuning-operator/tuned-fds9k" Jul 05 13:15:17.989939 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:17.988995 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/5386321b-a286-4e02-9b76-b81ed7a920ea-sys-fs\") pod \"aws-ebs-csi-driver-node-mkr9r\" (UID: \"5386321b-a286-4e02-9b76-b81ed7a920ea\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mkr9r" Jul 05 13:15:17.989939 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:17.989029 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/e8e7535a-8f4e-4fd6-95d2-4cb8888b284b-konnectivity-ca\") pod \"konnectivity-agent-kpmmd\" (UID: \"e8e7535a-8f4e-4fd6-95d2-4cb8888b284b\") " pod="kube-system/konnectivity-agent-kpmmd" Jul 05 13:15:17.989939 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:17.989054 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/7bdbd204-0fb7-4ebf-af05-432c370a814b-ovnkube-config\") pod \"ovnkube-node-656m6\" (UID: \"7bdbd204-0fb7-4ebf-af05-432c370a814b\") " pod="openshift-ovn-kubernetes/ovnkube-node-656m6" Jul 05 13:15:17.989939 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:17.989079 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/c01b0489-2ef4-4fbe-9f9e-1870bc580ee9-hosts-file\") pod \"node-resolver-btxlt\" (UID: \"c01b0489-2ef4-4fbe-9f9e-1870bc580ee9\") " pod="openshift-dns/node-resolver-btxlt" Jul 05 13:15:17.989939 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:17.989108 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/5386321b-a286-4e02-9b76-b81ed7a920ea-sys-fs\") pod \"aws-ebs-csi-driver-node-mkr9r\" (UID: \"5386321b-a286-4e02-9b76-b81ed7a920ea\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mkr9r" Jul 05 13:15:17.989939 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:17.989110 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/b9962dff-a80f-4ece-bb92-b9761bcf6334-run\") pod \"tuned-fds9k\" (UID: \"b9962dff-a80f-4ece-bb92-b9761bcf6334\") " pod="openshift-cluster-node-tuning-operator/tuned-fds9k" Jul 05 13:15:17.990498 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:17.989144 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/5386321b-a286-4e02-9b76-b81ed7a920ea-registration-dir\") pod \"aws-ebs-csi-driver-node-mkr9r\" (UID: \"5386321b-a286-4e02-9b76-b81ed7a920ea\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mkr9r" Jul 05 13:15:17.990498 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:17.989183 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/eda62bac-3d03-4f52-b055-a77147e9ee97-multus-conf-dir\") pod \"multus-lq7qp\" (UID: \"eda62bac-3d03-4f52-b055-a77147e9ee97\") " pod="openshift-multus/multus-lq7qp" Jul 05 13:15:17.990498 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:17.989208 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/7bdbd204-0fb7-4ebf-af05-432c370a814b-host-kubelet\") pod \"ovnkube-node-656m6\" (UID: \"7bdbd204-0fb7-4ebf-af05-432c370a814b\") " pod="openshift-ovn-kubernetes/ovnkube-node-656m6" Jul 05 13:15:17.990498 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:17.989237 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/5386321b-a286-4e02-9b76-b81ed7a920ea-registration-dir\") pod \"aws-ebs-csi-driver-node-mkr9r\" (UID: \"5386321b-a286-4e02-9b76-b81ed7a920ea\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mkr9r" Jul 05 13:15:17.990498 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:17.989283 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/7bdbd204-0fb7-4ebf-af05-432c370a814b-ovn-node-metrics-cert\") pod \"ovnkube-node-656m6\" (UID: \"7bdbd204-0fb7-4ebf-af05-432c370a814b\") " pod="openshift-ovn-kubernetes/ovnkube-node-656m6" Jul 05 13:15:17.990498 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:17.989323 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4ff28158-61b2-4cb9-8b92-2ac0793c23bb-original-pull-secret\") pod \"global-pull-secret-syncer-72444\" (UID: \"4ff28158-61b2-4cb9-8b92-2ac0793c23bb\") " pod="kube-system/global-pull-secret-syncer-72444" Jul 05 13:15:17.990498 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:17.989346 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/eda62bac-3d03-4f52-b055-a77147e9ee97-multus-conf-dir\") pod \"multus-lq7qp\" (UID: \"eda62bac-3d03-4f52-b055-a77147e9ee97\") " pod="openshift-multus/multus-lq7qp" Jul 05 13:15:17.990498 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:17.989360 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/5386321b-a286-4e02-9b76-b81ed7a920ea-kubelet-dir\") pod \"aws-ebs-csi-driver-node-mkr9r\" (UID: \"5386321b-a286-4e02-9b76-b81ed7a920ea\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mkr9r" Jul 05 13:15:17.990498 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:17.989402 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/eda62bac-3d03-4f52-b055-a77147e9ee97-multus-daemon-config\") pod \"multus-lq7qp\" (UID: \"eda62bac-3d03-4f52-b055-a77147e9ee97\") " pod="openshift-multus/multus-lq7qp" Jul 05 13:15:17.990498 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:17.989454 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/5386321b-a286-4e02-9b76-b81ed7a920ea-kubelet-dir\") pod \"aws-ebs-csi-driver-node-mkr9r\" (UID: \"5386321b-a286-4e02-9b76-b81ed7a920ea\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mkr9r" Jul 05 13:15:17.990498 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:17.989943 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/eda62bac-3d03-4f52-b055-a77147e9ee97-multus-daemon-config\") pod \"multus-lq7qp\" (UID: \"eda62bac-3d03-4f52-b055-a77147e9ee97\") " pod="openshift-multus/multus-lq7qp" Jul 05 13:15:17.996130 ip-10-0-130-191 kubenswrapper[2578]: E0705 13:15:17.996107 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 05 13:15:17.996239 ip-10-0-130-191 kubenswrapper[2578]: E0705 13:15:17.996147 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 05 13:15:17.996239 ip-10-0-130-191 kubenswrapper[2578]: E0705 13:15:17.996161 2578 projected.go:196] Error preparing data for projected volume kube-api-access-czl8p for pod openshift-network-diagnostics/network-check-target-jldjr: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 05 13:15:17.996239 ip-10-0-130-191 kubenswrapper[2578]: E0705 13:15:17.996227 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/50dc9dcd-a00c-4f85-b3dc-dbac06316c3e-kube-api-access-czl8p podName:50dc9dcd-a00c-4f85-b3dc-dbac06316c3e nodeName:}" failed. No retries permitted until 2026-07-05 13:15:18.496208172 +0000 UTC m=+2.954771271 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-czl8p" (UniqueName: "kubernetes.io/projected/50dc9dcd-a00c-4f85-b3dc-dbac06316c3e-kube-api-access-czl8p") pod "network-check-target-jldjr" (UID: "50dc9dcd-a00c-4f85-b3dc-dbac06316c3e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 05 13:15:17.996765 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:17.996731 2578 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 05 13:15:18.001039 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:18.000900 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kdsgp\" (UniqueName: \"kubernetes.io/projected/5386321b-a286-4e02-9b76-b81ed7a920ea-kube-api-access-kdsgp\") pod \"aws-ebs-csi-driver-node-mkr9r\" (UID: \"5386321b-a286-4e02-9b76-b81ed7a920ea\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mkr9r" Jul 05 13:15:18.001039 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:18.000949 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5g68t\" (UniqueName: \"kubernetes.io/projected/c3c5ffe7-eb78-4b28-abb6-106dd0bbb008-kube-api-access-5g68t\") pod \"iptables-alerter-b2c6n\" (UID: \"c3c5ffe7-eb78-4b28-abb6-106dd0bbb008\") " pod="openshift-network-operator/iptables-alerter-b2c6n" Jul 05 13:15:18.001039 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:18.000929 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xk9dk\" (UniqueName: \"kubernetes.io/projected/eda62bac-3d03-4f52-b055-a77147e9ee97-kube-api-access-xk9dk\") pod \"multus-lq7qp\" (UID: \"eda62bac-3d03-4f52-b055-a77147e9ee97\") " pod="openshift-multus/multus-lq7qp" Jul 05 13:15:18.007667 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:18.007608 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-191.ec2.internal" event={"ID":"8e4a5e8e2cce016390634f883dc8c308","Type":"ContainerStarted","Data":"6e4acd89fc2f1a3d4235cecd785fe34a878b37f94b6c651e7a0758f4d64bc7f6"} Jul 05 13:15:18.009147 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:18.009112 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-191.ec2.internal" event={"ID":"d82e074a4be1637d97b91534bd80eb21","Type":"ContainerStarted","Data":"059eb809d8f4e8a37b74e61056a536a80825062b0ce477b7520dcb464c338aff"} Jul 05 13:15:18.090412 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:18.090330 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wdf5k\" (UniqueName: \"kubernetes.io/projected/7bdbd204-0fb7-4ebf-af05-432c370a814b-kube-api-access-wdf5k\") pod \"ovnkube-node-656m6\" (UID: \"7bdbd204-0fb7-4ebf-af05-432c370a814b\") " pod="openshift-ovn-kubernetes/ovnkube-node-656m6" Jul 05 13:15:18.090412 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:18.090372 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/b9962dff-a80f-4ece-bb92-b9761bcf6334-etc-tuned\") pod \"tuned-fds9k\" (UID: \"b9962dff-a80f-4ece-bb92-b9761bcf6334\") " pod="openshift-cluster-node-tuning-operator/tuned-fds9k" Jul 05 13:15:18.090667 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:18.090415 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7bdbd204-0fb7-4ebf-af05-432c370a814b-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-656m6\" (UID: \"7bdbd204-0fb7-4ebf-af05-432c370a814b\") " pod="openshift-ovn-kubernetes/ovnkube-node-656m6" Jul 05 13:15:18.090667 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:18.090443 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-985mk\" (UniqueName: \"kubernetes.io/projected/c01b0489-2ef4-4fbe-9f9e-1870bc580ee9-kube-api-access-985mk\") pod \"node-resolver-btxlt\" (UID: \"c01b0489-2ef4-4fbe-9f9e-1870bc580ee9\") " pod="openshift-dns/node-resolver-btxlt" Jul 05 13:15:18.090667 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:18.090462 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b9962dff-a80f-4ece-bb92-b9761bcf6334-etc-kubernetes\") pod \"tuned-fds9k\" (UID: \"b9962dff-a80f-4ece-bb92-b9761bcf6334\") " pod="openshift-cluster-node-tuning-operator/tuned-fds9k" Jul 05 13:15:18.090667 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:18.090477 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/4ff28158-61b2-4cb9-8b92-2ac0793c23bb-kubelet-config\") pod \"global-pull-secret-syncer-72444\" (UID: \"4ff28158-61b2-4cb9-8b92-2ac0793c23bb\") " pod="kube-system/global-pull-secret-syncer-72444" Jul 05 13:15:18.090667 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:18.090492 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/545d8134-4ad2-4c62-b302-64b8e10bc493-cni-binary-copy\") pod \"multus-additional-cni-plugins-7n8f2\" (UID: \"545d8134-4ad2-4c62-b302-64b8e10bc493\") " pod="openshift-multus/multus-additional-cni-plugins-7n8f2" Jul 05 13:15:18.090667 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:18.090514 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/7bdbd204-0fb7-4ebf-af05-432c370a814b-systemd-units\") pod \"ovnkube-node-656m6\" (UID: \"7bdbd204-0fb7-4ebf-af05-432c370a814b\") " pod="openshift-ovn-kubernetes/ovnkube-node-656m6" Jul 05 13:15:18.090667 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:18.090529 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7bdbd204-0fb7-4ebf-af05-432c370a814b-etc-openvswitch\") pod \"ovnkube-node-656m6\" (UID: \"7bdbd204-0fb7-4ebf-af05-432c370a814b\") " pod="openshift-ovn-kubernetes/ovnkube-node-656m6" Jul 05 13:15:18.090667 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:18.090549 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7bdbd204-0fb7-4ebf-af05-432c370a814b-run-openvswitch\") pod \"ovnkube-node-656m6\" (UID: \"7bdbd204-0fb7-4ebf-af05-432c370a814b\") " pod="openshift-ovn-kubernetes/ovnkube-node-656m6" Jul 05 13:15:18.090667 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:18.090569 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b9962dff-a80f-4ece-bb92-b9761bcf6334-sys\") pod \"tuned-fds9k\" (UID: \"b9962dff-a80f-4ece-bb92-b9761bcf6334\") " pod="openshift-cluster-node-tuning-operator/tuned-fds9k" Jul 05 13:15:18.090667 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:18.090605 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7bdbd204-0fb7-4ebf-af05-432c370a814b-var-lib-openvswitch\") pod \"ovnkube-node-656m6\" (UID: \"7bdbd204-0fb7-4ebf-af05-432c370a814b\") " pod="openshift-ovn-kubernetes/ovnkube-node-656m6" Jul 05 13:15:18.090667 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:18.090627 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/7bdbd204-0fb7-4ebf-af05-432c370a814b-node-log\") pod \"ovnkube-node-656m6\" (UID: \"7bdbd204-0fb7-4ebf-af05-432c370a814b\") " pod="openshift-ovn-kubernetes/ovnkube-node-656m6" Jul 05 13:15:18.090667 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:18.090659 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/7bdbd204-0fb7-4ebf-af05-432c370a814b-host-cni-bin\") pod \"ovnkube-node-656m6\" (UID: \"7bdbd204-0fb7-4ebf-af05-432c370a814b\") " pod="openshift-ovn-kubernetes/ovnkube-node-656m6" Jul 05 13:15:18.091115 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:18.090682 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b9962dff-a80f-4ece-bb92-b9761bcf6334-var-lib-kubelet\") pod \"tuned-fds9k\" (UID: \"b9962dff-a80f-4ece-bb92-b9761bcf6334\") " pod="openshift-cluster-node-tuning-operator/tuned-fds9k" Jul 05 13:15:18.091115 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:18.090692 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7bdbd204-0fb7-4ebf-af05-432c370a814b-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-656m6\" (UID: \"7bdbd204-0fb7-4ebf-af05-432c370a814b\") " pod="openshift-ovn-kubernetes/ovnkube-node-656m6" Jul 05 13:15:18.091115 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:18.090706 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/545d8134-4ad2-4c62-b302-64b8e10bc493-tuning-conf-dir\") pod \"multus-additional-cni-plugins-7n8f2\" (UID: \"545d8134-4ad2-4c62-b302-64b8e10bc493\") " pod="openshift-multus/multus-additional-cni-plugins-7n8f2" Jul 05 13:15:18.091115 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:18.090728 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/7bdbd204-0fb7-4ebf-af05-432c370a814b-log-socket\") pod \"ovnkube-node-656m6\" (UID: \"7bdbd204-0fb7-4ebf-af05-432c370a814b\") " pod="openshift-ovn-kubernetes/ovnkube-node-656m6" Jul 05 13:15:18.091115 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:18.090751 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/7bdbd204-0fb7-4ebf-af05-432c370a814b-host-cni-netd\") pod \"ovnkube-node-656m6\" (UID: \"7bdbd204-0fb7-4ebf-af05-432c370a814b\") " pod="openshift-ovn-kubernetes/ovnkube-node-656m6" Jul 05 13:15:18.091115 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:18.090772 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b9962dff-a80f-4ece-bb92-b9761bcf6334-host\") pod \"tuned-fds9k\" (UID: \"b9962dff-a80f-4ece-bb92-b9761bcf6334\") " pod="openshift-cluster-node-tuning-operator/tuned-fds9k" Jul 05 13:15:18.091115 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:18.090796 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/e8e7535a-8f4e-4fd6-95d2-4cb8888b284b-konnectivity-ca\") pod \"konnectivity-agent-kpmmd\" (UID: \"e8e7535a-8f4e-4fd6-95d2-4cb8888b284b\") " pod="kube-system/konnectivity-agent-kpmmd" Jul 05 13:15:18.091115 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:18.090819 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/7bdbd204-0fb7-4ebf-af05-432c370a814b-ovnkube-config\") pod \"ovnkube-node-656m6\" (UID: \"7bdbd204-0fb7-4ebf-af05-432c370a814b\") " pod="openshift-ovn-kubernetes/ovnkube-node-656m6" Jul 05 13:15:18.091115 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:18.090842 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/c01b0489-2ef4-4fbe-9f9e-1870bc580ee9-hosts-file\") pod \"node-resolver-btxlt\" (UID: \"c01b0489-2ef4-4fbe-9f9e-1870bc580ee9\") " pod="openshift-dns/node-resolver-btxlt" Jul 05 13:15:18.091115 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:18.090864 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/b9962dff-a80f-4ece-bb92-b9761bcf6334-run\") pod \"tuned-fds9k\" (UID: \"b9962dff-a80f-4ece-bb92-b9761bcf6334\") " pod="openshift-cluster-node-tuning-operator/tuned-fds9k" Jul 05 13:15:18.091115 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:18.090887 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/7bdbd204-0fb7-4ebf-af05-432c370a814b-host-kubelet\") pod \"ovnkube-node-656m6\" (UID: \"7bdbd204-0fb7-4ebf-af05-432c370a814b\") " pod="openshift-ovn-kubernetes/ovnkube-node-656m6" Jul 05 13:15:18.091115 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:18.090915 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/7bdbd204-0fb7-4ebf-af05-432c370a814b-ovn-node-metrics-cert\") pod \"ovnkube-node-656m6\" (UID: \"7bdbd204-0fb7-4ebf-af05-432c370a814b\") " pod="openshift-ovn-kubernetes/ovnkube-node-656m6" Jul 05 13:15:18.091115 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:18.090933 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b9962dff-a80f-4ece-bb92-b9761bcf6334-etc-kubernetes\") pod \"tuned-fds9k\" (UID: \"b9962dff-a80f-4ece-bb92-b9761bcf6334\") " pod="openshift-cluster-node-tuning-operator/tuned-fds9k" Jul 05 13:15:18.091115 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:18.090938 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4ff28158-61b2-4cb9-8b92-2ac0793c23bb-original-pull-secret\") pod \"global-pull-secret-syncer-72444\" (UID: \"4ff28158-61b2-4cb9-8b92-2ac0793c23bb\") " pod="kube-system/global-pull-secret-syncer-72444" Jul 05 13:15:18.091115 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:18.090986 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/7bdbd204-0fb7-4ebf-af05-432c370a814b-host-slash\") pod \"ovnkube-node-656m6\" (UID: \"7bdbd204-0fb7-4ebf-af05-432c370a814b\") " pod="openshift-ovn-kubernetes/ovnkube-node-656m6" Jul 05 13:15:18.091115 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:18.091013 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/7bdbd204-0fb7-4ebf-af05-432c370a814b-run-systemd\") pod \"ovnkube-node-656m6\" (UID: \"7bdbd204-0fb7-4ebf-af05-432c370a814b\") " pod="openshift-ovn-kubernetes/ovnkube-node-656m6" Jul 05 13:15:18.091115 ip-10-0-130-191 kubenswrapper[2578]: E0705 13:15:18.091018 2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 05 13:15:18.091115 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:18.091041 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b9962dff-a80f-4ece-bb92-b9761bcf6334-lib-modules\") pod \"tuned-fds9k\" (UID: \"b9962dff-a80f-4ece-bb92-b9761bcf6334\") " pod="openshift-cluster-node-tuning-operator/tuned-fds9k" Jul 05 13:15:18.091888 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:18.091066 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/b14dc74b-da46-4d14-8fd6-fd9357838fe2-serviceca\") pod \"node-ca-g7nxb\" (UID: \"b14dc74b-da46-4d14-8fd6-fd9357838fe2\") " pod="openshift-image-registry/node-ca-g7nxb" Jul 05 13:15:18.091888 ip-10-0-130-191 kubenswrapper[2578]: E0705 13:15:18.091077 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4ff28158-61b2-4cb9-8b92-2ac0793c23bb-original-pull-secret podName:4ff28158-61b2-4cb9-8b92-2ac0793c23bb nodeName:}" failed. No retries permitted until 2026-07-05 13:15:18.591061284 +0000 UTC m=+3.049624368 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/4ff28158-61b2-4cb9-8b92-2ac0793c23bb-original-pull-secret") pod "global-pull-secret-syncer-72444" (UID: "4ff28158-61b2-4cb9-8b92-2ac0793c23bb") : object "kube-system"/"original-pull-secret" not registered Jul 05 13:15:18.091888 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:18.091105 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/7bdbd204-0fb7-4ebf-af05-432c370a814b-ovnkube-script-lib\") pod \"ovnkube-node-656m6\" (UID: \"7bdbd204-0fb7-4ebf-af05-432c370a814b\") " pod="openshift-ovn-kubernetes/ovnkube-node-656m6" Jul 05 13:15:18.091888 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:18.091129 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b9962dff-a80f-4ece-bb92-b9761bcf6334-tmp\") pod \"tuned-fds9k\" (UID: \"b9962dff-a80f-4ece-bb92-b9761bcf6334\") " pod="openshift-cluster-node-tuning-operator/tuned-fds9k" Jul 05 13:15:18.091888 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:18.091153 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b14dc74b-da46-4d14-8fd6-fd9357838fe2-host\") pod \"node-ca-g7nxb\" (UID: \"b14dc74b-da46-4d14-8fd6-fd9357838fe2\") " pod="openshift-image-registry/node-ca-g7nxb" Jul 05 13:15:18.091888 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:18.091188 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/545d8134-4ad2-4c62-b302-64b8e10bc493-system-cni-dir\") pod \"multus-additional-cni-plugins-7n8f2\" (UID: \"545d8134-4ad2-4c62-b302-64b8e10bc493\") " pod="openshift-multus/multus-additional-cni-plugins-7n8f2" Jul 05 13:15:18.091888 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:18.091222 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/545d8134-4ad2-4c62-b302-64b8e10bc493-cnibin\") pod \"multus-additional-cni-plugins-7n8f2\" (UID: \"545d8134-4ad2-4c62-b302-64b8e10bc493\") " pod="openshift-multus/multus-additional-cni-plugins-7n8f2" Jul 05 13:15:18.091888 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:18.091246 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/545d8134-4ad2-4c62-b302-64b8e10bc493-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-7n8f2\" (UID: \"545d8134-4ad2-4c62-b302-64b8e10bc493\") " pod="openshift-multus/multus-additional-cni-plugins-7n8f2" Jul 05 13:15:18.091888 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:18.091274 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gs6jp\" (UniqueName: \"kubernetes.io/projected/5ad92660-5823-4f14-adcc-763d06385f45-kube-api-access-gs6jp\") pod \"network-metrics-daemon-dxwj6\" (UID: \"5ad92660-5823-4f14-adcc-763d06385f45\") " pod="openshift-multus/network-metrics-daemon-dxwj6" Jul 05 13:15:18.091888 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:18.091296 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/7bdbd204-0fb7-4ebf-af05-432c370a814b-host-run-netns\") pod \"ovnkube-node-656m6\" (UID: \"7bdbd204-0fb7-4ebf-af05-432c370a814b\") " pod="openshift-ovn-kubernetes/ovnkube-node-656m6" Jul 05 13:15:18.091888 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:18.091318 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c01b0489-2ef4-4fbe-9f9e-1870bc580ee9-tmp-dir\") pod \"node-resolver-btxlt\" (UID: \"c01b0489-2ef4-4fbe-9f9e-1870bc580ee9\") " pod="openshift-dns/node-resolver-btxlt" Jul 05 13:15:18.091888 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:18.091342 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5ad92660-5823-4f14-adcc-763d06385f45-metrics-certs\") pod \"network-metrics-daemon-dxwj6\" (UID: \"5ad92660-5823-4f14-adcc-763d06385f45\") " pod="openshift-multus/network-metrics-daemon-dxwj6" Jul 05 13:15:18.091888 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:18.091366 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/b9962dff-a80f-4ece-bb92-b9761bcf6334-etc-modprobe-d\") pod \"tuned-fds9k\" (UID: \"b9962dff-a80f-4ece-bb92-b9761bcf6334\") " pod="openshift-cluster-node-tuning-operator/tuned-fds9k" Jul 05 13:15:18.091888 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:18.091388 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/b9962dff-a80f-4ece-bb92-b9761bcf6334-etc-systemd\") pod \"tuned-fds9k\" (UID: \"b9962dff-a80f-4ece-bb92-b9761bcf6334\") " pod="openshift-cluster-node-tuning-operator/tuned-fds9k" Jul 05 13:15:18.091888 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:18.091411 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-skvcw\" (UniqueName: \"kubernetes.io/projected/b14dc74b-da46-4d14-8fd6-fd9357838fe2-kube-api-access-skvcw\") pod \"node-ca-g7nxb\" (UID: \"b14dc74b-da46-4d14-8fd6-fd9357838fe2\") " pod="openshift-image-registry/node-ca-g7nxb" Jul 05 13:15:18.091888 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:18.091453 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7bdbd204-0fb7-4ebf-af05-432c370a814b-host-run-ovn-kubernetes\") pod \"ovnkube-node-656m6\" (UID: \"7bdbd204-0fb7-4ebf-af05-432c370a814b\") " pod="openshift-ovn-kubernetes/ovnkube-node-656m6" Jul 05 13:15:18.092339 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:18.091483 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/b14dc74b-da46-4d14-8fd6-fd9357838fe2-serviceca\") pod \"node-ca-g7nxb\" (UID: \"b14dc74b-da46-4d14-8fd6-fd9357838fe2\") " pod="openshift-image-registry/node-ca-g7nxb" Jul 05 13:15:18.092339 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:18.091487 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/b9962dff-a80f-4ece-bb92-b9761bcf6334-etc-sysctl-d\") pod \"tuned-fds9k\" (UID: \"b9962dff-a80f-4ece-bb92-b9761bcf6334\") " pod="openshift-cluster-node-tuning-operator/tuned-fds9k" Jul 05 13:15:18.092339 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:18.091535 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/4ff28158-61b2-4cb9-8b92-2ac0793c23bb-dbus\") pod \"global-pull-secret-syncer-72444\" (UID: \"4ff28158-61b2-4cb9-8b92-2ac0793c23bb\") " pod="kube-system/global-pull-secret-syncer-72444" Jul 05 13:15:18.092339 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:18.091564 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8hghn\" (UniqueName: \"kubernetes.io/projected/545d8134-4ad2-4c62-b302-64b8e10bc493-kube-api-access-8hghn\") pod \"multus-additional-cni-plugins-7n8f2\" (UID: \"545d8134-4ad2-4c62-b302-64b8e10bc493\") " pod="openshift-multus/multus-additional-cni-plugins-7n8f2" Jul 05 13:15:18.092339 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:18.091618 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/b9962dff-a80f-4ece-bb92-b9761bcf6334-etc-sysconfig\") pod \"tuned-fds9k\" (UID: \"b9962dff-a80f-4ece-bb92-b9761bcf6334\") " pod="openshift-cluster-node-tuning-operator/tuned-fds9k" Jul 05 13:15:18.092339 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:18.091634 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/b9962dff-a80f-4ece-bb92-b9761bcf6334-etc-sysctl-d\") pod \"tuned-fds9k\" (UID: \"b9962dff-a80f-4ece-bb92-b9761bcf6334\") " pod="openshift-cluster-node-tuning-operator/tuned-fds9k" Jul 05 13:15:18.092339 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:18.091647 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/7bdbd204-0fb7-4ebf-af05-432c370a814b-systemd-units\") pod \"ovnkube-node-656m6\" (UID: \"7bdbd204-0fb7-4ebf-af05-432c370a814b\") " pod="openshift-ovn-kubernetes/ovnkube-node-656m6" Jul 05 13:15:18.092339 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:18.091651 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/545d8134-4ad2-4c62-b302-64b8e10bc493-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-7n8f2\" (UID: \"545d8134-4ad2-4c62-b302-64b8e10bc493\") " pod="openshift-multus/multus-additional-cni-plugins-7n8f2" Jul 05 13:15:18.092339 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:18.091700 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/7bdbd204-0fb7-4ebf-af05-432c370a814b-env-overrides\") pod \"ovnkube-node-656m6\" (UID: \"7bdbd204-0fb7-4ebf-af05-432c370a814b\") " pod="openshift-ovn-kubernetes/ovnkube-node-656m6" Jul 05 13:15:18.092339 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:18.091703 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/4ff28158-61b2-4cb9-8b92-2ac0793c23bb-kubelet-config\") pod \"global-pull-secret-syncer-72444\" (UID: \"4ff28158-61b2-4cb9-8b92-2ac0793c23bb\") " pod="kube-system/global-pull-secret-syncer-72444" Jul 05 13:15:18.092339 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:18.091726 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/b9962dff-a80f-4ece-bb92-b9761bcf6334-etc-sysctl-conf\") pod \"tuned-fds9k\" (UID: \"b9962dff-a80f-4ece-bb92-b9761bcf6334\") " pod="openshift-cluster-node-tuning-operator/tuned-fds9k" Jul 05 13:15:18.092339 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:18.091749 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/7bdbd204-0fb7-4ebf-af05-432c370a814b-host-slash\") pod \"ovnkube-node-656m6\" (UID: \"7bdbd204-0fb7-4ebf-af05-432c370a814b\") " pod="openshift-ovn-kubernetes/ovnkube-node-656m6" Jul 05 13:15:18.092339 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:18.091751 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wlgrd\" (UniqueName: \"kubernetes.io/projected/b9962dff-a80f-4ece-bb92-b9761bcf6334-kube-api-access-wlgrd\") pod \"tuned-fds9k\" (UID: \"b9962dff-a80f-4ece-bb92-b9761bcf6334\") " pod="openshift-cluster-node-tuning-operator/tuned-fds9k" Jul 05 13:15:18.092339 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:18.091793 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/e8e7535a-8f4e-4fd6-95d2-4cb8888b284b-agent-certs\") pod \"konnectivity-agent-kpmmd\" (UID: \"e8e7535a-8f4e-4fd6-95d2-4cb8888b284b\") " pod="kube-system/konnectivity-agent-kpmmd" Jul 05 13:15:18.092339 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:18.091818 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/545d8134-4ad2-4c62-b302-64b8e10bc493-os-release\") pod \"multus-additional-cni-plugins-7n8f2\" (UID: \"545d8134-4ad2-4c62-b302-64b8e10bc493\") " pod="openshift-multus/multus-additional-cni-plugins-7n8f2" Jul 05 13:15:18.092339 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:18.091844 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/7bdbd204-0fb7-4ebf-af05-432c370a814b-run-ovn\") pod \"ovnkube-node-656m6\" (UID: \"7bdbd204-0fb7-4ebf-af05-432c370a814b\") " pod="openshift-ovn-kubernetes/ovnkube-node-656m6" Jul 05 13:15:18.092339 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:18.091902 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/7bdbd204-0fb7-4ebf-af05-432c370a814b-run-ovn\") pod \"ovnkube-node-656m6\" (UID: \"7bdbd204-0fb7-4ebf-af05-432c370a814b\") " pod="openshift-ovn-kubernetes/ovnkube-node-656m6" Jul 05 13:15:18.092857 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:18.091924 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/7bdbd204-0fb7-4ebf-af05-432c370a814b-run-systemd\") pod \"ovnkube-node-656m6\" (UID: \"7bdbd204-0fb7-4ebf-af05-432c370a814b\") " pod="openshift-ovn-kubernetes/ovnkube-node-656m6" Jul 05 13:15:18.092857 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:18.091966 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7bdbd204-0fb7-4ebf-af05-432c370a814b-etc-openvswitch\") pod \"ovnkube-node-656m6\" (UID: \"7bdbd204-0fb7-4ebf-af05-432c370a814b\") " pod="openshift-ovn-kubernetes/ovnkube-node-656m6" Jul 05 13:15:18.092857 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:18.091970 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/545d8134-4ad2-4c62-b302-64b8e10bc493-os-release\") pod \"multus-additional-cni-plugins-7n8f2\" (UID: \"545d8134-4ad2-4c62-b302-64b8e10bc493\") " pod="openshift-multus/multus-additional-cni-plugins-7n8f2" Jul 05 13:15:18.092857 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:18.092022 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7bdbd204-0fb7-4ebf-af05-432c370a814b-run-openvswitch\") pod \"ovnkube-node-656m6\" (UID: \"7bdbd204-0fb7-4ebf-af05-432c370a814b\") " pod="openshift-ovn-kubernetes/ovnkube-node-656m6" Jul 05 13:15:18.092857 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:18.092072 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b9962dff-a80f-4ece-bb92-b9761bcf6334-sys\") pod \"tuned-fds9k\" (UID: \"b9962dff-a80f-4ece-bb92-b9761bcf6334\") " pod="openshift-cluster-node-tuning-operator/tuned-fds9k" Jul 05 13:15:18.092857 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:18.092113 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7bdbd204-0fb7-4ebf-af05-432c370a814b-var-lib-openvswitch\") pod \"ovnkube-node-656m6\" (UID: \"7bdbd204-0fb7-4ebf-af05-432c370a814b\") " pod="openshift-ovn-kubernetes/ovnkube-node-656m6" Jul 05 13:15:18.092857 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:18.092169 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/7bdbd204-0fb7-4ebf-af05-432c370a814b-node-log\") pod \"ovnkube-node-656m6\" (UID: \"7bdbd204-0fb7-4ebf-af05-432c370a814b\") " pod="openshift-ovn-kubernetes/ovnkube-node-656m6" Jul 05 13:15:18.092857 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:18.092216 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/545d8134-4ad2-4c62-b302-64b8e10bc493-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-7n8f2\" (UID: \"545d8134-4ad2-4c62-b302-64b8e10bc493\") " pod="openshift-multus/multus-additional-cni-plugins-7n8f2" Jul 05 13:15:18.092857 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:18.092325 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b9962dff-a80f-4ece-bb92-b9761bcf6334-lib-modules\") pod \"tuned-fds9k\" (UID: \"b9962dff-a80f-4ece-bb92-b9761bcf6334\") " pod="openshift-cluster-node-tuning-operator/tuned-fds9k" Jul 05 13:15:18.092857 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:18.092414 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/4ff28158-61b2-4cb9-8b92-2ac0793c23bb-dbus\") pod \"global-pull-secret-syncer-72444\" (UID: \"4ff28158-61b2-4cb9-8b92-2ac0793c23bb\") " pod="kube-system/global-pull-secret-syncer-72444" Jul 05 13:15:18.092857 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:18.092531 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/545d8134-4ad2-4c62-b302-64b8e10bc493-cni-binary-copy\") pod \"multus-additional-cni-plugins-7n8f2\" (UID: \"545d8134-4ad2-4c62-b302-64b8e10bc493\") " pod="openshift-multus/multus-additional-cni-plugins-7n8f2" Jul 05 13:15:18.092857 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:18.092603 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/7bdbd204-0fb7-4ebf-af05-432c370a814b-host-cni-bin\") pod \"ovnkube-node-656m6\" (UID: \"7bdbd204-0fb7-4ebf-af05-432c370a814b\") " pod="openshift-ovn-kubernetes/ovnkube-node-656m6" Jul 05 13:15:18.092857 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:18.092655 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b9962dff-a80f-4ece-bb92-b9761bcf6334-var-lib-kubelet\") pod \"tuned-fds9k\" (UID: \"b9962dff-a80f-4ece-bb92-b9761bcf6334\") " pod="openshift-cluster-node-tuning-operator/tuned-fds9k" Jul 05 13:15:18.092857 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:18.092655 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/b9962dff-a80f-4ece-bb92-b9761bcf6334-etc-sysconfig\") pod \"tuned-fds9k\" (UID: \"b9962dff-a80f-4ece-bb92-b9761bcf6334\") " pod="openshift-cluster-node-tuning-operator/tuned-fds9k" Jul 05 13:15:18.092857 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:18.092818 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/545d8134-4ad2-4c62-b302-64b8e10bc493-tuning-conf-dir\") pod \"multus-additional-cni-plugins-7n8f2\" (UID: \"545d8134-4ad2-4c62-b302-64b8e10bc493\") " pod="openshift-multus/multus-additional-cni-plugins-7n8f2" Jul 05 13:15:18.093296 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:18.092863 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/7bdbd204-0fb7-4ebf-af05-432c370a814b-log-socket\") pod \"ovnkube-node-656m6\" (UID: \"7bdbd204-0fb7-4ebf-af05-432c370a814b\") " pod="openshift-ovn-kubernetes/ovnkube-node-656m6" Jul 05 13:15:18.093296 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:18.092922 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/7bdbd204-0fb7-4ebf-af05-432c370a814b-host-cni-netd\") pod \"ovnkube-node-656m6\" (UID: \"7bdbd204-0fb7-4ebf-af05-432c370a814b\") " pod="openshift-ovn-kubernetes/ovnkube-node-656m6" Jul 05 13:15:18.093296 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:18.092965 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b9962dff-a80f-4ece-bb92-b9761bcf6334-host\") pod \"tuned-fds9k\" (UID: \"b9962dff-a80f-4ece-bb92-b9761bcf6334\") " pod="openshift-cluster-node-tuning-operator/tuned-fds9k" Jul 05 13:15:18.093296 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:18.093227 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/545d8134-4ad2-4c62-b302-64b8e10bc493-cnibin\") pod \"multus-additional-cni-plugins-7n8f2\" (UID: \"545d8134-4ad2-4c62-b302-64b8e10bc493\") " pod="openshift-multus/multus-additional-cni-plugins-7n8f2" Jul 05 13:15:18.093463 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:18.093439 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/e8e7535a-8f4e-4fd6-95d2-4cb8888b284b-konnectivity-ca\") pod \"konnectivity-agent-kpmmd\" (UID: \"e8e7535a-8f4e-4fd6-95d2-4cb8888b284b\") " pod="kube-system/konnectivity-agent-kpmmd" Jul 05 13:15:18.094760 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:18.093718 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/7bdbd204-0fb7-4ebf-af05-432c370a814b-ovnkube-config\") pod \"ovnkube-node-656m6\" (UID: \"7bdbd204-0fb7-4ebf-af05-432c370a814b\") " pod="openshift-ovn-kubernetes/ovnkube-node-656m6" Jul 05 13:15:18.094760 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:18.093837 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/c01b0489-2ef4-4fbe-9f9e-1870bc580ee9-hosts-file\") pod \"node-resolver-btxlt\" (UID: \"c01b0489-2ef4-4fbe-9f9e-1870bc580ee9\") " pod="openshift-dns/node-resolver-btxlt" Jul 05 13:15:18.094760 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:18.093852 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/545d8134-4ad2-4c62-b302-64b8e10bc493-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-7n8f2\" (UID: \"545d8134-4ad2-4c62-b302-64b8e10bc493\") " pod="openshift-multus/multus-additional-cni-plugins-7n8f2" Jul 05 13:15:18.094760 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:18.093892 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/7bdbd204-0fb7-4ebf-af05-432c370a814b-host-kubelet\") pod \"ovnkube-node-656m6\" (UID: \"7bdbd204-0fb7-4ebf-af05-432c370a814b\") " pod="openshift-ovn-kubernetes/ovnkube-node-656m6" Jul 05 13:15:18.094760 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:18.093898 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/b9962dff-a80f-4ece-bb92-b9761bcf6334-run\") pod \"tuned-fds9k\" (UID: \"b9962dff-a80f-4ece-bb92-b9761bcf6334\") " pod="openshift-cluster-node-tuning-operator/tuned-fds9k" Jul 05 13:15:18.094760 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:18.093942 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/7bdbd204-0fb7-4ebf-af05-432c370a814b-host-run-netns\") pod \"ovnkube-node-656m6\" (UID: \"7bdbd204-0fb7-4ebf-af05-432c370a814b\") " pod="openshift-ovn-kubernetes/ovnkube-node-656m6" Jul 05 13:15:18.094760 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:18.094196 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c01b0489-2ef4-4fbe-9f9e-1870bc580ee9-tmp-dir\") pod \"node-resolver-btxlt\" (UID: \"c01b0489-2ef4-4fbe-9f9e-1870bc580ee9\") " pod="openshift-dns/node-resolver-btxlt" Jul 05 13:15:18.094760 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:18.094276 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7bdbd204-0fb7-4ebf-af05-432c370a814b-host-run-ovn-kubernetes\") pod \"ovnkube-node-656m6\" (UID: \"7bdbd204-0fb7-4ebf-af05-432c370a814b\") " pod="openshift-ovn-kubernetes/ovnkube-node-656m6" Jul 05 13:15:18.094760 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:18.094434 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/e8e7535a-8f4e-4fd6-95d2-4cb8888b284b-agent-certs\") pod \"konnectivity-agent-kpmmd\" (UID: \"e8e7535a-8f4e-4fd6-95d2-4cb8888b284b\") " pod="kube-system/konnectivity-agent-kpmmd" Jul 05 13:15:18.094760 ip-10-0-130-191 kubenswrapper[2578]: E0705 13:15:18.094504 2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 05 13:15:18.094760 ip-10-0-130-191 kubenswrapper[2578]: E0705 13:15:18.094547 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5ad92660-5823-4f14-adcc-763d06385f45-metrics-certs podName:5ad92660-5823-4f14-adcc-763d06385f45 nodeName:}" failed. No retries permitted until 2026-07-05 13:15:18.594536458 +0000 UTC m=+3.053099540 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5ad92660-5823-4f14-adcc-763d06385f45-metrics-certs") pod "network-metrics-daemon-dxwj6" (UID: "5ad92660-5823-4f14-adcc-763d06385f45") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 05 13:15:18.094760 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:18.094557 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/b9962dff-a80f-4ece-bb92-b9761bcf6334-etc-modprobe-d\") pod \"tuned-fds9k\" (UID: \"b9962dff-a80f-4ece-bb92-b9761bcf6334\") " pod="openshift-cluster-node-tuning-operator/tuned-fds9k" Jul 05 13:15:18.094760 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:18.094602 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/b9962dff-a80f-4ece-bb92-b9761bcf6334-etc-systemd\") pod \"tuned-fds9k\" (UID: \"b9962dff-a80f-4ece-bb92-b9761bcf6334\") " pod="openshift-cluster-node-tuning-operator/tuned-fds9k" Jul 05 13:15:18.094760 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:18.094604 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/7bdbd204-0fb7-4ebf-af05-432c370a814b-env-overrides\") pod \"ovnkube-node-656m6\" (UID: \"7bdbd204-0fb7-4ebf-af05-432c370a814b\") " pod="openshift-ovn-kubernetes/ovnkube-node-656m6" Jul 05 13:15:18.094760 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:18.094670 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b14dc74b-da46-4d14-8fd6-fd9357838fe2-host\") pod \"node-ca-g7nxb\" (UID: \"b14dc74b-da46-4d14-8fd6-fd9357838fe2\") " pod="openshift-image-registry/node-ca-g7nxb" Jul 05 13:15:18.094760 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:18.094701 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/b9962dff-a80f-4ece-bb92-b9761bcf6334-etc-sysctl-conf\") pod \"tuned-fds9k\" (UID: \"b9962dff-a80f-4ece-bb92-b9761bcf6334\") " pod="openshift-cluster-node-tuning-operator/tuned-fds9k" Jul 05 13:15:18.094760 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:18.094710 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/545d8134-4ad2-4c62-b302-64b8e10bc493-system-cni-dir\") pod \"multus-additional-cni-plugins-7n8f2\" (UID: \"545d8134-4ad2-4c62-b302-64b8e10bc493\") " pod="openshift-multus/multus-additional-cni-plugins-7n8f2" Jul 05 13:15:18.095877 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:18.095161 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/7bdbd204-0fb7-4ebf-af05-432c370a814b-ovnkube-script-lib\") pod \"ovnkube-node-656m6\" (UID: \"7bdbd204-0fb7-4ebf-af05-432c370a814b\") " pod="openshift-ovn-kubernetes/ovnkube-node-656m6" Jul 05 13:15:18.097228 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:18.097201 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b9962dff-a80f-4ece-bb92-b9761bcf6334-tmp\") pod \"tuned-fds9k\" (UID: \"b9962dff-a80f-4ece-bb92-b9761bcf6334\") " pod="openshift-cluster-node-tuning-operator/tuned-fds9k" Jul 05 13:15:18.098349 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:18.097915 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/7bdbd204-0fb7-4ebf-af05-432c370a814b-ovn-node-metrics-cert\") pod \"ovnkube-node-656m6\" (UID: \"7bdbd204-0fb7-4ebf-af05-432c370a814b\") " pod="openshift-ovn-kubernetes/ovnkube-node-656m6" Jul 05 13:15:18.098349 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:18.098106 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/b9962dff-a80f-4ece-bb92-b9761bcf6334-etc-tuned\") pod \"tuned-fds9k\" (UID: \"b9962dff-a80f-4ece-bb92-b9761bcf6334\") " pod="openshift-cluster-node-tuning-operator/tuned-fds9k" Jul 05 13:15:18.099628 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:18.099609 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wlgrd\" (UniqueName: \"kubernetes.io/projected/b9962dff-a80f-4ece-bb92-b9761bcf6334-kube-api-access-wlgrd\") pod \"tuned-fds9k\" (UID: \"b9962dff-a80f-4ece-bb92-b9761bcf6334\") " pod="openshift-cluster-node-tuning-operator/tuned-fds9k" Jul 05 13:15:18.100555 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:18.100003 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-985mk\" (UniqueName: \"kubernetes.io/projected/c01b0489-2ef4-4fbe-9f9e-1870bc580ee9-kube-api-access-985mk\") pod \"node-resolver-btxlt\" (UID: \"c01b0489-2ef4-4fbe-9f9e-1870bc580ee9\") " pod="openshift-dns/node-resolver-btxlt" Jul 05 13:15:18.100555 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:18.100053 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wdf5k\" (UniqueName: \"kubernetes.io/projected/7bdbd204-0fb7-4ebf-af05-432c370a814b-kube-api-access-wdf5k\") pod \"ovnkube-node-656m6\" (UID: \"7bdbd204-0fb7-4ebf-af05-432c370a814b\") " pod="openshift-ovn-kubernetes/ovnkube-node-656m6" Jul 05 13:15:18.100853 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:18.100830 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8hghn\" (UniqueName: \"kubernetes.io/projected/545d8134-4ad2-4c62-b302-64b8e10bc493-kube-api-access-8hghn\") pod \"multus-additional-cni-plugins-7n8f2\" (UID: \"545d8134-4ad2-4c62-b302-64b8e10bc493\") " pod="openshift-multus/multus-additional-cni-plugins-7n8f2" Jul 05 13:15:18.103322 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:18.103038 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-skvcw\" (UniqueName: \"kubernetes.io/projected/b14dc74b-da46-4d14-8fd6-fd9357838fe2-kube-api-access-skvcw\") pod \"node-ca-g7nxb\" (UID: \"b14dc74b-da46-4d14-8fd6-fd9357838fe2\") " pod="openshift-image-registry/node-ca-g7nxb" Jul 05 13:15:18.104079 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:18.104057 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gs6jp\" (UniqueName: \"kubernetes.io/projected/5ad92660-5823-4f14-adcc-763d06385f45-kube-api-access-gs6jp\") pod \"network-metrics-daemon-dxwj6\" (UID: \"5ad92660-5823-4f14-adcc-763d06385f45\") " pod="openshift-multus/network-metrics-daemon-dxwj6" Jul 05 13:15:18.185335 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:18.185305 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mkr9r" Jul 05 13:15:18.195402 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:18.195379 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-lq7qp" Jul 05 13:15:18.195678 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:18.195654 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5386321b_a286_4e02_9b76_b81ed7a920ea.slice/crio-95ec5c59b72f4053959b67c93d961201c6f69b36944da18dac047649c5d4d5bf WatchSource:0}: Error finding container 95ec5c59b72f4053959b67c93d961201c6f69b36944da18dac047649c5d4d5bf: Status 404 returned error can't find the container with id 95ec5c59b72f4053959b67c93d961201c6f69b36944da18dac047649c5d4d5bf Jul 05 13:15:18.204752 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:18.204715 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeda62bac_3d03_4f52_b055_a77147e9ee97.slice/crio-287ec4b5d0e2a2dd3c08177d38ddc723df8b92699cb45a959c4a970077d9bd89 WatchSource:0}: Error finding container 287ec4b5d0e2a2dd3c08177d38ddc723df8b92699cb45a959c4a970077d9bd89: Status 404 returned error can't find the container with id 287ec4b5d0e2a2dd3c08177d38ddc723df8b92699cb45a959c4a970077d9bd89 Jul 05 13:15:18.212010 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:18.211841 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-656m6" Jul 05 13:15:18.220984 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:18.220826 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-b2c6n" Jul 05 13:15:18.220984 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:18.220979 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7bdbd204_0fb7_4ebf_af05_432c370a814b.slice/crio-29221b33eb2d89414004251722e6ad76dbb04cfda3865e9cf8644e619d375e8c WatchSource:0}: Error finding container 29221b33eb2d89414004251722e6ad76dbb04cfda3865e9cf8644e619d375e8c: Status 404 returned error can't find the container with id 29221b33eb2d89414004251722e6ad76dbb04cfda3865e9cf8644e619d375e8c Jul 05 13:15:18.228050 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:18.228032 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-g7nxb" Jul 05 13:15:18.228953 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:18.228933 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc3c5ffe7_eb78_4b28_abb6_106dd0bbb008.slice/crio-f096c926fb83466a52bd33ccc5b69c7374bdaa5eb1a62e5848ba192cefc04fda WatchSource:0}: Error finding container f096c926fb83466a52bd33ccc5b69c7374bdaa5eb1a62e5848ba192cefc04fda: Status 404 returned error can't find the container with id f096c926fb83466a52bd33ccc5b69c7374bdaa5eb1a62e5848ba192cefc04fda Jul 05 13:15:18.237277 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:18.237256 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-kpmmd" Jul 05 13:15:18.245037 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:18.245015 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-btxlt" Jul 05 13:15:18.254681 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:18.254663 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-fds9k" Jul 05 13:15:18.262208 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:18.262190 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-7n8f2" Jul 05 13:15:18.595740 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:18.595707 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4ff28158-61b2-4cb9-8b92-2ac0793c23bb-original-pull-secret\") pod \"global-pull-secret-syncer-72444\" (UID: \"4ff28158-61b2-4cb9-8b92-2ac0793c23bb\") " pod="kube-system/global-pull-secret-syncer-72444" Jul 05 13:15:18.595903 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:18.595778 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5ad92660-5823-4f14-adcc-763d06385f45-metrics-certs\") pod \"network-metrics-daemon-dxwj6\" (UID: \"5ad92660-5823-4f14-adcc-763d06385f45\") " pod="openshift-multus/network-metrics-daemon-dxwj6" Jul 05 13:15:18.595903 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:18.595825 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-czl8p\" (UniqueName: \"kubernetes.io/projected/50dc9dcd-a00c-4f85-b3dc-dbac06316c3e-kube-api-access-czl8p\") pod \"network-check-target-jldjr\" (UID: \"50dc9dcd-a00c-4f85-b3dc-dbac06316c3e\") " pod="openshift-network-diagnostics/network-check-target-jldjr" Jul 05 13:15:18.596014 ip-10-0-130-191 kubenswrapper[2578]: E0705 13:15:18.595968 2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 05 13:15:18.596014 ip-10-0-130-191 kubenswrapper[2578]: E0705 13:15:18.595982 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 05 13:15:18.596014 ip-10-0-130-191 kubenswrapper[2578]: E0705 13:15:18.596001 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 05 13:15:18.596014 ip-10-0-130-191 kubenswrapper[2578]: E0705 13:15:18.596014 2578 projected.go:196] Error preparing data for projected volume kube-api-access-czl8p for pod openshift-network-diagnostics/network-check-target-jldjr: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 05 13:15:18.596146 ip-10-0-130-191 kubenswrapper[2578]: E0705 13:15:18.596039 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5ad92660-5823-4f14-adcc-763d06385f45-metrics-certs podName:5ad92660-5823-4f14-adcc-763d06385f45 nodeName:}" failed. No retries permitted until 2026-07-05 13:15:19.596019573 +0000 UTC m=+4.054582656 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5ad92660-5823-4f14-adcc-763d06385f45-metrics-certs") pod "network-metrics-daemon-dxwj6" (UID: "5ad92660-5823-4f14-adcc-763d06385f45") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 05 13:15:18.596146 ip-10-0-130-191 kubenswrapper[2578]: E0705 13:15:18.595966 2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 05 13:15:18.596146 ip-10-0-130-191 kubenswrapper[2578]: E0705 13:15:18.596063 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/50dc9dcd-a00c-4f85-b3dc-dbac06316c3e-kube-api-access-czl8p podName:50dc9dcd-a00c-4f85-b3dc-dbac06316c3e nodeName:}" failed. No retries permitted until 2026-07-05 13:15:19.596053489 +0000 UTC m=+4.054616571 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-czl8p" (UniqueName: "kubernetes.io/projected/50dc9dcd-a00c-4f85-b3dc-dbac06316c3e-kube-api-access-czl8p") pod "network-check-target-jldjr" (UID: "50dc9dcd-a00c-4f85-b3dc-dbac06316c3e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 05 13:15:18.596146 ip-10-0-130-191 kubenswrapper[2578]: E0705 13:15:18.596091 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4ff28158-61b2-4cb9-8b92-2ac0793c23bb-original-pull-secret podName:4ff28158-61b2-4cb9-8b92-2ac0793c23bb nodeName:}" failed. No retries permitted until 2026-07-05 13:15:19.596072497 +0000 UTC m=+4.054635580 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/4ff28158-61b2-4cb9-8b92-2ac0793c23bb-original-pull-secret") pod "global-pull-secret-syncer-72444" (UID: "4ff28158-61b2-4cb9-8b92-2ac0793c23bb") : object "kube-system"/"original-pull-secret" not registered Jul 05 13:15:18.923230 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:18.923158 2578 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-04 13:10:17 +0000 UTC" deadline="2028-04-02 09:01:38.857176964 +0000 UTC" Jul 05 13:15:18.923230 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:18.923188 2578 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15283h46m19.933991954s" Jul 05 13:15:19.002781 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:19.002751 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dxwj6" Jul 05 13:15:19.002929 ip-10-0-130-191 kubenswrapper[2578]: E0705 13:15:19.002903 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dxwj6" podUID="5ad92660-5823-4f14-adcc-763d06385f45" Jul 05 13:15:19.017203 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:19.017173 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-b2c6n" event={"ID":"c3c5ffe7-eb78-4b28-abb6-106dd0bbb008","Type":"ContainerStarted","Data":"f096c926fb83466a52bd33ccc5b69c7374bdaa5eb1a62e5848ba192cefc04fda"} Jul 05 13:15:19.018336 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:19.018306 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-656m6" event={"ID":"7bdbd204-0fb7-4ebf-af05-432c370a814b","Type":"ContainerStarted","Data":"29221b33eb2d89414004251722e6ad76dbb04cfda3865e9cf8644e619d375e8c"} Jul 05 13:15:19.019439 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:19.019415 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-lq7qp" event={"ID":"eda62bac-3d03-4f52-b055-a77147e9ee97","Type":"ContainerStarted","Data":"287ec4b5d0e2a2dd3c08177d38ddc723df8b92699cb45a959c4a970077d9bd89"} Jul 05 13:15:19.020566 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:19.020543 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mkr9r" event={"ID":"5386321b-a286-4e02-9b76-b81ed7a920ea","Type":"ContainerStarted","Data":"95ec5c59b72f4053959b67c93d961201c6f69b36944da18dac047649c5d4d5bf"} Jul 05 13:15:19.240357 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:19.240321 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb14dc74b_da46_4d14_8fd6_fd9357838fe2.slice/crio-7a03cab59fd413e834b56357979e32714a14b2c59f3e2649ba3161fba815ecf2 WatchSource:0}: Error finding container 7a03cab59fd413e834b56357979e32714a14b2c59f3e2649ba3161fba815ecf2: Status 404 returned error can't find the container with id 7a03cab59fd413e834b56357979e32714a14b2c59f3e2649ba3161fba815ecf2 Jul 05 13:15:19.290107 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:19.290074 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb9962dff_a80f_4ece_bb92_b9761bcf6334.slice/crio-235fc8403519da9f1d8c1ab970c3ed7da0978bbe2cf75e137f81dab109c2d57c WatchSource:0}: Error finding container 235fc8403519da9f1d8c1ab970c3ed7da0978bbe2cf75e137f81dab109c2d57c: Status 404 returned error can't find the container with id 235fc8403519da9f1d8c1ab970c3ed7da0978bbe2cf75e137f81dab109c2d57c Jul 05 13:15:19.291188 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:19.291167 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod545d8134_4ad2_4c62_b302_64b8e10bc493.slice/crio-928053e8448160acadd5855d3e8f914f53849eeb73b958817a29ffd5dc13f993 WatchSource:0}: Error finding container 928053e8448160acadd5855d3e8f914f53849eeb73b958817a29ffd5dc13f993: Status 404 returned error can't find the container with id 928053e8448160acadd5855d3e8f914f53849eeb73b958817a29ffd5dc13f993 Jul 05 13:15:19.291783 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:19.291764 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode8e7535a_8f4e_4fd6_95d2_4cb8888b284b.slice/crio-5e4b598631656465e0de1d7df5ac65eea2794045faf497bfd96b7cbcc86afacb WatchSource:0}: Error finding container 5e4b598631656465e0de1d7df5ac65eea2794045faf497bfd96b7cbcc86afacb: Status 404 returned error can't find the container with id 5e4b598631656465e0de1d7df5ac65eea2794045faf497bfd96b7cbcc86afacb Jul 05 13:15:19.295455 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:19.295427 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc01b0489_2ef4_4fbe_9f9e_1870bc580ee9.slice/crio-f784bea6ee881ebdf0e3f98f715f5a524098c7dcc531bd18773e766b76d74cbc WatchSource:0}: Error finding container f784bea6ee881ebdf0e3f98f715f5a524098c7dcc531bd18773e766b76d74cbc: Status 404 returned error can't find the container with id f784bea6ee881ebdf0e3f98f715f5a524098c7dcc531bd18773e766b76d74cbc Jul 05 13:15:19.601999 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:19.601312 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5ad92660-5823-4f14-adcc-763d06385f45-metrics-certs\") pod \"network-metrics-daemon-dxwj6\" (UID: \"5ad92660-5823-4f14-adcc-763d06385f45\") " pod="openshift-multus/network-metrics-daemon-dxwj6" Jul 05 13:15:19.601999 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:19.601568 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-czl8p\" (UniqueName: \"kubernetes.io/projected/50dc9dcd-a00c-4f85-b3dc-dbac06316c3e-kube-api-access-czl8p\") pod \"network-check-target-jldjr\" (UID: \"50dc9dcd-a00c-4f85-b3dc-dbac06316c3e\") " pod="openshift-network-diagnostics/network-check-target-jldjr" Jul 05 13:15:19.601999 ip-10-0-130-191 kubenswrapper[2578]: E0705 13:15:19.601575 2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 05 13:15:19.601999 ip-10-0-130-191 kubenswrapper[2578]: E0705 13:15:19.601705 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5ad92660-5823-4f14-adcc-763d06385f45-metrics-certs podName:5ad92660-5823-4f14-adcc-763d06385f45 nodeName:}" failed. No retries permitted until 2026-07-05 13:15:21.601683609 +0000 UTC m=+6.060246709 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5ad92660-5823-4f14-adcc-763d06385f45-metrics-certs") pod "network-metrics-daemon-dxwj6" (UID: "5ad92660-5823-4f14-adcc-763d06385f45") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 05 13:15:19.601999 ip-10-0-130-191 kubenswrapper[2578]: E0705 13:15:19.601711 2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 05 13:15:19.601999 ip-10-0-130-191 kubenswrapper[2578]: E0705 13:15:19.601762 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4ff28158-61b2-4cb9-8b92-2ac0793c23bb-original-pull-secret podName:4ff28158-61b2-4cb9-8b92-2ac0793c23bb nodeName:}" failed. No retries permitted until 2026-07-05 13:15:21.601746405 +0000 UTC m=+6.060309488 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/4ff28158-61b2-4cb9-8b92-2ac0793c23bb-original-pull-secret") pod "global-pull-secret-syncer-72444" (UID: "4ff28158-61b2-4cb9-8b92-2ac0793c23bb") : object "kube-system"/"original-pull-secret" not registered Jul 05 13:15:19.601999 ip-10-0-130-191 kubenswrapper[2578]: E0705 13:15:19.601838 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 05 13:15:19.601999 ip-10-0-130-191 kubenswrapper[2578]: E0705 13:15:19.601849 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 05 13:15:19.601999 ip-10-0-130-191 kubenswrapper[2578]: E0705 13:15:19.601860 2578 projected.go:196] Error preparing data for projected volume kube-api-access-czl8p for pod openshift-network-diagnostics/network-check-target-jldjr: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 05 13:15:19.601999 ip-10-0-130-191 kubenswrapper[2578]: E0705 13:15:19.601891 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/50dc9dcd-a00c-4f85-b3dc-dbac06316c3e-kube-api-access-czl8p podName:50dc9dcd-a00c-4f85-b3dc-dbac06316c3e nodeName:}" failed. No retries permitted until 2026-07-05 13:15:21.60188408 +0000 UTC m=+6.060447162 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-czl8p" (UniqueName: "kubernetes.io/projected/50dc9dcd-a00c-4f85-b3dc-dbac06316c3e-kube-api-access-czl8p") pod "network-check-target-jldjr" (UID: "50dc9dcd-a00c-4f85-b3dc-dbac06316c3e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 05 13:15:19.601999 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:19.601634 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4ff28158-61b2-4cb9-8b92-2ac0793c23bb-original-pull-secret\") pod \"global-pull-secret-syncer-72444\" (UID: \"4ff28158-61b2-4cb9-8b92-2ac0793c23bb\") " pod="kube-system/global-pull-secret-syncer-72444" Jul 05 13:15:19.923798 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:19.923656 2578 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-04 13:10:17 +0000 UTC" deadline="2027-12-09 08:12:04.736472602 +0000 UTC" Jul 05 13:15:19.923798 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:19.923716 2578 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12522h56m44.812760983s" Jul 05 13:15:20.008271 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:20.007690 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jldjr" Jul 05 13:15:20.008271 ip-10-0-130-191 kubenswrapper[2578]: E0705 13:15:20.007805 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jldjr" podUID="50dc9dcd-a00c-4f85-b3dc-dbac06316c3e" Jul 05 13:15:20.008271 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:20.007878 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-72444" Jul 05 13:15:20.008271 ip-10-0-130-191 kubenswrapper[2578]: E0705 13:15:20.007946 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-72444" podUID="4ff28158-61b2-4cb9-8b92-2ac0793c23bb" Jul 05 13:15:20.026926 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:20.026850 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7n8f2" event={"ID":"545d8134-4ad2-4c62-b302-64b8e10bc493","Type":"ContainerStarted","Data":"928053e8448160acadd5855d3e8f914f53849eeb73b958817a29ffd5dc13f993"} Jul 05 13:15:20.035740 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:20.035646 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-fds9k" event={"ID":"b9962dff-a80f-4ece-bb92-b9761bcf6334","Type":"ContainerStarted","Data":"235fc8403519da9f1d8c1ab970c3ed7da0978bbe2cf75e137f81dab109c2d57c"} Jul 05 13:15:20.054594 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:20.054553 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-191.ec2.internal" event={"ID":"d82e074a4be1637d97b91534bd80eb21","Type":"ContainerStarted","Data":"b83b945590515867453e5d4f188023a6e815713861870169d1e54745023d51fa"} Jul 05 13:15:20.070790 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:20.070744 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-191.ec2.internal" podStartSLOduration=3.070728618 podStartE2EDuration="3.070728618s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-05 13:15:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 13:15:20.070528269 +0000 UTC m=+4.529091391" watchObservedRunningTime="2026-07-05 13:15:20.070728618 +0000 UTC m=+4.529291726" Jul 05 13:15:20.080033 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:20.079943 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-btxlt" event={"ID":"c01b0489-2ef4-4fbe-9f9e-1870bc580ee9","Type":"ContainerStarted","Data":"f784bea6ee881ebdf0e3f98f715f5a524098c7dcc531bd18773e766b76d74cbc"} Jul 05 13:15:20.086063 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:20.085995 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-kpmmd" event={"ID":"e8e7535a-8f4e-4fd6-95d2-4cb8888b284b","Type":"ContainerStarted","Data":"5e4b598631656465e0de1d7df5ac65eea2794045faf497bfd96b7cbcc86afacb"} Jul 05 13:15:20.090934 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:20.090886 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-g7nxb" event={"ID":"b14dc74b-da46-4d14-8fd6-fd9357838fe2","Type":"ContainerStarted","Data":"7a03cab59fd413e834b56357979e32714a14b2c59f3e2649ba3161fba815ecf2"} Jul 05 13:15:20.097619 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:20.097432 2578 generic.go:356] "Generic (PLEG): container finished" podID="8e4a5e8e2cce016390634f883dc8c308" containerID="4fec3c31818b47be0e581383937ac5edd7315b5aa97946715a254606b1a12c04" exitCode=0 Jul 05 13:15:20.097619 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:20.097468 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-191.ec2.internal" event={"ID":"8e4a5e8e2cce016390634f883dc8c308","Type":"ContainerDied","Data":"4fec3c31818b47be0e581383937ac5edd7315b5aa97946715a254606b1a12c04"} Jul 05 13:15:21.003180 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:21.003144 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dxwj6" Jul 05 13:15:21.003644 ip-10-0-130-191 kubenswrapper[2578]: E0705 13:15:21.003298 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dxwj6" podUID="5ad92660-5823-4f14-adcc-763d06385f45" Jul 05 13:15:21.140061 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:21.139645 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-191.ec2.internal" event={"ID":"8e4a5e8e2cce016390634f883dc8c308","Type":"ContainerStarted","Data":"b4554bdd281134bec8757db2ae2c363598af552692484a0b02630032c5233d3a"} Jul 05 13:15:21.618456 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:21.618408 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5ad92660-5823-4f14-adcc-763d06385f45-metrics-certs\") pod \"network-metrics-daemon-dxwj6\" (UID: \"5ad92660-5823-4f14-adcc-763d06385f45\") " pod="openshift-multus/network-metrics-daemon-dxwj6" Jul 05 13:15:21.618732 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:21.618467 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-czl8p\" (UniqueName: \"kubernetes.io/projected/50dc9dcd-a00c-4f85-b3dc-dbac06316c3e-kube-api-access-czl8p\") pod \"network-check-target-jldjr\" (UID: \"50dc9dcd-a00c-4f85-b3dc-dbac06316c3e\") " pod="openshift-network-diagnostics/network-check-target-jldjr" Jul 05 13:15:21.618732 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:21.618526 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4ff28158-61b2-4cb9-8b92-2ac0793c23bb-original-pull-secret\") pod \"global-pull-secret-syncer-72444\" (UID: \"4ff28158-61b2-4cb9-8b92-2ac0793c23bb\") " pod="kube-system/global-pull-secret-syncer-72444" Jul 05 13:15:21.618732 ip-10-0-130-191 kubenswrapper[2578]: E0705 13:15:21.618679 2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 05 13:15:21.618894 ip-10-0-130-191 kubenswrapper[2578]: E0705 13:15:21.618741 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4ff28158-61b2-4cb9-8b92-2ac0793c23bb-original-pull-secret podName:4ff28158-61b2-4cb9-8b92-2ac0793c23bb nodeName:}" failed. No retries permitted until 2026-07-05 13:15:25.618722671 +0000 UTC m=+10.077285754 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/4ff28158-61b2-4cb9-8b92-2ac0793c23bb-original-pull-secret") pod "global-pull-secret-syncer-72444" (UID: "4ff28158-61b2-4cb9-8b92-2ac0793c23bb") : object "kube-system"/"original-pull-secret" not registered Jul 05 13:15:21.628780 ip-10-0-130-191 kubenswrapper[2578]: E0705 13:15:21.628733 2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 05 13:15:21.628780 ip-10-0-130-191 kubenswrapper[2578]: E0705 13:15:21.628766 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 05 13:15:21.628963 ip-10-0-130-191 kubenswrapper[2578]: E0705 13:15:21.628795 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 05 13:15:21.628963 ip-10-0-130-191 kubenswrapper[2578]: E0705 13:15:21.628809 2578 projected.go:196] Error preparing data for projected volume kube-api-access-czl8p for pod openshift-network-diagnostics/network-check-target-jldjr: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 05 13:15:21.628963 ip-10-0-130-191 kubenswrapper[2578]: E0705 13:15:21.628818 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5ad92660-5823-4f14-adcc-763d06385f45-metrics-certs podName:5ad92660-5823-4f14-adcc-763d06385f45 nodeName:}" failed. No retries permitted until 2026-07-05 13:15:25.628798606 +0000 UTC m=+10.087361693 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5ad92660-5823-4f14-adcc-763d06385f45-metrics-certs") pod "network-metrics-daemon-dxwj6" (UID: "5ad92660-5823-4f14-adcc-763d06385f45") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 05 13:15:21.628963 ip-10-0-130-191 kubenswrapper[2578]: E0705 13:15:21.628868 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/50dc9dcd-a00c-4f85-b3dc-dbac06316c3e-kube-api-access-czl8p podName:50dc9dcd-a00c-4f85-b3dc-dbac06316c3e nodeName:}" failed. No retries permitted until 2026-07-05 13:15:25.628851321 +0000 UTC m=+10.087414417 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-czl8p" (UniqueName: "kubernetes.io/projected/50dc9dcd-a00c-4f85-b3dc-dbac06316c3e-kube-api-access-czl8p") pod "network-check-target-jldjr" (UID: "50dc9dcd-a00c-4f85-b3dc-dbac06316c3e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 05 13:15:22.005456 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:22.005382 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-72444" Jul 05 13:15:22.005900 ip-10-0-130-191 kubenswrapper[2578]: E0705 13:15:22.005505 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-72444" podUID="4ff28158-61b2-4cb9-8b92-2ac0793c23bb" Jul 05 13:15:22.005900 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:22.005877 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jldjr" Jul 05 13:15:22.006005 ip-10-0-130-191 kubenswrapper[2578]: E0705 13:15:22.005956 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jldjr" podUID="50dc9dcd-a00c-4f85-b3dc-dbac06316c3e" Jul 05 13:15:23.003315 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:23.003284 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dxwj6" Jul 05 13:15:23.003480 ip-10-0-130-191 kubenswrapper[2578]: E0705 13:15:23.003418 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dxwj6" podUID="5ad92660-5823-4f14-adcc-763d06385f45" Jul 05 13:15:24.005937 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:24.005903 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jldjr" Jul 05 13:15:24.006370 ip-10-0-130-191 kubenswrapper[2578]: E0705 13:15:24.006028 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jldjr" podUID="50dc9dcd-a00c-4f85-b3dc-dbac06316c3e" Jul 05 13:15:24.006432 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:24.006416 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-72444" Jul 05 13:15:24.006539 ip-10-0-130-191 kubenswrapper[2578]: E0705 13:15:24.006509 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-72444" podUID="4ff28158-61b2-4cb9-8b92-2ac0793c23bb" Jul 05 13:15:25.004041 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:25.003499 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dxwj6" Jul 05 13:15:25.004041 ip-10-0-130-191 kubenswrapper[2578]: E0705 13:15:25.003661 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dxwj6" podUID="5ad92660-5823-4f14-adcc-763d06385f45" Jul 05 13:15:25.656173 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:25.655338 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5ad92660-5823-4f14-adcc-763d06385f45-metrics-certs\") pod \"network-metrics-daemon-dxwj6\" (UID: \"5ad92660-5823-4f14-adcc-763d06385f45\") " pod="openshift-multus/network-metrics-daemon-dxwj6" Jul 05 13:15:25.656173 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:25.655402 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-czl8p\" (UniqueName: \"kubernetes.io/projected/50dc9dcd-a00c-4f85-b3dc-dbac06316c3e-kube-api-access-czl8p\") pod \"network-check-target-jldjr\" (UID: \"50dc9dcd-a00c-4f85-b3dc-dbac06316c3e\") " pod="openshift-network-diagnostics/network-check-target-jldjr" Jul 05 13:15:25.656173 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:25.655452 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4ff28158-61b2-4cb9-8b92-2ac0793c23bb-original-pull-secret\") pod \"global-pull-secret-syncer-72444\" (UID: \"4ff28158-61b2-4cb9-8b92-2ac0793c23bb\") " pod="kube-system/global-pull-secret-syncer-72444" Jul 05 13:15:25.656173 ip-10-0-130-191 kubenswrapper[2578]: E0705 13:15:25.655597 2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 05 13:15:25.656173 ip-10-0-130-191 kubenswrapper[2578]: E0705 13:15:25.655655 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4ff28158-61b2-4cb9-8b92-2ac0793c23bb-original-pull-secret podName:4ff28158-61b2-4cb9-8b92-2ac0793c23bb nodeName:}" failed. No retries permitted until 2026-07-05 13:15:33.655637981 +0000 UTC m=+18.114201083 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/4ff28158-61b2-4cb9-8b92-2ac0793c23bb-original-pull-secret") pod "global-pull-secret-syncer-72444" (UID: "4ff28158-61b2-4cb9-8b92-2ac0793c23bb") : object "kube-system"/"original-pull-secret" not registered Jul 05 13:15:25.656173 ip-10-0-130-191 kubenswrapper[2578]: E0705 13:15:25.655957 2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 05 13:15:25.656173 ip-10-0-130-191 kubenswrapper[2578]: E0705 13:15:25.655995 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5ad92660-5823-4f14-adcc-763d06385f45-metrics-certs podName:5ad92660-5823-4f14-adcc-763d06385f45 nodeName:}" failed. No retries permitted until 2026-07-05 13:15:33.655986018 +0000 UTC m=+18.114549112 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5ad92660-5823-4f14-adcc-763d06385f45-metrics-certs") pod "network-metrics-daemon-dxwj6" (UID: "5ad92660-5823-4f14-adcc-763d06385f45") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 05 13:15:25.656173 ip-10-0-130-191 kubenswrapper[2578]: E0705 13:15:25.656067 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 05 13:15:25.656173 ip-10-0-130-191 kubenswrapper[2578]: E0705 13:15:25.656080 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 05 13:15:25.656173 ip-10-0-130-191 kubenswrapper[2578]: E0705 13:15:25.656091 2578 projected.go:196] Error preparing data for projected volume kube-api-access-czl8p for pod openshift-network-diagnostics/network-check-target-jldjr: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 05 13:15:25.656173 ip-10-0-130-191 kubenswrapper[2578]: E0705 13:15:25.656129 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/50dc9dcd-a00c-4f85-b3dc-dbac06316c3e-kube-api-access-czl8p podName:50dc9dcd-a00c-4f85-b3dc-dbac06316c3e nodeName:}" failed. No retries permitted until 2026-07-05 13:15:33.656117899 +0000 UTC m=+18.114680998 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-czl8p" (UniqueName: "kubernetes.io/projected/50dc9dcd-a00c-4f85-b3dc-dbac06316c3e-kube-api-access-czl8p") pod "network-check-target-jldjr" (UID: "50dc9dcd-a00c-4f85-b3dc-dbac06316c3e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 05 13:15:26.003482 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:26.003406 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jldjr" Jul 05 13:15:26.003642 ip-10-0-130-191 kubenswrapper[2578]: E0705 13:15:26.003492 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jldjr" podUID="50dc9dcd-a00c-4f85-b3dc-dbac06316c3e" Jul 05 13:15:26.003818 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:26.003800 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-72444" Jul 05 13:15:26.003933 ip-10-0-130-191 kubenswrapper[2578]: E0705 13:15:26.003899 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-72444" podUID="4ff28158-61b2-4cb9-8b92-2ac0793c23bb" Jul 05 13:15:27.004268 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:27.003762 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dxwj6" Jul 05 13:15:27.004268 ip-10-0-130-191 kubenswrapper[2578]: E0705 13:15:27.003911 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dxwj6" podUID="5ad92660-5823-4f14-adcc-763d06385f45" Jul 05 13:15:28.003429 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:28.003393 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-72444" Jul 05 13:15:28.003663 ip-10-0-130-191 kubenswrapper[2578]: E0705 13:15:28.003528 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-72444" podUID="4ff28158-61b2-4cb9-8b92-2ac0793c23bb" Jul 05 13:15:28.003920 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:28.003903 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jldjr" Jul 05 13:15:28.003992 ip-10-0-130-191 kubenswrapper[2578]: E0705 13:15:28.003973 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jldjr" podUID="50dc9dcd-a00c-4f85-b3dc-dbac06316c3e" Jul 05 13:15:29.003450 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:29.003414 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dxwj6" Jul 05 13:15:29.003973 ip-10-0-130-191 kubenswrapper[2578]: E0705 13:15:29.003568 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dxwj6" podUID="5ad92660-5823-4f14-adcc-763d06385f45" Jul 05 13:15:30.003812 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:30.003776 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-72444" Jul 05 13:15:30.004251 ip-10-0-130-191 kubenswrapper[2578]: E0705 13:15:30.003903 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-72444" podUID="4ff28158-61b2-4cb9-8b92-2ac0793c23bb" Jul 05 13:15:30.004308 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:30.004255 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jldjr" Jul 05 13:15:30.004357 ip-10-0-130-191 kubenswrapper[2578]: E0705 13:15:30.004346 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jldjr" podUID="50dc9dcd-a00c-4f85-b3dc-dbac06316c3e" Jul 05 13:15:31.002888 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:31.002852 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dxwj6" Jul 05 13:15:31.003065 ip-10-0-130-191 kubenswrapper[2578]: E0705 13:15:31.003007 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dxwj6" podUID="5ad92660-5823-4f14-adcc-763d06385f45" Jul 05 13:15:32.003448 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:32.003410 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jldjr" Jul 05 13:15:32.003928 ip-10-0-130-191 kubenswrapper[2578]: E0705 13:15:32.003556 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jldjr" podUID="50dc9dcd-a00c-4f85-b3dc-dbac06316c3e" Jul 05 13:15:32.003993 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:32.003937 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-72444" Jul 05 13:15:32.004082 ip-10-0-130-191 kubenswrapper[2578]: E0705 13:15:32.004044 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-72444" podUID="4ff28158-61b2-4cb9-8b92-2ac0793c23bb" Jul 05 13:15:33.003365 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:33.003336 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dxwj6" Jul 05 13:15:33.003626 ip-10-0-130-191 kubenswrapper[2578]: E0705 13:15:33.003477 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dxwj6" podUID="5ad92660-5823-4f14-adcc-763d06385f45" Jul 05 13:15:33.711026 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:33.710990 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-czl8p\" (UniqueName: \"kubernetes.io/projected/50dc9dcd-a00c-4f85-b3dc-dbac06316c3e-kube-api-access-czl8p\") pod \"network-check-target-jldjr\" (UID: \"50dc9dcd-a00c-4f85-b3dc-dbac06316c3e\") " pod="openshift-network-diagnostics/network-check-target-jldjr" Jul 05 13:15:33.711317 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:33.711044 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4ff28158-61b2-4cb9-8b92-2ac0793c23bb-original-pull-secret\") pod \"global-pull-secret-syncer-72444\" (UID: \"4ff28158-61b2-4cb9-8b92-2ac0793c23bb\") " pod="kube-system/global-pull-secret-syncer-72444" Jul 05 13:15:33.711317 ip-10-0-130-191 kubenswrapper[2578]: E0705 13:15:33.711144 2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 05 13:15:33.711317 ip-10-0-130-191 kubenswrapper[2578]: E0705 13:15:33.711178 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 05 13:15:33.711317 ip-10-0-130-191 kubenswrapper[2578]: E0705 13:15:33.711201 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4ff28158-61b2-4cb9-8b92-2ac0793c23bb-original-pull-secret podName:4ff28158-61b2-4cb9-8b92-2ac0793c23bb nodeName:}" failed. No retries permitted until 2026-07-05 13:15:49.711183782 +0000 UTC m=+34.169746864 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/4ff28158-61b2-4cb9-8b92-2ac0793c23bb-original-pull-secret") pod "global-pull-secret-syncer-72444" (UID: "4ff28158-61b2-4cb9-8b92-2ac0793c23bb") : object "kube-system"/"original-pull-secret" not registered Jul 05 13:15:33.711317 ip-10-0-130-191 kubenswrapper[2578]: E0705 13:15:33.711202 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 05 13:15:33.711317 ip-10-0-130-191 kubenswrapper[2578]: E0705 13:15:33.711220 2578 projected.go:196] Error preparing data for projected volume kube-api-access-czl8p for pod openshift-network-diagnostics/network-check-target-jldjr: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 05 13:15:33.711317 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:33.711223 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5ad92660-5823-4f14-adcc-763d06385f45-metrics-certs\") pod \"network-metrics-daemon-dxwj6\" (UID: \"5ad92660-5823-4f14-adcc-763d06385f45\") " pod="openshift-multus/network-metrics-daemon-dxwj6" Jul 05 13:15:33.711317 ip-10-0-130-191 kubenswrapper[2578]: E0705 13:15:33.711282 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/50dc9dcd-a00c-4f85-b3dc-dbac06316c3e-kube-api-access-czl8p podName:50dc9dcd-a00c-4f85-b3dc-dbac06316c3e nodeName:}" failed. No retries permitted until 2026-07-05 13:15:49.711266518 +0000 UTC m=+34.169829620 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-czl8p" (UniqueName: "kubernetes.io/projected/50dc9dcd-a00c-4f85-b3dc-dbac06316c3e-kube-api-access-czl8p") pod "network-check-target-jldjr" (UID: "50dc9dcd-a00c-4f85-b3dc-dbac06316c3e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 05 13:15:33.711317 ip-10-0-130-191 kubenswrapper[2578]: E0705 13:15:33.711320 2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 05 13:15:33.711807 ip-10-0-130-191 kubenswrapper[2578]: E0705 13:15:33.711355 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5ad92660-5823-4f14-adcc-763d06385f45-metrics-certs podName:5ad92660-5823-4f14-adcc-763d06385f45 nodeName:}" failed. No retries permitted until 2026-07-05 13:15:49.711343172 +0000 UTC m=+34.169906254 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5ad92660-5823-4f14-adcc-763d06385f45-metrics-certs") pod "network-metrics-daemon-dxwj6" (UID: "5ad92660-5823-4f14-adcc-763d06385f45") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 05 13:15:34.003079 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:34.002999 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jldjr" Jul 05 13:15:34.003246 ip-10-0-130-191 kubenswrapper[2578]: E0705 13:15:34.003122 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jldjr" podUID="50dc9dcd-a00c-4f85-b3dc-dbac06316c3e" Jul 05 13:15:34.003246 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:34.003176 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-72444" Jul 05 13:15:34.003348 ip-10-0-130-191 kubenswrapper[2578]: E0705 13:15:34.003254 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-72444" podUID="4ff28158-61b2-4cb9-8b92-2ac0793c23bb" Jul 05 13:15:35.002903 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:35.002869 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dxwj6" Jul 05 13:15:35.003358 ip-10-0-130-191 kubenswrapper[2578]: E0705 13:15:35.003011 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dxwj6" podUID="5ad92660-5823-4f14-adcc-763d06385f45" Jul 05 13:15:36.004176 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:36.004144 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jldjr" Jul 05 13:15:36.004532 ip-10-0-130-191 kubenswrapper[2578]: E0705 13:15:36.004230 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jldjr" podUID="50dc9dcd-a00c-4f85-b3dc-dbac06316c3e" Jul 05 13:15:36.004532 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:36.004322 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-72444" Jul 05 13:15:36.004532 ip-10-0-130-191 kubenswrapper[2578]: E0705 13:15:36.004437 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-72444" podUID="4ff28158-61b2-4cb9-8b92-2ac0793c23bb" Jul 05 13:15:37.003510 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:37.003475 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dxwj6" Jul 05 13:15:37.003699 ip-10-0-130-191 kubenswrapper[2578]: E0705 13:15:37.003607 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dxwj6" podUID="5ad92660-5823-4f14-adcc-763d06385f45" Jul 05 13:15:38.003310 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:38.003143 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-72444" Jul 05 13:15:38.004017 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:38.003143 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jldjr" Jul 05 13:15:38.004017 ip-10-0-130-191 kubenswrapper[2578]: E0705 13:15:38.003407 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-72444" podUID="4ff28158-61b2-4cb9-8b92-2ac0793c23bb" Jul 05 13:15:38.004017 ip-10-0-130-191 kubenswrapper[2578]: E0705 13:15:38.003437 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jldjr" podUID="50dc9dcd-a00c-4f85-b3dc-dbac06316c3e" Jul 05 13:15:38.171711 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:38.171691 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-656m6_7bdbd204-0fb7-4ebf-af05-432c370a814b/ovn-acl-logging/0.log" Jul 05 13:15:38.171997 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:38.171977 2578 generic.go:356] "Generic (PLEG): container finished" podID="7bdbd204-0fb7-4ebf-af05-432c370a814b" containerID="35f5ea614dd12cde6d7da96bfc74a4f5d88a4b6317ff3168d162bc5a6b4f8173" exitCode=1 Jul 05 13:15:38.172086 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:38.172047 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-656m6" event={"ID":"7bdbd204-0fb7-4ebf-af05-432c370a814b","Type":"ContainerDied","Data":"35f5ea614dd12cde6d7da96bfc74a4f5d88a4b6317ff3168d162bc5a6b4f8173"} Jul 05 13:15:38.172129 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:38.172122 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-656m6" event={"ID":"7bdbd204-0fb7-4ebf-af05-432c370a814b","Type":"ContainerStarted","Data":"ee00569539d912013b018de96a762beeb2a680455e1d23169ea972819cc8f237"} Jul 05 13:15:38.173325 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:38.173306 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-lq7qp" event={"ID":"eda62bac-3d03-4f52-b055-a77147e9ee97","Type":"ContainerStarted","Data":"728cb54e35ce042d2d131012ea699e10230a98dbdc6f6658cd9cfea8c079d67d"} Jul 05 13:15:38.174795 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:38.174773 2578 generic.go:356] "Generic (PLEG): container finished" podID="545d8134-4ad2-4c62-b302-64b8e10bc493" containerID="465a8dddaf1fd94e34ccc03832db71a60a58e87ed05956334466029d431f9887" exitCode=0 Jul 05 13:15:38.174879 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:38.174844 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7n8f2" event={"ID":"545d8134-4ad2-4c62-b302-64b8e10bc493","Type":"ContainerDied","Data":"465a8dddaf1fd94e34ccc03832db71a60a58e87ed05956334466029d431f9887"} Jul 05 13:15:38.176564 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:38.176544 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-fds9k" event={"ID":"b9962dff-a80f-4ece-bb92-b9761bcf6334","Type":"ContainerStarted","Data":"70f859358cd9ec31337a83959e6051df45a438bf6637a28ffcc643848ee0cc23"} Jul 05 13:15:38.178140 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:38.178026 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mkr9r" event={"ID":"5386321b-a286-4e02-9b76-b81ed7a920ea","Type":"ContainerStarted","Data":"fc3429a2537f5c6ee86f8af8591cb0c6a762c50fe1704af42221c909adbacec8"} Jul 05 13:15:38.179422 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:38.179404 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-btxlt" event={"ID":"c01b0489-2ef4-4fbe-9f9e-1870bc580ee9","Type":"ContainerStarted","Data":"eebec60aec7118e047437e99131ed92f5691dd4aa99ac7f8cafabf24e0a4aad0"} Jul 05 13:15:38.180731 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:38.180714 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-kpmmd" event={"ID":"e8e7535a-8f4e-4fd6-95d2-4cb8888b284b","Type":"ContainerStarted","Data":"c9ee088b4ba039d6b12ebf6949995656e68174b3f6025846d7fb3ee979a27526"} Jul 05 13:15:38.182287 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:38.182269 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-g7nxb" event={"ID":"b14dc74b-da46-4d14-8fd6-fd9357838fe2","Type":"ContainerStarted","Data":"389bee17e584fec82e63ca5d4e3a4ccfd41fcc8df459b6311991bd945cb83826"} Jul 05 13:15:38.191968 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:38.191920 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-191.ec2.internal" podStartSLOduration=21.19190821 podStartE2EDuration="21.19190821s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-05 13:15:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 13:15:21.155598984 +0000 UTC m=+5.614162087" watchObservedRunningTime="2026-07-05 13:15:38.19190821 +0000 UTC m=+22.650471325" Jul 05 13:15:38.192218 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:38.192200 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-lq7qp" podStartSLOduration=2.803667109 podStartE2EDuration="22.192193382s" totalImagesPullingTime="19.388526273s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-05 13:15:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 13:15:38.19156811 +0000 UTC m=+22.650131215" watchObservedRunningTime="2026-07-05 13:15:38.192193382 +0000 UTC m=+22.650756485" Jul 05 13:15:38.206561 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:38.206527 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-kpmmd" podStartSLOduration=3.937303953 podStartE2EDuration="22.206515378s" totalImagesPullingTime="18.269211425s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 13:15:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 13:15:38.206512782 +0000 UTC m=+22.665075885" watchObservedRunningTime="2026-07-05 13:15:38.206515378 +0000 UTC m=+22.665078481" Jul 05 13:15:38.268539 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:38.268366 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-g7nxb" podStartSLOduration=3.966260493 podStartE2EDuration="22.268345249s" totalImagesPullingTime="18.302084756s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-05 13:15:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 13:15:38.247615358 +0000 UTC m=+22.706178461" watchObservedRunningTime="2026-07-05 13:15:38.268345249 +0000 UTC m=+22.726908353" Jul 05 13:15:38.268951 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:38.268917 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-btxlt" podStartSLOduration=4.000393797 podStartE2EDuration="22.268905042s" totalImagesPullingTime="18.268511245s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-05 13:15:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 13:15:38.268030926 +0000 UTC m=+22.726594026" watchObservedRunningTime="2026-07-05 13:15:38.268905042 +0000 UTC m=+22.727468147" Jul 05 13:15:38.289176 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:38.289129 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-fds9k" podStartSLOduration=4.015834005 podStartE2EDuration="22.289112113s" totalImagesPullingTime="18.273278108s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-05 13:15:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 13:15:38.289112952 +0000 UTC m=+22.747676060" watchObservedRunningTime="2026-07-05 13:15:38.289112113 +0000 UTC m=+22.747675210" Jul 05 13:15:39.002841 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:39.002802 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dxwj6" Jul 05 13:15:39.003050 ip-10-0-130-191 kubenswrapper[2578]: E0705 13:15:39.002935 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dxwj6" podUID="5ad92660-5823-4f14-adcc-763d06385f45" Jul 05 13:15:39.185633 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:39.185600 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-b2c6n" event={"ID":"c3c5ffe7-eb78-4b28-abb6-106dd0bbb008","Type":"ContainerStarted","Data":"05026f0635ccff2bf15829aab512a9ad51bb2393a11492c0cd7f8ef24ed21a92"} Jul 05 13:15:39.188429 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:39.188412 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-656m6_7bdbd204-0fb7-4ebf-af05-432c370a814b/ovn-acl-logging/0.log" Jul 05 13:15:39.188868 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:39.188843 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-656m6" event={"ID":"7bdbd204-0fb7-4ebf-af05-432c370a814b","Type":"ContainerStarted","Data":"d2598d69d8fd012eb5be570fdfe98710e524b92eaf5ab86b28cea5839c06646c"} Jul 05 13:15:39.188957 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:39.188903 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-656m6" event={"ID":"7bdbd204-0fb7-4ebf-af05-432c370a814b","Type":"ContainerStarted","Data":"d13f52a058cafca6d003d255b3b73d07c79f3e990d2f025c0a9d8b44b972d7c6"} Jul 05 13:15:39.188957 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:39.188934 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-656m6" event={"ID":"7bdbd204-0fb7-4ebf-af05-432c370a814b","Type":"ContainerStarted","Data":"c8a04f51d307443953a1dcd2a935cbfd2cce567054d2bbf69ade2ff54eff88dd"} Jul 05 13:15:39.189068 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:39.188967 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-656m6" event={"ID":"7bdbd204-0fb7-4ebf-af05-432c370a814b","Type":"ContainerStarted","Data":"1c2b97f4bd293fc6467b21d9e4a90aa396efd543f3fd3ecfbb25f2e260b6491f"} Jul 05 13:15:39.313741 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:39.313716 2578 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 05 13:15:39.949101 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:39.948981 2578 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-05T13:15:39.313737017Z","UUID":"f2e3d5ea-12ca-4471-97ae-976e4898ae88","Handler":null,"Name":"","Endpoint":""} Jul 05 13:15:39.950969 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:39.950943 2578 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 05 13:15:39.951109 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:39.951002 2578 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 05 13:15:40.003752 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:40.003722 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-72444" Jul 05 13:15:40.003752 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:40.003759 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jldjr" Jul 05 13:15:40.003991 ip-10-0-130-191 kubenswrapper[2578]: E0705 13:15:40.003852 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-72444" podUID="4ff28158-61b2-4cb9-8b92-2ac0793c23bb" Jul 05 13:15:40.004213 ip-10-0-130-191 kubenswrapper[2578]: E0705 13:15:40.004178 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jldjr" podUID="50dc9dcd-a00c-4f85-b3dc-dbac06316c3e" Jul 05 13:15:40.194605 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:40.193430 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mkr9r" event={"ID":"5386321b-a286-4e02-9b76-b81ed7a920ea","Type":"ContainerStarted","Data":"626f48617916b5efc08f95a2de4390b89f267a566ccb7226721707aed61b8f78"} Jul 05 13:15:41.003535 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:41.003300 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dxwj6" Jul 05 13:15:41.003729 ip-10-0-130-191 kubenswrapper[2578]: E0705 13:15:41.003614 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dxwj6" podUID="5ad92660-5823-4f14-adcc-763d06385f45" Jul 05 13:15:41.199668 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:41.199635 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-656m6_7bdbd204-0fb7-4ebf-af05-432c370a814b/ovn-acl-logging/0.log" Jul 05 13:15:41.200398 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:41.200367 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-656m6" event={"ID":"7bdbd204-0fb7-4ebf-af05-432c370a814b","Type":"ContainerStarted","Data":"dbc02809972da15a13c20294b64765907ecf7c5a18d6bfee9c964ebe0ffe515c"} Jul 05 13:15:41.202558 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:41.202512 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mkr9r" event={"ID":"5386321b-a286-4e02-9b76-b81ed7a920ea","Type":"ContainerStarted","Data":"f460944913d5f5960f7ace99a9ffbf63c9ca58f2305e8e78e9746e4fe99b13f0"} Jul 05 13:15:41.223697 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:41.223644 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-b2c6n" podStartSLOduration=5.911720064 podStartE2EDuration="25.223629691s" totalImagesPullingTime="19.311909627s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-05 13:15:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 13:15:39.199959335 +0000 UTC m=+23.658522440" watchObservedRunningTime="2026-07-05 13:15:41.223629691 +0000 UTC m=+25.682192795" Jul 05 13:15:41.820093 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:41.820055 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-kpmmd" Jul 05 13:15:41.820675 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:41.820656 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-kpmmd" Jul 05 13:15:41.835834 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:41.835794 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mkr9r" podStartSLOduration=3.648598699 podStartE2EDuration="25.835781376s" totalImagesPullingTime="22.187182677s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-05 13:15:16 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 13:15:41.223288283 +0000 UTC m=+25.681851387" watchObservedRunningTime="2026-07-05 13:15:41.835781376 +0000 UTC m=+26.294344480" Jul 05 13:15:42.003088 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:42.003057 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-72444" Jul 05 13:15:42.003264 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:42.003058 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jldjr" Jul 05 13:15:42.003264 ip-10-0-130-191 kubenswrapper[2578]: E0705 13:15:42.003208 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-72444" podUID="4ff28158-61b2-4cb9-8b92-2ac0793c23bb" Jul 05 13:15:42.003376 ip-10-0-130-191 kubenswrapper[2578]: E0705 13:15:42.003326 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jldjr" podUID="50dc9dcd-a00c-4f85-b3dc-dbac06316c3e" Jul 05 13:15:42.206197 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:42.205769 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-kpmmd" Jul 05 13:15:42.206197 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:42.206162 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-kpmmd" Jul 05 13:15:43.003416 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:43.003382 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dxwj6" Jul 05 13:15:43.003620 ip-10-0-130-191 kubenswrapper[2578]: E0705 13:15:43.003506 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dxwj6" podUID="5ad92660-5823-4f14-adcc-763d06385f45" Jul 05 13:15:44.003005 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:44.002968 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-72444" Jul 05 13:15:44.003667 ip-10-0-130-191 kubenswrapper[2578]: E0705 13:15:44.003121 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-72444" podUID="4ff28158-61b2-4cb9-8b92-2ac0793c23bb" Jul 05 13:15:44.003667 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:44.003138 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jldjr" Jul 05 13:15:44.003667 ip-10-0-130-191 kubenswrapper[2578]: E0705 13:15:44.003252 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jldjr" podUID="50dc9dcd-a00c-4f85-b3dc-dbac06316c3e" Jul 05 13:15:45.003730 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:45.003490 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dxwj6" Jul 05 13:15:45.004211 ip-10-0-130-191 kubenswrapper[2578]: E0705 13:15:45.003756 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dxwj6" podUID="5ad92660-5823-4f14-adcc-763d06385f45" Jul 05 13:15:45.214830 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:45.214795 2578 generic.go:356] "Generic (PLEG): container finished" podID="545d8134-4ad2-4c62-b302-64b8e10bc493" containerID="5fc1584c72d8652d41e5415726c8c9ad82af3a243c0861f2b46293580c775fcd" exitCode=0 Jul 05 13:15:45.215004 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:45.214885 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7n8f2" event={"ID":"545d8134-4ad2-4c62-b302-64b8e10bc493","Type":"ContainerDied","Data":"5fc1584c72d8652d41e5415726c8c9ad82af3a243c0861f2b46293580c775fcd"} Jul 05 13:15:45.218434 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:45.218414 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-656m6_7bdbd204-0fb7-4ebf-af05-432c370a814b/ovn-acl-logging/0.log" Jul 05 13:15:45.218807 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:45.218782 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-656m6" event={"ID":"7bdbd204-0fb7-4ebf-af05-432c370a814b","Type":"ContainerStarted","Data":"b0c74b1eebc83280aa21f29ae1750a3a76f07998cca4249ec1e9fafb3421dfcd"} Jul 05 13:15:45.218995 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:45.218983 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-656m6" Jul 05 13:15:45.219038 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:45.219021 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-656m6" Jul 05 13:15:45.219227 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:45.219214 2578 scope.go:117] "RemoveContainer" containerID="35f5ea614dd12cde6d7da96bfc74a4f5d88a4b6317ff3168d162bc5a6b4f8173" Jul 05 13:15:45.234566 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:45.234546 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-656m6" Jul 05 13:15:46.004483 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:46.004452 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-72444" Jul 05 13:15:46.004960 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:46.004564 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jldjr" Jul 05 13:15:46.004960 ip-10-0-130-191 kubenswrapper[2578]: E0705 13:15:46.004593 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-72444" podUID="4ff28158-61b2-4cb9-8b92-2ac0793c23bb" Jul 05 13:15:46.004960 ip-10-0-130-191 kubenswrapper[2578]: E0705 13:15:46.004656 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jldjr" podUID="50dc9dcd-a00c-4f85-b3dc-dbac06316c3e" Jul 05 13:15:46.224762 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:46.224732 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-656m6_7bdbd204-0fb7-4ebf-af05-432c370a814b/ovn-acl-logging/0.log" Jul 05 13:15:46.225077 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:46.225057 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-656m6" event={"ID":"7bdbd204-0fb7-4ebf-af05-432c370a814b","Type":"ContainerStarted","Data":"4acffe762c4d3b27c6cb7227de6dee8ab7e49de1556e43df563bcf30ceb5df01"} Jul 05 13:15:46.225248 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:46.225234 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-656m6" Jul 05 13:15:46.241329 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:46.241302 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-656m6" Jul 05 13:15:46.256092 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:46.255995 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-656m6" podStartSLOduration=10.61631449 podStartE2EDuration="30.255980076s" totalImagesPullingTime="19.639665586s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-05 13:15:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 13:15:46.255604786 +0000 UTC m=+30.714167889" watchObservedRunningTime="2026-07-05 13:15:46.255980076 +0000 UTC m=+30.714543178" Jul 05 13:15:46.382655 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:46.382623 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-72444"] Jul 05 13:15:46.382826 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:46.382733 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-72444" Jul 05 13:15:46.382883 ip-10-0-130-191 kubenswrapper[2578]: E0705 13:15:46.382831 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-72444" podUID="4ff28158-61b2-4cb9-8b92-2ac0793c23bb" Jul 05 13:15:46.391142 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:46.389002 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-jldjr"] Jul 05 13:15:46.391142 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:46.389093 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-dxwj6"] Jul 05 13:15:46.391142 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:46.389195 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dxwj6" Jul 05 13:15:46.391142 ip-10-0-130-191 kubenswrapper[2578]: E0705 13:15:46.389320 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dxwj6" podUID="5ad92660-5823-4f14-adcc-763d06385f45" Jul 05 13:15:46.391142 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:46.389389 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jldjr" Jul 05 13:15:46.391142 ip-10-0-130-191 kubenswrapper[2578]: E0705 13:15:46.389475 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jldjr" podUID="50dc9dcd-a00c-4f85-b3dc-dbac06316c3e" Jul 05 13:15:47.229087 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:47.229000 2578 generic.go:356] "Generic (PLEG): container finished" podID="545d8134-4ad2-4c62-b302-64b8e10bc493" containerID="9e7f2342ffa15291b737a004ea7186bbf46b8d61da214fd2e4ce8abeb20ad32c" exitCode=0 Jul 05 13:15:47.229087 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:47.229045 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7n8f2" event={"ID":"545d8134-4ad2-4c62-b302-64b8e10bc493","Type":"ContainerDied","Data":"9e7f2342ffa15291b737a004ea7186bbf46b8d61da214fd2e4ce8abeb20ad32c"} Jul 05 13:15:48.003394 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:48.003306 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-72444" Jul 05 13:15:48.003394 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:48.003349 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jldjr" Jul 05 13:15:48.003631 ip-10-0-130-191 kubenswrapper[2578]: E0705 13:15:48.003416 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-72444" podUID="4ff28158-61b2-4cb9-8b92-2ac0793c23bb" Jul 05 13:15:48.003631 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:48.003307 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dxwj6" Jul 05 13:15:48.003631 ip-10-0-130-191 kubenswrapper[2578]: E0705 13:15:48.003525 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jldjr" podUID="50dc9dcd-a00c-4f85-b3dc-dbac06316c3e" Jul 05 13:15:48.003770 ip-10-0-130-191 kubenswrapper[2578]: E0705 13:15:48.003645 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dxwj6" podUID="5ad92660-5823-4f14-adcc-763d06385f45" Jul 05 13:15:49.235321 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:49.235288 2578 generic.go:356] "Generic (PLEG): container finished" podID="545d8134-4ad2-4c62-b302-64b8e10bc493" containerID="2f80adc43cd19a3b7fb2c8f3401c7bca7d7a66c91bf338182c59d4ea3044e93b" exitCode=0 Jul 05 13:15:49.235789 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:49.235341 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7n8f2" event={"ID":"545d8134-4ad2-4c62-b302-64b8e10bc493","Type":"ContainerDied","Data":"2f80adc43cd19a3b7fb2c8f3401c7bca7d7a66c91bf338182c59d4ea3044e93b"} Jul 05 13:15:49.736100 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:49.736065 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4ff28158-61b2-4cb9-8b92-2ac0793c23bb-original-pull-secret\") pod \"global-pull-secret-syncer-72444\" (UID: \"4ff28158-61b2-4cb9-8b92-2ac0793c23bb\") " pod="kube-system/global-pull-secret-syncer-72444" Jul 05 13:15:49.736247 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:49.736107 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5ad92660-5823-4f14-adcc-763d06385f45-metrics-certs\") pod \"network-metrics-daemon-dxwj6\" (UID: \"5ad92660-5823-4f14-adcc-763d06385f45\") " pod="openshift-multus/network-metrics-daemon-dxwj6" Jul 05 13:15:49.736247 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:49.736132 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-czl8p\" (UniqueName: \"kubernetes.io/projected/50dc9dcd-a00c-4f85-b3dc-dbac06316c3e-kube-api-access-czl8p\") pod \"network-check-target-jldjr\" (UID: \"50dc9dcd-a00c-4f85-b3dc-dbac06316c3e\") " pod="openshift-network-diagnostics/network-check-target-jldjr" Jul 05 13:15:49.736247 ip-10-0-130-191 kubenswrapper[2578]: E0705 13:15:49.736207 2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 05 13:15:49.736247 ip-10-0-130-191 kubenswrapper[2578]: E0705 13:15:49.736222 2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 05 13:15:49.736247 ip-10-0-130-191 kubenswrapper[2578]: E0705 13:15:49.736229 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 05 13:15:49.736390 ip-10-0-130-191 kubenswrapper[2578]: E0705 13:15:49.736252 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 05 13:15:49.736390 ip-10-0-130-191 kubenswrapper[2578]: E0705 13:15:49.736263 2578 projected.go:196] Error preparing data for projected volume kube-api-access-czl8p for pod openshift-network-diagnostics/network-check-target-jldjr: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 05 13:15:49.736390 ip-10-0-130-191 kubenswrapper[2578]: E0705 13:15:49.736263 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4ff28158-61b2-4cb9-8b92-2ac0793c23bb-original-pull-secret podName:4ff28158-61b2-4cb9-8b92-2ac0793c23bb nodeName:}" failed. No retries permitted until 2026-07-05 13:16:21.736248471 +0000 UTC m=+66.194811553 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/4ff28158-61b2-4cb9-8b92-2ac0793c23bb-original-pull-secret") pod "global-pull-secret-syncer-72444" (UID: "4ff28158-61b2-4cb9-8b92-2ac0793c23bb") : object "kube-system"/"original-pull-secret" not registered Jul 05 13:15:49.736390 ip-10-0-130-191 kubenswrapper[2578]: E0705 13:15:49.736302 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5ad92660-5823-4f14-adcc-763d06385f45-metrics-certs podName:5ad92660-5823-4f14-adcc-763d06385f45 nodeName:}" failed. No retries permitted until 2026-07-05 13:16:21.736291862 +0000 UTC m=+66.194854944 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5ad92660-5823-4f14-adcc-763d06385f45-metrics-certs") pod "network-metrics-daemon-dxwj6" (UID: "5ad92660-5823-4f14-adcc-763d06385f45") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 05 13:15:49.736390 ip-10-0-130-191 kubenswrapper[2578]: E0705 13:15:49.736315 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/50dc9dcd-a00c-4f85-b3dc-dbac06316c3e-kube-api-access-czl8p podName:50dc9dcd-a00c-4f85-b3dc-dbac06316c3e nodeName:}" failed. No retries permitted until 2026-07-05 13:16:21.736306805 +0000 UTC m=+66.194869891 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-czl8p" (UniqueName: "kubernetes.io/projected/50dc9dcd-a00c-4f85-b3dc-dbac06316c3e-kube-api-access-czl8p") pod "network-check-target-jldjr" (UID: "50dc9dcd-a00c-4f85-b3dc-dbac06316c3e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 05 13:15:50.002920 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:50.002884 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jldjr" Jul 05 13:15:50.003074 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:50.002930 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-72444" Jul 05 13:15:50.003074 ip-10-0-130-191 kubenswrapper[2578]: E0705 13:15:50.003020 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jldjr" podUID="50dc9dcd-a00c-4f85-b3dc-dbac06316c3e" Jul 05 13:15:50.003074 ip-10-0-130-191 kubenswrapper[2578]: E0705 13:15:50.003059 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-72444" podUID="4ff28158-61b2-4cb9-8b92-2ac0793c23bb" Jul 05 13:15:50.003239 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:50.003118 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dxwj6" Jul 05 13:15:50.003239 ip-10-0-130-191 kubenswrapper[2578]: E0705 13:15:50.003220 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dxwj6" podUID="5ad92660-5823-4f14-adcc-763d06385f45" Jul 05 13:15:51.349712 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:51.349683 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-191.ec2.internal" event="NodeReady" Jul 05 13:15:51.350173 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:51.349815 2578 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 05 13:15:51.382967 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:51.382942 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-57b9f5fbdc-gsjvv"] Jul 05 13:15:51.385935 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:51.385914 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-57b9f5fbdc-gsjvv" Jul 05 13:15:51.386079 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:51.386002 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-6bffdf9778-ljfps"] Jul 05 13:15:51.388673 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:51.388628 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 05 13:15:51.388673 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:51.388638 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 05 13:15:51.388673 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:51.388661 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-6fcfs\"" Jul 05 13:15:51.388922 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:51.388908 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 05 13:15:51.392558 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:51.389943 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6bffdf9778-ljfps" Jul 05 13:15:51.394237 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:51.394214 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\"" Jul 05 13:15:51.394340 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:51.394270 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\"" Jul 05 13:15:51.394406 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:51.394346 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-9lt9j\"" Jul 05 13:15:51.395816 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:51.395495 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 05 13:15:51.401236 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:51.400948 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6bffdf9778-ljfps"] Jul 05 13:15:51.401236 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:51.401218 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-57b9f5fbdc-gsjvv"] Jul 05 13:15:51.402679 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:51.402654 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-w8bmv"] Jul 05 13:15:51.406564 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:51.406545 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-qn45h"] Jul 05 13:15:51.406672 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:51.406656 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-w8bmv" Jul 05 13:15:51.409369 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:51.409341 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 05 13:15:51.409691 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:51.409669 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 05 13:15:51.409813 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:51.409701 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-9gzst\"" Jul 05 13:15:51.409878 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:51.409821 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-qn45h" Jul 05 13:15:51.412094 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:51.412076 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 05 13:15:51.412373 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:51.412359 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 05 13:15:51.412559 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:51.412540 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-5scw5\"" Jul 05 13:15:51.412641 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:51.412375 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 05 13:15:51.414697 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:51.414679 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-qn45h"] Jul 05 13:15:51.417516 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:51.417486 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-w8bmv"] Jul 05 13:15:51.547047 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:51.547021 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/30164fef-3f92-4b9d-8c8f-ea7e6a3f1047-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-ljfps\" (UID: \"30164fef-3f92-4b9d-8c8f-ea7e6a3f1047\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-ljfps" Jul 05 13:15:51.547210 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:51.547064 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/d933ee11-a974-467b-9dce-f898bcf3863b-registry-certificates\") pod \"image-registry-57b9f5fbdc-gsjvv\" (UID: \"d933ee11-a974-467b-9dce-f898bcf3863b\") " pod="openshift-image-registry/image-registry-57b9f5fbdc-gsjvv" Jul 05 13:15:51.547210 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:51.547083 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d933ee11-a974-467b-9dce-f898bcf3863b-registry-tls\") pod \"image-registry-57b9f5fbdc-gsjvv\" (UID: \"d933ee11-a974-467b-9dce-f898bcf3863b\") " pod="openshift-image-registry/image-registry-57b9f5fbdc-gsjvv" Jul 05 13:15:51.547210 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:51.547102 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/79875c5a-6f9e-4d4f-9de9-5d974249f92b-tmp-dir\") pod \"dns-default-w8bmv\" (UID: \"79875c5a-6f9e-4d4f-9de9-5d974249f92b\") " pod="openshift-dns/dns-default-w8bmv" Jul 05 13:15:51.547376 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:51.547214 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/d933ee11-a974-467b-9dce-f898bcf3863b-installation-pull-secrets\") pod \"image-registry-57b9f5fbdc-gsjvv\" (UID: \"d933ee11-a974-467b-9dce-f898bcf3863b\") " pod="openshift-image-registry/image-registry-57b9f5fbdc-gsjvv" Jul 05 13:15:51.547376 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:51.547250 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d933ee11-a974-467b-9dce-f898bcf3863b-bound-sa-token\") pod \"image-registry-57b9f5fbdc-gsjvv\" (UID: \"d933ee11-a974-467b-9dce-f898bcf3863b\") " pod="openshift-image-registry/image-registry-57b9f5fbdc-gsjvv" Jul 05 13:15:51.547376 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:51.547275 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/853f791d-d350-4626-989e-270b15b79453-cert\") pod \"ingress-canary-qn45h\" (UID: \"853f791d-d350-4626-989e-270b15b79453\") " pod="openshift-ingress-canary/ingress-canary-qn45h" Jul 05 13:15:51.547376 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:51.547300 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/79875c5a-6f9e-4d4f-9de9-5d974249f92b-config-volume\") pod \"dns-default-w8bmv\" (UID: \"79875c5a-6f9e-4d4f-9de9-5d974249f92b\") " pod="openshift-dns/dns-default-w8bmv" Jul 05 13:15:51.547376 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:51.547324 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9lt68\" (UniqueName: \"kubernetes.io/projected/853f791d-d350-4626-989e-270b15b79453-kube-api-access-9lt68\") pod \"ingress-canary-qn45h\" (UID: \"853f791d-d350-4626-989e-270b15b79453\") " pod="openshift-ingress-canary/ingress-canary-qn45h" Jul 05 13:15:51.547376 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:51.547351 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/d933ee11-a974-467b-9dce-f898bcf3863b-image-registry-private-configuration\") pod \"image-registry-57b9f5fbdc-gsjvv\" (UID: \"d933ee11-a974-467b-9dce-f898bcf3863b\") " pod="openshift-image-registry/image-registry-57b9f5fbdc-gsjvv" Jul 05 13:15:51.547562 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:51.547384 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/d933ee11-a974-467b-9dce-f898bcf3863b-trusted-ca\") pod \"image-registry-57b9f5fbdc-gsjvv\" (UID: \"d933ee11-a974-467b-9dce-f898bcf3863b\") " pod="openshift-image-registry/image-registry-57b9f5fbdc-gsjvv" Jul 05 13:15:51.547562 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:51.547438 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/d933ee11-a974-467b-9dce-f898bcf3863b-ca-trust-extracted\") pod \"image-registry-57b9f5fbdc-gsjvv\" (UID: \"d933ee11-a974-467b-9dce-f898bcf3863b\") " pod="openshift-image-registry/image-registry-57b9f5fbdc-gsjvv" Jul 05 13:15:51.547562 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:51.547473 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k2cfn\" (UniqueName: \"kubernetes.io/projected/d933ee11-a974-467b-9dce-f898bcf3863b-kube-api-access-k2cfn\") pod \"image-registry-57b9f5fbdc-gsjvv\" (UID: \"d933ee11-a974-467b-9dce-f898bcf3863b\") " pod="openshift-image-registry/image-registry-57b9f5fbdc-gsjvv" Jul 05 13:15:51.547562 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:51.547492 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/79875c5a-6f9e-4d4f-9de9-5d974249f92b-metrics-tls\") pod \"dns-default-w8bmv\" (UID: \"79875c5a-6f9e-4d4f-9de9-5d974249f92b\") " pod="openshift-dns/dns-default-w8bmv" Jul 05 13:15:51.547562 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:51.547506 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z2v7b\" (UniqueName: \"kubernetes.io/projected/79875c5a-6f9e-4d4f-9de9-5d974249f92b-kube-api-access-z2v7b\") pod \"dns-default-w8bmv\" (UID: \"79875c5a-6f9e-4d4f-9de9-5d974249f92b\") " pod="openshift-dns/dns-default-w8bmv" Jul 05 13:15:51.547562 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:51.547521 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/30164fef-3f92-4b9d-8c8f-ea7e6a3f1047-nginx-conf\") pod \"networking-console-plugin-6bffdf9778-ljfps\" (UID: \"30164fef-3f92-4b9d-8c8f-ea7e6a3f1047\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-ljfps" Jul 05 13:15:51.648893 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:51.648846 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/d933ee11-a974-467b-9dce-f898bcf3863b-installation-pull-secrets\") pod \"image-registry-57b9f5fbdc-gsjvv\" (UID: \"d933ee11-a974-467b-9dce-f898bcf3863b\") " pod="openshift-image-registry/image-registry-57b9f5fbdc-gsjvv" Jul 05 13:15:51.648893 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:51.648880 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d933ee11-a974-467b-9dce-f898bcf3863b-bound-sa-token\") pod \"image-registry-57b9f5fbdc-gsjvv\" (UID: \"d933ee11-a974-467b-9dce-f898bcf3863b\") " pod="openshift-image-registry/image-registry-57b9f5fbdc-gsjvv" Jul 05 13:15:51.649122 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:51.648904 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/853f791d-d350-4626-989e-270b15b79453-cert\") pod \"ingress-canary-qn45h\" (UID: \"853f791d-d350-4626-989e-270b15b79453\") " pod="openshift-ingress-canary/ingress-canary-qn45h" Jul 05 13:15:51.649122 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:51.648926 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/79875c5a-6f9e-4d4f-9de9-5d974249f92b-config-volume\") pod \"dns-default-w8bmv\" (UID: \"79875c5a-6f9e-4d4f-9de9-5d974249f92b\") " pod="openshift-dns/dns-default-w8bmv" Jul 05 13:15:51.649219 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:51.649198 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9lt68\" (UniqueName: \"kubernetes.io/projected/853f791d-d350-4626-989e-270b15b79453-kube-api-access-9lt68\") pod \"ingress-canary-qn45h\" (UID: \"853f791d-d350-4626-989e-270b15b79453\") " pod="openshift-ingress-canary/ingress-canary-qn45h" Jul 05 13:15:51.649270 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:51.649247 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/d933ee11-a974-467b-9dce-f898bcf3863b-image-registry-private-configuration\") pod \"image-registry-57b9f5fbdc-gsjvv\" (UID: \"d933ee11-a974-467b-9dce-f898bcf3863b\") " pod="openshift-image-registry/image-registry-57b9f5fbdc-gsjvv" Jul 05 13:15:51.649319 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:51.649279 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/d933ee11-a974-467b-9dce-f898bcf3863b-trusted-ca\") pod \"image-registry-57b9f5fbdc-gsjvv\" (UID: \"d933ee11-a974-467b-9dce-f898bcf3863b\") " pod="openshift-image-registry/image-registry-57b9f5fbdc-gsjvv" Jul 05 13:15:51.649319 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:51.649313 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/d933ee11-a974-467b-9dce-f898bcf3863b-ca-trust-extracted\") pod \"image-registry-57b9f5fbdc-gsjvv\" (UID: \"d933ee11-a974-467b-9dce-f898bcf3863b\") " pod="openshift-image-registry/image-registry-57b9f5fbdc-gsjvv" Jul 05 13:15:51.649383 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:51.649359 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k2cfn\" (UniqueName: \"kubernetes.io/projected/d933ee11-a974-467b-9dce-f898bcf3863b-kube-api-access-k2cfn\") pod \"image-registry-57b9f5fbdc-gsjvv\" (UID: \"d933ee11-a974-467b-9dce-f898bcf3863b\") " pod="openshift-image-registry/image-registry-57b9f5fbdc-gsjvv" Jul 05 13:15:51.649383 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:51.649380 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/79875c5a-6f9e-4d4f-9de9-5d974249f92b-metrics-tls\") pod \"dns-default-w8bmv\" (UID: \"79875c5a-6f9e-4d4f-9de9-5d974249f92b\") " pod="openshift-dns/dns-default-w8bmv" Jul 05 13:15:51.649454 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:51.649395 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z2v7b\" (UniqueName: \"kubernetes.io/projected/79875c5a-6f9e-4d4f-9de9-5d974249f92b-kube-api-access-z2v7b\") pod \"dns-default-w8bmv\" (UID: \"79875c5a-6f9e-4d4f-9de9-5d974249f92b\") " pod="openshift-dns/dns-default-w8bmv" Jul 05 13:15:51.649454 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:51.649409 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/30164fef-3f92-4b9d-8c8f-ea7e6a3f1047-nginx-conf\") pod \"networking-console-plugin-6bffdf9778-ljfps\" (UID: \"30164fef-3f92-4b9d-8c8f-ea7e6a3f1047\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-ljfps" Jul 05 13:15:51.649454 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:51.649433 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/30164fef-3f92-4b9d-8c8f-ea7e6a3f1047-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-ljfps\" (UID: \"30164fef-3f92-4b9d-8c8f-ea7e6a3f1047\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-ljfps" Jul 05 13:15:51.649545 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:51.649459 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/d933ee11-a974-467b-9dce-f898bcf3863b-registry-certificates\") pod \"image-registry-57b9f5fbdc-gsjvv\" (UID: \"d933ee11-a974-467b-9dce-f898bcf3863b\") " pod="openshift-image-registry/image-registry-57b9f5fbdc-gsjvv" Jul 05 13:15:51.649545 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:51.649487 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d933ee11-a974-467b-9dce-f898bcf3863b-registry-tls\") pod \"image-registry-57b9f5fbdc-gsjvv\" (UID: \"d933ee11-a974-467b-9dce-f898bcf3863b\") " pod="openshift-image-registry/image-registry-57b9f5fbdc-gsjvv" Jul 05 13:15:51.649545 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:51.649508 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/79875c5a-6f9e-4d4f-9de9-5d974249f92b-tmp-dir\") pod \"dns-default-w8bmv\" (UID: \"79875c5a-6f9e-4d4f-9de9-5d974249f92b\") " pod="openshift-dns/dns-default-w8bmv" Jul 05 13:15:51.649845 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:51.649808 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/79875c5a-6f9e-4d4f-9de9-5d974249f92b-tmp-dir\") pod \"dns-default-w8bmv\" (UID: \"79875c5a-6f9e-4d4f-9de9-5d974249f92b\") " pod="openshift-dns/dns-default-w8bmv" Jul 05 13:15:51.649975 ip-10-0-130-191 kubenswrapper[2578]: E0705 13:15:51.649900 2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 05 13:15:51.649975 ip-10-0-130-191 kubenswrapper[2578]: E0705 13:15:51.649961 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/79875c5a-6f9e-4d4f-9de9-5d974249f92b-metrics-tls podName:79875c5a-6f9e-4d4f-9de9-5d974249f92b nodeName:}" failed. No retries permitted until 2026-07-05 13:15:52.149945465 +0000 UTC m=+36.608508564 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/79875c5a-6f9e-4d4f-9de9-5d974249f92b-metrics-tls") pod "dns-default-w8bmv" (UID: "79875c5a-6f9e-4d4f-9de9-5d974249f92b") : secret "dns-default-metrics-tls" not found Jul 05 13:15:51.650088 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:51.650074 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/d933ee11-a974-467b-9dce-f898bcf3863b-ca-trust-extracted\") pod \"image-registry-57b9f5fbdc-gsjvv\" (UID: \"d933ee11-a974-467b-9dce-f898bcf3863b\") " pod="openshift-image-registry/image-registry-57b9f5fbdc-gsjvv" Jul 05 13:15:51.650183 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:51.649455 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/79875c5a-6f9e-4d4f-9de9-5d974249f92b-config-volume\") pod \"dns-default-w8bmv\" (UID: \"79875c5a-6f9e-4d4f-9de9-5d974249f92b\") " pod="openshift-dns/dns-default-w8bmv" Jul 05 13:15:51.650608 ip-10-0-130-191 kubenswrapper[2578]: E0705 13:15:51.650259 2578 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 05 13:15:51.650608 ip-10-0-130-191 kubenswrapper[2578]: E0705 13:15:51.650275 2578 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-57b9f5fbdc-gsjvv: secret "image-registry-tls" not found Jul 05 13:15:51.650608 ip-10-0-130-191 kubenswrapper[2578]: E0705 13:15:51.650311 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d933ee11-a974-467b-9dce-f898bcf3863b-registry-tls podName:d933ee11-a974-467b-9dce-f898bcf3863b nodeName:}" failed. No retries permitted until 2026-07-05 13:15:52.150299322 +0000 UTC m=+36.608862405 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/d933ee11-a974-467b-9dce-f898bcf3863b-registry-tls") pod "image-registry-57b9f5fbdc-gsjvv" (UID: "d933ee11-a974-467b-9dce-f898bcf3863b") : secret "image-registry-tls" not found Jul 05 13:15:51.650608 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:51.650344 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/d933ee11-a974-467b-9dce-f898bcf3863b-trusted-ca\") pod \"image-registry-57b9f5fbdc-gsjvv\" (UID: \"d933ee11-a974-467b-9dce-f898bcf3863b\") " pod="openshift-image-registry/image-registry-57b9f5fbdc-gsjvv" Jul 05 13:15:51.650608 ip-10-0-130-191 kubenswrapper[2578]: E0705 13:15:51.650421 2578 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 05 13:15:51.650608 ip-10-0-130-191 kubenswrapper[2578]: E0705 13:15:51.650471 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/30164fef-3f92-4b9d-8c8f-ea7e6a3f1047-networking-console-plugin-cert podName:30164fef-3f92-4b9d-8c8f-ea7e6a3f1047 nodeName:}" failed. No retries permitted until 2026-07-05 13:15:52.150457902 +0000 UTC m=+36.609020985 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/30164fef-3f92-4b9d-8c8f-ea7e6a3f1047-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-ljfps" (UID: "30164fef-3f92-4b9d-8c8f-ea7e6a3f1047") : secret "networking-console-plugin-cert" not found Jul 05 13:15:51.650608 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:51.650471 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/d933ee11-a974-467b-9dce-f898bcf3863b-registry-certificates\") pod \"image-registry-57b9f5fbdc-gsjvv\" (UID: \"d933ee11-a974-467b-9dce-f898bcf3863b\") " pod="openshift-image-registry/image-registry-57b9f5fbdc-gsjvv" Jul 05 13:15:51.650995 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:51.650885 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/30164fef-3f92-4b9d-8c8f-ea7e6a3f1047-nginx-conf\") pod \"networking-console-plugin-6bffdf9778-ljfps\" (UID: \"30164fef-3f92-4b9d-8c8f-ea7e6a3f1047\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-ljfps" Jul 05 13:15:51.653713 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:51.653695 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/d933ee11-a974-467b-9dce-f898bcf3863b-installation-pull-secrets\") pod \"image-registry-57b9f5fbdc-gsjvv\" (UID: \"d933ee11-a974-467b-9dce-f898bcf3863b\") " pod="openshift-image-registry/image-registry-57b9f5fbdc-gsjvv" Jul 05 13:15:51.653788 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:51.653721 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/853f791d-d350-4626-989e-270b15b79453-cert\") pod \"ingress-canary-qn45h\" (UID: \"853f791d-d350-4626-989e-270b15b79453\") " pod="openshift-ingress-canary/ingress-canary-qn45h" Jul 05 13:15:51.653788 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:51.653765 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/d933ee11-a974-467b-9dce-f898bcf3863b-image-registry-private-configuration\") pod \"image-registry-57b9f5fbdc-gsjvv\" (UID: \"d933ee11-a974-467b-9dce-f898bcf3863b\") " pod="openshift-image-registry/image-registry-57b9f5fbdc-gsjvv" Jul 05 13:15:51.665854 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:51.665825 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z2v7b\" (UniqueName: \"kubernetes.io/projected/79875c5a-6f9e-4d4f-9de9-5d974249f92b-kube-api-access-z2v7b\") pod \"dns-default-w8bmv\" (UID: \"79875c5a-6f9e-4d4f-9de9-5d974249f92b\") " pod="openshift-dns/dns-default-w8bmv" Jul 05 13:15:51.666035 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:51.666018 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9lt68\" (UniqueName: \"kubernetes.io/projected/853f791d-d350-4626-989e-270b15b79453-kube-api-access-9lt68\") pod \"ingress-canary-qn45h\" (UID: \"853f791d-d350-4626-989e-270b15b79453\") " pod="openshift-ingress-canary/ingress-canary-qn45h" Jul 05 13:15:51.666216 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:51.666199 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d933ee11-a974-467b-9dce-f898bcf3863b-bound-sa-token\") pod \"image-registry-57b9f5fbdc-gsjvv\" (UID: \"d933ee11-a974-467b-9dce-f898bcf3863b\") " pod="openshift-image-registry/image-registry-57b9f5fbdc-gsjvv" Jul 05 13:15:51.666689 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:51.666672 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k2cfn\" (UniqueName: \"kubernetes.io/projected/d933ee11-a974-467b-9dce-f898bcf3863b-kube-api-access-k2cfn\") pod \"image-registry-57b9f5fbdc-gsjvv\" (UID: \"d933ee11-a974-467b-9dce-f898bcf3863b\") " pod="openshift-image-registry/image-registry-57b9f5fbdc-gsjvv" Jul 05 13:15:51.724623 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:51.724589 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-qn45h" Jul 05 13:15:51.864393 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:51.864238 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-qn45h"] Jul 05 13:15:51.867469 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:15:51.867441 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod853f791d_d350_4626_989e_270b15b79453.slice/crio-fb9a37424d5002aa72a671edd04091bf219e5517d3558e133efe1c9d7153d336 WatchSource:0}: Error finding container fb9a37424d5002aa72a671edd04091bf219e5517d3558e133efe1c9d7153d336: Status 404 returned error can't find the container with id fb9a37424d5002aa72a671edd04091bf219e5517d3558e133efe1c9d7153d336 Jul 05 13:15:52.003639 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:52.003611 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jldjr" Jul 05 13:15:52.003814 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:52.003607 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-72444" Jul 05 13:15:52.003814 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:52.003795 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dxwj6" Jul 05 13:15:52.007198 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:52.006997 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 05 13:15:52.007198 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:52.006997 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 05 13:15:52.007198 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:52.007035 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-jgr78\"" Jul 05 13:15:52.007198 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:52.007058 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-852z8\"" Jul 05 13:15:52.007493 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:52.007313 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 05 13:15:52.007493 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:52.007368 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 05 13:15:52.154039 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:52.154002 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/79875c5a-6f9e-4d4f-9de9-5d974249f92b-metrics-tls\") pod \"dns-default-w8bmv\" (UID: \"79875c5a-6f9e-4d4f-9de9-5d974249f92b\") " pod="openshift-dns/dns-default-w8bmv" Jul 05 13:15:52.154213 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:52.154052 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/30164fef-3f92-4b9d-8c8f-ea7e6a3f1047-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-ljfps\" (UID: \"30164fef-3f92-4b9d-8c8f-ea7e6a3f1047\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-ljfps" Jul 05 13:15:52.154213 ip-10-0-130-191 kubenswrapper[2578]: E0705 13:15:52.154129 2578 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 05 13:15:52.154213 ip-10-0-130-191 kubenswrapper[2578]: E0705 13:15:52.154137 2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 05 13:15:52.154213 ip-10-0-130-191 kubenswrapper[2578]: E0705 13:15:52.154176 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/30164fef-3f92-4b9d-8c8f-ea7e6a3f1047-networking-console-plugin-cert podName:30164fef-3f92-4b9d-8c8f-ea7e6a3f1047 nodeName:}" failed. No retries permitted until 2026-07-05 13:15:53.154162196 +0000 UTC m=+37.612725278 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/30164fef-3f92-4b9d-8c8f-ea7e6a3f1047-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-ljfps" (UID: "30164fef-3f92-4b9d-8c8f-ea7e6a3f1047") : secret "networking-console-plugin-cert" not found Jul 05 13:15:52.154213 ip-10-0-130-191 kubenswrapper[2578]: E0705 13:15:52.154195 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/79875c5a-6f9e-4d4f-9de9-5d974249f92b-metrics-tls podName:79875c5a-6f9e-4d4f-9de9-5d974249f92b nodeName:}" failed. No retries permitted until 2026-07-05 13:15:53.15418104 +0000 UTC m=+37.612744126 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/79875c5a-6f9e-4d4f-9de9-5d974249f92b-metrics-tls") pod "dns-default-w8bmv" (UID: "79875c5a-6f9e-4d4f-9de9-5d974249f92b") : secret "dns-default-metrics-tls" not found Jul 05 13:15:52.154506 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:52.154232 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d933ee11-a974-467b-9dce-f898bcf3863b-registry-tls\") pod \"image-registry-57b9f5fbdc-gsjvv\" (UID: \"d933ee11-a974-467b-9dce-f898bcf3863b\") " pod="openshift-image-registry/image-registry-57b9f5fbdc-gsjvv" Jul 05 13:15:52.154506 ip-10-0-130-191 kubenswrapper[2578]: E0705 13:15:52.154330 2578 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 05 13:15:52.154506 ip-10-0-130-191 kubenswrapper[2578]: E0705 13:15:52.154338 2578 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-57b9f5fbdc-gsjvv: secret "image-registry-tls" not found Jul 05 13:15:52.154506 ip-10-0-130-191 kubenswrapper[2578]: E0705 13:15:52.154360 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d933ee11-a974-467b-9dce-f898bcf3863b-registry-tls podName:d933ee11-a974-467b-9dce-f898bcf3863b nodeName:}" failed. No retries permitted until 2026-07-05 13:15:53.154353186 +0000 UTC m=+37.612916268 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/d933ee11-a974-467b-9dce-f898bcf3863b-registry-tls") pod "image-registry-57b9f5fbdc-gsjvv" (UID: "d933ee11-a974-467b-9dce-f898bcf3863b") : secret "image-registry-tls" not found Jul 05 13:15:52.244329 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:52.244276 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-qn45h" event={"ID":"853f791d-d350-4626-989e-270b15b79453","Type":"ContainerStarted","Data":"fb9a37424d5002aa72a671edd04091bf219e5517d3558e133efe1c9d7153d336"} Jul 05 13:15:53.163926 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:53.163888 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d933ee11-a974-467b-9dce-f898bcf3863b-registry-tls\") pod \"image-registry-57b9f5fbdc-gsjvv\" (UID: \"d933ee11-a974-467b-9dce-f898bcf3863b\") " pod="openshift-image-registry/image-registry-57b9f5fbdc-gsjvv" Jul 05 13:15:53.164522 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:53.163998 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/79875c5a-6f9e-4d4f-9de9-5d974249f92b-metrics-tls\") pod \"dns-default-w8bmv\" (UID: \"79875c5a-6f9e-4d4f-9de9-5d974249f92b\") " pod="openshift-dns/dns-default-w8bmv" Jul 05 13:15:53.164522 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:53.164035 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/30164fef-3f92-4b9d-8c8f-ea7e6a3f1047-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-ljfps\" (UID: \"30164fef-3f92-4b9d-8c8f-ea7e6a3f1047\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-ljfps" Jul 05 13:15:53.164522 ip-10-0-130-191 kubenswrapper[2578]: E0705 13:15:53.164063 2578 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 05 13:15:53.164522 ip-10-0-130-191 kubenswrapper[2578]: E0705 13:15:53.164087 2578 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-57b9f5fbdc-gsjvv: secret "image-registry-tls" not found Jul 05 13:15:53.164522 ip-10-0-130-191 kubenswrapper[2578]: E0705 13:15:53.164105 2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 05 13:15:53.164522 ip-10-0-130-191 kubenswrapper[2578]: E0705 13:15:53.164134 2578 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 05 13:15:53.164522 ip-10-0-130-191 kubenswrapper[2578]: E0705 13:15:53.164158 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d933ee11-a974-467b-9dce-f898bcf3863b-registry-tls podName:d933ee11-a974-467b-9dce-f898bcf3863b nodeName:}" failed. No retries permitted until 2026-07-05 13:15:55.164137115 +0000 UTC m=+39.622700217 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/d933ee11-a974-467b-9dce-f898bcf3863b-registry-tls") pod "image-registry-57b9f5fbdc-gsjvv" (UID: "d933ee11-a974-467b-9dce-f898bcf3863b") : secret "image-registry-tls" not found Jul 05 13:15:53.164522 ip-10-0-130-191 kubenswrapper[2578]: E0705 13:15:53.164181 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/30164fef-3f92-4b9d-8c8f-ea7e6a3f1047-networking-console-plugin-cert podName:30164fef-3f92-4b9d-8c8f-ea7e6a3f1047 nodeName:}" failed. No retries permitted until 2026-07-05 13:15:55.16416557 +0000 UTC m=+39.622728656 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/30164fef-3f92-4b9d-8c8f-ea7e6a3f1047-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-ljfps" (UID: "30164fef-3f92-4b9d-8c8f-ea7e6a3f1047") : secret "networking-console-plugin-cert" not found Jul 05 13:15:53.164522 ip-10-0-130-191 kubenswrapper[2578]: E0705 13:15:53.164205 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/79875c5a-6f9e-4d4f-9de9-5d974249f92b-metrics-tls podName:79875c5a-6f9e-4d4f-9de9-5d974249f92b nodeName:}" failed. No retries permitted until 2026-07-05 13:15:55.164195121 +0000 UTC m=+39.622758205 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/79875c5a-6f9e-4d4f-9de9-5d974249f92b-metrics-tls") pod "dns-default-w8bmv" (UID: "79875c5a-6f9e-4d4f-9de9-5d974249f92b") : secret "dns-default-metrics-tls" not found Jul 05 13:15:55.182243 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:55.182205 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/79875c5a-6f9e-4d4f-9de9-5d974249f92b-metrics-tls\") pod \"dns-default-w8bmv\" (UID: \"79875c5a-6f9e-4d4f-9de9-5d974249f92b\") " pod="openshift-dns/dns-default-w8bmv" Jul 05 13:15:55.182243 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:55.182253 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/30164fef-3f92-4b9d-8c8f-ea7e6a3f1047-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-ljfps\" (UID: \"30164fef-3f92-4b9d-8c8f-ea7e6a3f1047\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-ljfps" Jul 05 13:15:55.182723 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:55.182287 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d933ee11-a974-467b-9dce-f898bcf3863b-registry-tls\") pod \"image-registry-57b9f5fbdc-gsjvv\" (UID: \"d933ee11-a974-467b-9dce-f898bcf3863b\") " pod="openshift-image-registry/image-registry-57b9f5fbdc-gsjvv" Jul 05 13:15:55.182723 ip-10-0-130-191 kubenswrapper[2578]: E0705 13:15:55.182358 2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 05 13:15:55.182723 ip-10-0-130-191 kubenswrapper[2578]: E0705 13:15:55.182415 2578 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 05 13:15:55.182723 ip-10-0-130-191 kubenswrapper[2578]: E0705 13:15:55.182417 2578 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 05 13:15:55.182723 ip-10-0-130-191 kubenswrapper[2578]: E0705 13:15:55.182430 2578 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-57b9f5fbdc-gsjvv: secret "image-registry-tls" not found Jul 05 13:15:55.182723 ip-10-0-130-191 kubenswrapper[2578]: E0705 13:15:55.182437 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/79875c5a-6f9e-4d4f-9de9-5d974249f92b-metrics-tls podName:79875c5a-6f9e-4d4f-9de9-5d974249f92b nodeName:}" failed. No retries permitted until 2026-07-05 13:15:59.182417133 +0000 UTC m=+43.640980219 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/79875c5a-6f9e-4d4f-9de9-5d974249f92b-metrics-tls") pod "dns-default-w8bmv" (UID: "79875c5a-6f9e-4d4f-9de9-5d974249f92b") : secret "dns-default-metrics-tls" not found Jul 05 13:15:55.182723 ip-10-0-130-191 kubenswrapper[2578]: E0705 13:15:55.182479 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d933ee11-a974-467b-9dce-f898bcf3863b-registry-tls podName:d933ee11-a974-467b-9dce-f898bcf3863b nodeName:}" failed. No retries permitted until 2026-07-05 13:15:59.182466643 +0000 UTC m=+43.641029726 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/d933ee11-a974-467b-9dce-f898bcf3863b-registry-tls") pod "image-registry-57b9f5fbdc-gsjvv" (UID: "d933ee11-a974-467b-9dce-f898bcf3863b") : secret "image-registry-tls" not found Jul 05 13:15:55.182723 ip-10-0-130-191 kubenswrapper[2578]: E0705 13:15:55.182492 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/30164fef-3f92-4b9d-8c8f-ea7e6a3f1047-networking-console-plugin-cert podName:30164fef-3f92-4b9d-8c8f-ea7e6a3f1047 nodeName:}" failed. No retries permitted until 2026-07-05 13:15:59.182486246 +0000 UTC m=+43.641049328 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/30164fef-3f92-4b9d-8c8f-ea7e6a3f1047-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-ljfps" (UID: "30164fef-3f92-4b9d-8c8f-ea7e6a3f1047") : secret "networking-console-plugin-cert" not found Jul 05 13:15:56.260206 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:56.260163 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7n8f2" event={"ID":"545d8134-4ad2-4c62-b302-64b8e10bc493","Type":"ContainerStarted","Data":"884ab023e7d871e70d286e96fa891d75aba69c69c3706cdd36ee4f8d45bfa3f5"} Jul 05 13:15:56.261667 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:56.261644 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-qn45h" event={"ID":"853f791d-d350-4626-989e-270b15b79453","Type":"ContainerStarted","Data":"321b464d42f2b50edaf85ce5d863b70f43f0661156ae7e8a24b0cb9efdc4f44e"} Jul 05 13:15:56.301056 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:56.301003 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-qn45h" podStartSLOduration=1.166004463 podStartE2EDuration="5.300987957s" totalImagesPullingTime="4.134983494s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 13:15:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 13:15:56.300251157 +0000 UTC m=+40.758814260" watchObservedRunningTime="2026-07-05 13:15:56.300987957 +0000 UTC m=+40.759551061" Jul 05 13:15:57.266907 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:57.266871 2578 generic.go:356] "Generic (PLEG): container finished" podID="545d8134-4ad2-4c62-b302-64b8e10bc493" containerID="884ab023e7d871e70d286e96fa891d75aba69c69c3706cdd36ee4f8d45bfa3f5" exitCode=0 Jul 05 13:15:57.267367 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:57.266949 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7n8f2" event={"ID":"545d8134-4ad2-4c62-b302-64b8e10bc493","Type":"ContainerDied","Data":"884ab023e7d871e70d286e96fa891d75aba69c69c3706cdd36ee4f8d45bfa3f5"} Jul 05 13:15:58.271609 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:58.271565 2578 generic.go:356] "Generic (PLEG): container finished" podID="545d8134-4ad2-4c62-b302-64b8e10bc493" containerID="75a8c53d7195e074c1b2c84542ab748d3e29bf71d3269dbe67f9b11754e2d255" exitCode=0 Jul 05 13:15:58.271942 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:58.271630 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7n8f2" event={"ID":"545d8134-4ad2-4c62-b302-64b8e10bc493","Type":"ContainerDied","Data":"75a8c53d7195e074c1b2c84542ab748d3e29bf71d3269dbe67f9b11754e2d255"} Jul 05 13:15:59.214773 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:59.214742 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/79875c5a-6f9e-4d4f-9de9-5d974249f92b-metrics-tls\") pod \"dns-default-w8bmv\" (UID: \"79875c5a-6f9e-4d4f-9de9-5d974249f92b\") " pod="openshift-dns/dns-default-w8bmv" Jul 05 13:15:59.214941 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:59.214784 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/30164fef-3f92-4b9d-8c8f-ea7e6a3f1047-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-ljfps\" (UID: \"30164fef-3f92-4b9d-8c8f-ea7e6a3f1047\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-ljfps" Jul 05 13:15:59.214941 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:59.214812 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d933ee11-a974-467b-9dce-f898bcf3863b-registry-tls\") pod \"image-registry-57b9f5fbdc-gsjvv\" (UID: \"d933ee11-a974-467b-9dce-f898bcf3863b\") " pod="openshift-image-registry/image-registry-57b9f5fbdc-gsjvv" Jul 05 13:15:59.214941 ip-10-0-130-191 kubenswrapper[2578]: E0705 13:15:59.214895 2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 05 13:15:59.214941 ip-10-0-130-191 kubenswrapper[2578]: E0705 13:15:59.214908 2578 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 05 13:15:59.215061 ip-10-0-130-191 kubenswrapper[2578]: E0705 13:15:59.214913 2578 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 05 13:15:59.215061 ip-10-0-130-191 kubenswrapper[2578]: E0705 13:15:59.214949 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/79875c5a-6f9e-4d4f-9de9-5d974249f92b-metrics-tls podName:79875c5a-6f9e-4d4f-9de9-5d974249f92b nodeName:}" failed. No retries permitted until 2026-07-05 13:16:07.21493444 +0000 UTC m=+51.673497525 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/79875c5a-6f9e-4d4f-9de9-5d974249f92b-metrics-tls") pod "dns-default-w8bmv" (UID: "79875c5a-6f9e-4d4f-9de9-5d974249f92b") : secret "dns-default-metrics-tls" not found Jul 05 13:15:59.215061 ip-10-0-130-191 kubenswrapper[2578]: E0705 13:15:59.214956 2578 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-57b9f5fbdc-gsjvv: secret "image-registry-tls" not found Jul 05 13:15:59.215061 ip-10-0-130-191 kubenswrapper[2578]: E0705 13:15:59.214963 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/30164fef-3f92-4b9d-8c8f-ea7e6a3f1047-networking-console-plugin-cert podName:30164fef-3f92-4b9d-8c8f-ea7e6a3f1047 nodeName:}" failed. No retries permitted until 2026-07-05 13:16:07.214957805 +0000 UTC m=+51.673520891 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/30164fef-3f92-4b9d-8c8f-ea7e6a3f1047-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-ljfps" (UID: "30164fef-3f92-4b9d-8c8f-ea7e6a3f1047") : secret "networking-console-plugin-cert" not found Jul 05 13:15:59.215061 ip-10-0-130-191 kubenswrapper[2578]: E0705 13:15:59.214984 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d933ee11-a974-467b-9dce-f898bcf3863b-registry-tls podName:d933ee11-a974-467b-9dce-f898bcf3863b nodeName:}" failed. No retries permitted until 2026-07-05 13:16:07.214972706 +0000 UTC m=+51.673535817 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/d933ee11-a974-467b-9dce-f898bcf3863b-registry-tls") pod "image-registry-57b9f5fbdc-gsjvv" (UID: "d933ee11-a974-467b-9dce-f898bcf3863b") : secret "image-registry-tls" not found Jul 05 13:15:59.276455 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:59.276415 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7n8f2" event={"ID":"545d8134-4ad2-4c62-b302-64b8e10bc493","Type":"ContainerStarted","Data":"7670417a08e964066cf75c5f3f83b675dbe8902c6aff01679ed3d1306a970d41"} Jul 05 13:15:59.301548 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:15:59.301502 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-7n8f2" podStartSLOduration=9.336245541 podStartE2EDuration="43.301488228s" totalImagesPullingTime="33.965242687s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-05 13:15:16 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 13:15:59.299903105 +0000 UTC m=+43.758466208" watchObservedRunningTime="2026-07-05 13:15:59.301488228 +0000 UTC m=+43.760051338" Jul 05 13:16:07.275315 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:16:07.275279 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/79875c5a-6f9e-4d4f-9de9-5d974249f92b-metrics-tls\") pod \"dns-default-w8bmv\" (UID: \"79875c5a-6f9e-4d4f-9de9-5d974249f92b\") " pod="openshift-dns/dns-default-w8bmv" Jul 05 13:16:07.275771 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:16:07.275323 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/30164fef-3f92-4b9d-8c8f-ea7e6a3f1047-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-ljfps\" (UID: \"30164fef-3f92-4b9d-8c8f-ea7e6a3f1047\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-ljfps" Jul 05 13:16:07.275771 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:16:07.275362 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d933ee11-a974-467b-9dce-f898bcf3863b-registry-tls\") pod \"image-registry-57b9f5fbdc-gsjvv\" (UID: \"d933ee11-a974-467b-9dce-f898bcf3863b\") " pod="openshift-image-registry/image-registry-57b9f5fbdc-gsjvv" Jul 05 13:16:07.275771 ip-10-0-130-191 kubenswrapper[2578]: E0705 13:16:07.275440 2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 05 13:16:07.275771 ip-10-0-130-191 kubenswrapper[2578]: E0705 13:16:07.275443 2578 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 05 13:16:07.275771 ip-10-0-130-191 kubenswrapper[2578]: E0705 13:16:07.275460 2578 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-57b9f5fbdc-gsjvv: secret "image-registry-tls" not found Jul 05 13:16:07.275771 ip-10-0-130-191 kubenswrapper[2578]: E0705 13:16:07.275500 2578 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 05 13:16:07.275771 ip-10-0-130-191 kubenswrapper[2578]: E0705 13:16:07.275505 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d933ee11-a974-467b-9dce-f898bcf3863b-registry-tls podName:d933ee11-a974-467b-9dce-f898bcf3863b nodeName:}" failed. No retries permitted until 2026-07-05 13:16:23.275488133 +0000 UTC m=+67.734051216 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/d933ee11-a974-467b-9dce-f898bcf3863b-registry-tls") pod "image-registry-57b9f5fbdc-gsjvv" (UID: "d933ee11-a974-467b-9dce-f898bcf3863b") : secret "image-registry-tls" not found Jul 05 13:16:07.275771 ip-10-0-130-191 kubenswrapper[2578]: E0705 13:16:07.275606 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/79875c5a-6f9e-4d4f-9de9-5d974249f92b-metrics-tls podName:79875c5a-6f9e-4d4f-9de9-5d974249f92b nodeName:}" failed. No retries permitted until 2026-07-05 13:16:23.275572448 +0000 UTC m=+67.734135533 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/79875c5a-6f9e-4d4f-9de9-5d974249f92b-metrics-tls") pod "dns-default-w8bmv" (UID: "79875c5a-6f9e-4d4f-9de9-5d974249f92b") : secret "dns-default-metrics-tls" not found Jul 05 13:16:07.275771 ip-10-0-130-191 kubenswrapper[2578]: E0705 13:16:07.275618 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/30164fef-3f92-4b9d-8c8f-ea7e6a3f1047-networking-console-plugin-cert podName:30164fef-3f92-4b9d-8c8f-ea7e6a3f1047 nodeName:}" failed. No retries permitted until 2026-07-05 13:16:23.27561187 +0000 UTC m=+67.734174952 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/30164fef-3f92-4b9d-8c8f-ea7e6a3f1047-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-ljfps" (UID: "30164fef-3f92-4b9d-8c8f-ea7e6a3f1047") : secret "networking-console-plugin-cert" not found Jul 05 13:16:18.244725 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:16:18.244695 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-656m6" Jul 05 13:16:21.770739 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:16:21.770699 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-czl8p\" (UniqueName: \"kubernetes.io/projected/50dc9dcd-a00c-4f85-b3dc-dbac06316c3e-kube-api-access-czl8p\") pod \"network-check-target-jldjr\" (UID: \"50dc9dcd-a00c-4f85-b3dc-dbac06316c3e\") " pod="openshift-network-diagnostics/network-check-target-jldjr" Jul 05 13:16:21.770739 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:16:21.770750 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4ff28158-61b2-4cb9-8b92-2ac0793c23bb-original-pull-secret\") pod \"global-pull-secret-syncer-72444\" (UID: \"4ff28158-61b2-4cb9-8b92-2ac0793c23bb\") " pod="kube-system/global-pull-secret-syncer-72444" Jul 05 13:16:21.771222 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:16:21.770927 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5ad92660-5823-4f14-adcc-763d06385f45-metrics-certs\") pod \"network-metrics-daemon-dxwj6\" (UID: \"5ad92660-5823-4f14-adcc-763d06385f45\") " pod="openshift-multus/network-metrics-daemon-dxwj6" Jul 05 13:16:21.773934 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:16:21.773915 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 05 13:16:21.773997 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:16:21.773937 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 05 13:16:21.773997 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:16:21.773915 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 05 13:16:21.781175 ip-10-0-130-191 kubenswrapper[2578]: E0705 13:16:21.781146 2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 05 13:16:21.781285 ip-10-0-130-191 kubenswrapper[2578]: E0705 13:16:21.781218 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5ad92660-5823-4f14-adcc-763d06385f45-metrics-certs podName:5ad92660-5823-4f14-adcc-763d06385f45 nodeName:}" failed. No retries permitted until 2026-07-05 13:17:25.781199396 +0000 UTC m=+130.239762492 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5ad92660-5823-4f14-adcc-763d06385f45-metrics-certs") pod "network-metrics-daemon-dxwj6" (UID: "5ad92660-5823-4f14-adcc-763d06385f45") : secret "metrics-daemon-secret" not found Jul 05 13:16:21.783770 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:16:21.783750 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 05 13:16:21.784105 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:16:21.784089 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4ff28158-61b2-4cb9-8b92-2ac0793c23bb-original-pull-secret\") pod \"global-pull-secret-syncer-72444\" (UID: \"4ff28158-61b2-4cb9-8b92-2ac0793c23bb\") " pod="kube-system/global-pull-secret-syncer-72444" Jul 05 13:16:21.794152 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:16:21.794120 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-czl8p\" (UniqueName: \"kubernetes.io/projected/50dc9dcd-a00c-4f85-b3dc-dbac06316c3e-kube-api-access-czl8p\") pod \"network-check-target-jldjr\" (UID: \"50dc9dcd-a00c-4f85-b3dc-dbac06316c3e\") " pod="openshift-network-diagnostics/network-check-target-jldjr" Jul 05 13:16:22.019793 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:16:22.019767 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-852z8\"" Jul 05 13:16:22.026934 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:16:22.026893 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jldjr" Jul 05 13:16:22.026934 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:16:22.026910 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-72444" Jul 05 13:16:22.160363 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:16:22.160330 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-jldjr"] Jul 05 13:16:22.163405 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:16:22.163380 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod50dc9dcd_a00c_4f85_b3dc_dbac06316c3e.slice/crio-620d85c020e0905c227feff2870bf3e0058e8892176200913552d1336e755560 WatchSource:0}: Error finding container 620d85c020e0905c227feff2870bf3e0058e8892176200913552d1336e755560: Status 404 returned error can't find the container with id 620d85c020e0905c227feff2870bf3e0058e8892176200913552d1336e755560 Jul 05 13:16:22.173683 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:16:22.173662 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-72444"] Jul 05 13:16:22.178274 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:16:22.178249 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4ff28158_61b2_4cb9_8b92_2ac0793c23bb.slice/crio-39127b1bb282d7889857ccdae768d107b64d5e320274ecec2bc8c869888864e4 WatchSource:0}: Error finding container 39127b1bb282d7889857ccdae768d107b64d5e320274ecec2bc8c869888864e4: Status 404 returned error can't find the container with id 39127b1bb282d7889857ccdae768d107b64d5e320274ecec2bc8c869888864e4 Jul 05 13:16:22.327874 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:16:22.327783 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-jldjr" event={"ID":"50dc9dcd-a00c-4f85-b3dc-dbac06316c3e","Type":"ContainerStarted","Data":"620d85c020e0905c227feff2870bf3e0058e8892176200913552d1336e755560"} Jul 05 13:16:22.328801 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:16:22.328766 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-72444" event={"ID":"4ff28158-61b2-4cb9-8b92-2ac0793c23bb","Type":"ContainerStarted","Data":"39127b1bb282d7889857ccdae768d107b64d5e320274ecec2bc8c869888864e4"} Jul 05 13:16:23.285391 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:16:23.285356 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/79875c5a-6f9e-4d4f-9de9-5d974249f92b-metrics-tls\") pod \"dns-default-w8bmv\" (UID: \"79875c5a-6f9e-4d4f-9de9-5d974249f92b\") " pod="openshift-dns/dns-default-w8bmv" Jul 05 13:16:23.285872 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:16:23.285406 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/30164fef-3f92-4b9d-8c8f-ea7e6a3f1047-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-ljfps\" (UID: \"30164fef-3f92-4b9d-8c8f-ea7e6a3f1047\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-ljfps" Jul 05 13:16:23.285872 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:16:23.285442 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d933ee11-a974-467b-9dce-f898bcf3863b-registry-tls\") pod \"image-registry-57b9f5fbdc-gsjvv\" (UID: \"d933ee11-a974-467b-9dce-f898bcf3863b\") " pod="openshift-image-registry/image-registry-57b9f5fbdc-gsjvv" Jul 05 13:16:23.285872 ip-10-0-130-191 kubenswrapper[2578]: E0705 13:16:23.285538 2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 05 13:16:23.285872 ip-10-0-130-191 kubenswrapper[2578]: E0705 13:16:23.285556 2578 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 05 13:16:23.285872 ip-10-0-130-191 kubenswrapper[2578]: E0705 13:16:23.285569 2578 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-57b9f5fbdc-gsjvv: secret "image-registry-tls" not found Jul 05 13:16:23.285872 ip-10-0-130-191 kubenswrapper[2578]: E0705 13:16:23.285603 2578 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 05 13:16:23.285872 ip-10-0-130-191 kubenswrapper[2578]: E0705 13:16:23.285636 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/79875c5a-6f9e-4d4f-9de9-5d974249f92b-metrics-tls podName:79875c5a-6f9e-4d4f-9de9-5d974249f92b nodeName:}" failed. No retries permitted until 2026-07-05 13:16:55.28561605 +0000 UTC m=+99.744179146 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/79875c5a-6f9e-4d4f-9de9-5d974249f92b-metrics-tls") pod "dns-default-w8bmv" (UID: "79875c5a-6f9e-4d4f-9de9-5d974249f92b") : secret "dns-default-metrics-tls" not found Jul 05 13:16:23.285872 ip-10-0-130-191 kubenswrapper[2578]: E0705 13:16:23.285658 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d933ee11-a974-467b-9dce-f898bcf3863b-registry-tls podName:d933ee11-a974-467b-9dce-f898bcf3863b nodeName:}" failed. No retries permitted until 2026-07-05 13:16:55.285647627 +0000 UTC m=+99.744210715 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/d933ee11-a974-467b-9dce-f898bcf3863b-registry-tls") pod "image-registry-57b9f5fbdc-gsjvv" (UID: "d933ee11-a974-467b-9dce-f898bcf3863b") : secret "image-registry-tls" not found Jul 05 13:16:23.285872 ip-10-0-130-191 kubenswrapper[2578]: E0705 13:16:23.285674 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/30164fef-3f92-4b9d-8c8f-ea7e6a3f1047-networking-console-plugin-cert podName:30164fef-3f92-4b9d-8c8f-ea7e6a3f1047 nodeName:}" failed. No retries permitted until 2026-07-05 13:16:55.285663528 +0000 UTC m=+99.744226621 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/30164fef-3f92-4b9d-8c8f-ea7e6a3f1047-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-ljfps" (UID: "30164fef-3f92-4b9d-8c8f-ea7e6a3f1047") : secret "networking-console-plugin-cert" not found Jul 05 13:16:27.343864 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:16:27.343826 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-jldjr" event={"ID":"50dc9dcd-a00c-4f85-b3dc-dbac06316c3e","Type":"ContainerStarted","Data":"1699fcf16770404899717657ba00a74eadc571c2a0cd9ff0a38534d250153942"} Jul 05 13:16:27.344361 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:16:27.343931 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-jldjr" Jul 05 13:16:27.345434 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:16:27.345408 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-72444" event={"ID":"4ff28158-61b2-4cb9-8b92-2ac0793c23bb","Type":"ContainerStarted","Data":"5a489a82502c9d77deeb357196153280069224be8ab07fb3de12a7a33f05c6b4"} Jul 05 13:16:27.358846 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:16:27.358806 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-jldjr" podStartSLOduration=66.314107549 podStartE2EDuration="1m11.358794758s" totalImagesPullingTime="5.044687209s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 13:15:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 13:16:27.358308505 +0000 UTC m=+71.816871608" watchObservedRunningTime="2026-07-05 13:16:27.358794758 +0000 UTC m=+71.817357853" Jul 05 13:16:55.327235 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:16:55.327194 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d933ee11-a974-467b-9dce-f898bcf3863b-registry-tls\") pod \"image-registry-57b9f5fbdc-gsjvv\" (UID: \"d933ee11-a974-467b-9dce-f898bcf3863b\") " pod="openshift-image-registry/image-registry-57b9f5fbdc-gsjvv" Jul 05 13:16:55.327668 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:16:55.327273 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/79875c5a-6f9e-4d4f-9de9-5d974249f92b-metrics-tls\") pod \"dns-default-w8bmv\" (UID: \"79875c5a-6f9e-4d4f-9de9-5d974249f92b\") " pod="openshift-dns/dns-default-w8bmv" Jul 05 13:16:55.327668 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:16:55.327300 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/30164fef-3f92-4b9d-8c8f-ea7e6a3f1047-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-ljfps\" (UID: \"30164fef-3f92-4b9d-8c8f-ea7e6a3f1047\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-ljfps" Jul 05 13:16:55.327668 ip-10-0-130-191 kubenswrapper[2578]: E0705 13:16:55.327309 2578 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 05 13:16:55.327668 ip-10-0-130-191 kubenswrapper[2578]: E0705 13:16:55.327331 2578 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-57b9f5fbdc-gsjvv: secret "image-registry-tls" not found Jul 05 13:16:55.327668 ip-10-0-130-191 kubenswrapper[2578]: E0705 13:16:55.327396 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d933ee11-a974-467b-9dce-f898bcf3863b-registry-tls podName:d933ee11-a974-467b-9dce-f898bcf3863b nodeName:}" failed. No retries permitted until 2026-07-05 13:17:59.327379967 +0000 UTC m=+163.785943050 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/d933ee11-a974-467b-9dce-f898bcf3863b-registry-tls") pod "image-registry-57b9f5fbdc-gsjvv" (UID: "d933ee11-a974-467b-9dce-f898bcf3863b") : secret "image-registry-tls" not found Jul 05 13:16:55.327668 ip-10-0-130-191 kubenswrapper[2578]: E0705 13:16:55.327396 2578 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 05 13:16:55.327668 ip-10-0-130-191 kubenswrapper[2578]: E0705 13:16:55.327414 2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 05 13:16:55.327668 ip-10-0-130-191 kubenswrapper[2578]: E0705 13:16:55.327448 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/30164fef-3f92-4b9d-8c8f-ea7e6a3f1047-networking-console-plugin-cert podName:30164fef-3f92-4b9d-8c8f-ea7e6a3f1047 nodeName:}" failed. No retries permitted until 2026-07-05 13:17:59.327436861 +0000 UTC m=+163.785999943 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/30164fef-3f92-4b9d-8c8f-ea7e6a3f1047-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-ljfps" (UID: "30164fef-3f92-4b9d-8c8f-ea7e6a3f1047") : secret "networking-console-plugin-cert" not found Jul 05 13:16:55.327668 ip-10-0-130-191 kubenswrapper[2578]: E0705 13:16:55.327468 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/79875c5a-6f9e-4d4f-9de9-5d974249f92b-metrics-tls podName:79875c5a-6f9e-4d4f-9de9-5d974249f92b nodeName:}" failed. No retries permitted until 2026-07-05 13:17:59.32745349 +0000 UTC m=+163.786016576 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/79875c5a-6f9e-4d4f-9de9-5d974249f92b-metrics-tls") pod "dns-default-w8bmv" (UID: "79875c5a-6f9e-4d4f-9de9-5d974249f92b") : secret "dns-default-metrics-tls" not found Jul 05 13:16:58.350002 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:16:58.349969 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-jldjr" Jul 05 13:16:58.367646 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:16:58.367575 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-72444" podStartSLOduration=97.333929626 podStartE2EDuration="1m42.367558645s" totalImagesPullingTime="5.033629019s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-05 13:15:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 13:16:27.3732387 +0000 UTC m=+71.831801806" watchObservedRunningTime="2026-07-05 13:16:58.367558645 +0000 UTC m=+102.826121750" Jul 05 13:17:25.852528 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:17:25.852472 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5ad92660-5823-4f14-adcc-763d06385f45-metrics-certs\") pod \"network-metrics-daemon-dxwj6\" (UID: \"5ad92660-5823-4f14-adcc-763d06385f45\") " pod="openshift-multus/network-metrics-daemon-dxwj6" Jul 05 13:17:25.853056 ip-10-0-130-191 kubenswrapper[2578]: E0705 13:17:25.852687 2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 05 13:17:25.853056 ip-10-0-130-191 kubenswrapper[2578]: E0705 13:17:25.852768 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5ad92660-5823-4f14-adcc-763d06385f45-metrics-certs podName:5ad92660-5823-4f14-adcc-763d06385f45 nodeName:}" failed. No retries permitted until 2026-07-05 13:19:27.85275053 +0000 UTC m=+252.311313613 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5ad92660-5823-4f14-adcc-763d06385f45-metrics-certs") pod "network-metrics-daemon-dxwj6" (UID: "5ad92660-5823-4f14-adcc-763d06385f45") : secret "metrics-daemon-secret" not found Jul 05 13:17:54.398404 ip-10-0-130-191 kubenswrapper[2578]: E0705 13:17:54.398355 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[registry-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-image-registry/image-registry-57b9f5fbdc-gsjvv" podUID="d933ee11-a974-467b-9dce-f898bcf3863b" Jul 05 13:17:54.405566 ip-10-0-130-191 kubenswrapper[2578]: E0705 13:17:54.405522 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[networking-console-plugin-cert], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-network-console/networking-console-plugin-6bffdf9778-ljfps" podUID="30164fef-3f92-4b9d-8c8f-ea7e6a3f1047" Jul 05 13:17:54.418753 ip-10-0-130-191 kubenswrapper[2578]: E0705 13:17:54.418707 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-w8bmv" podUID="79875c5a-6f9e-4d4f-9de9-5d974249f92b" Jul 05 13:17:54.558864 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:17:54.558836 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-w8bmv" Jul 05 13:17:54.558864 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:17:54.558863 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-57b9f5fbdc-gsjvv" Jul 05 13:17:54.559035 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:17:54.558981 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6bffdf9778-ljfps" Jul 05 13:17:55.030693 ip-10-0-130-191 kubenswrapper[2578]: E0705 13:17:55.030638 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-dxwj6" podUID="5ad92660-5823-4f14-adcc-763d06385f45" Jul 05 13:17:59.397567 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:17:59.397523 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d933ee11-a974-467b-9dce-f898bcf3863b-registry-tls\") pod \"image-registry-57b9f5fbdc-gsjvv\" (UID: \"d933ee11-a974-467b-9dce-f898bcf3863b\") " pod="openshift-image-registry/image-registry-57b9f5fbdc-gsjvv" Jul 05 13:17:59.398024 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:17:59.397616 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/79875c5a-6f9e-4d4f-9de9-5d974249f92b-metrics-tls\") pod \"dns-default-w8bmv\" (UID: \"79875c5a-6f9e-4d4f-9de9-5d974249f92b\") " pod="openshift-dns/dns-default-w8bmv" Jul 05 13:17:59.398024 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:17:59.397644 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/30164fef-3f92-4b9d-8c8f-ea7e6a3f1047-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-ljfps\" (UID: \"30164fef-3f92-4b9d-8c8f-ea7e6a3f1047\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-ljfps" Jul 05 13:17:59.398024 ip-10-0-130-191 kubenswrapper[2578]: E0705 13:17:59.397679 2578 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 05 13:17:59.398024 ip-10-0-130-191 kubenswrapper[2578]: E0705 13:17:59.397698 2578 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-57b9f5fbdc-gsjvv: secret "image-registry-tls" not found Jul 05 13:17:59.398024 ip-10-0-130-191 kubenswrapper[2578]: E0705 13:17:59.397754 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d933ee11-a974-467b-9dce-f898bcf3863b-registry-tls podName:d933ee11-a974-467b-9dce-f898bcf3863b nodeName:}" failed. No retries permitted until 2026-07-05 13:20:01.397739668 +0000 UTC m=+285.856302750 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/d933ee11-a974-467b-9dce-f898bcf3863b-registry-tls") pod "image-registry-57b9f5fbdc-gsjvv" (UID: "d933ee11-a974-467b-9dce-f898bcf3863b") : secret "image-registry-tls" not found Jul 05 13:17:59.398024 ip-10-0-130-191 kubenswrapper[2578]: E0705 13:17:59.397754 2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 05 13:17:59.398024 ip-10-0-130-191 kubenswrapper[2578]: E0705 13:17:59.397754 2578 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 05 13:17:59.398024 ip-10-0-130-191 kubenswrapper[2578]: E0705 13:17:59.397816 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/79875c5a-6f9e-4d4f-9de9-5d974249f92b-metrics-tls podName:79875c5a-6f9e-4d4f-9de9-5d974249f92b nodeName:}" failed. No retries permitted until 2026-07-05 13:20:01.397804007 +0000 UTC m=+285.856367089 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/79875c5a-6f9e-4d4f-9de9-5d974249f92b-metrics-tls") pod "dns-default-w8bmv" (UID: "79875c5a-6f9e-4d4f-9de9-5d974249f92b") : secret "dns-default-metrics-tls" not found Jul 05 13:17:59.398024 ip-10-0-130-191 kubenswrapper[2578]: E0705 13:17:59.397906 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/30164fef-3f92-4b9d-8c8f-ea7e6a3f1047-networking-console-plugin-cert podName:30164fef-3f92-4b9d-8c8f-ea7e6a3f1047 nodeName:}" failed. No retries permitted until 2026-07-05 13:20:01.397885573 +0000 UTC m=+285.856448662 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/30164fef-3f92-4b9d-8c8f-ea7e6a3f1047-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-ljfps" (UID: "30164fef-3f92-4b9d-8c8f-ea7e6a3f1047") : secret "networking-console-plugin-cert" not found Jul 05 13:18:06.004411 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:18:06.004366 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dxwj6" Jul 05 13:19:21.079016 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:19:21.078974 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-5c847dcfc4-7pd6v"] Jul 05 13:19:21.080525 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:19:21.080510 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-x8wmq"] Jul 05 13:19:21.080668 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:19:21.080650 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5c847dcfc4-7pd6v" Jul 05 13:19:21.082094 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:19:21.082078 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-x8wmq" Jul 05 13:19:21.099059 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:19:21.098765 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\"" Jul 05 13:19:21.102615 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:19:21.102594 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-68b757fb67-tg2xx"] Jul 05 13:19:21.104140 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:19:21.104125 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-wgnw7"] Jul 05 13:19:21.104267 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:19:21.104240 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-68b757fb67-tg2xx" Jul 05 13:19:21.105792 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:19:21.105763 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5c847dcfc4-7pd6v"] Jul 05 13:19:21.105885 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:19:21.105797 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-x8wmq"] Jul 05 13:19:21.105885 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:19:21.105873 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-wgnw7" Jul 05 13:19:21.107766 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:19:21.107743 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\"" Jul 05 13:19:21.107882 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:19:21.107844 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\"" Jul 05 13:19:21.108108 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:19:21.108085 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 05 13:19:21.108108 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:19:21.108085 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 05 13:19:21.108285 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:19:21.108193 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\"" Jul 05 13:19:21.109349 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:19:21.109329 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-5ks72\"" Jul 05 13:19:21.115363 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:19:21.115347 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-2g9jp\"" Jul 05 13:19:21.115669 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:19:21.115657 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\"" Jul 05 13:19:21.116355 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:19:21.116336 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-zf5b2\"" Jul 05 13:19:21.116489 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:19:21.116474 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\"" Jul 05 13:19:21.120477 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:19:21.120459 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\"" Jul 05 13:19:21.120754 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:19:21.120741 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\"" Jul 05 13:19:21.131308 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:19:21.131291 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\"" Jul 05 13:19:21.131484 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:19:21.131464 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\"" Jul 05 13:19:21.131547 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:19:21.131499 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\"" Jul 05 13:19:21.131718 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:19:21.131706 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\"" Jul 05 13:19:21.131899 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:19:21.131884 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-gkd9f\"" Jul 05 13:19:21.132537 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:19:21.132523 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-wgnw7"] Jul 05 13:19:21.133270 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:19:21.133250 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\"" Jul 05 13:19:21.148205 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:19:21.148184 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-68b757fb67-tg2xx"] Jul 05 13:19:21.152281 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:19:21.152262 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\"" Jul 05 13:19:21.174043 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:19:21.174018 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/beb3b580-1920-42d3-a152-ac42772476cf-snapshots\") pod \"insights-operator-5c847dcfc4-7pd6v\" (UID: \"beb3b580-1920-42d3-a152-ac42772476cf\") " pod="openshift-insights/insights-operator-5c847dcfc4-7pd6v" Jul 05 13:19:21.174125 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:19:21.174045 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/beb3b580-1920-42d3-a152-ac42772476cf-trusted-ca-bundle\") pod \"insights-operator-5c847dcfc4-7pd6v\" (UID: \"beb3b580-1920-42d3-a152-ac42772476cf\") " pod="openshift-insights/insights-operator-5c847dcfc4-7pd6v" Jul 05 13:19:21.174125 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:19:21.174100 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pxqqv\" (UniqueName: \"kubernetes.io/projected/ff7bc8c4-d331-46ec-a64e-f32d6ded6d9c-kube-api-access-pxqqv\") pod \"volume-data-source-validator-5d456fdbcd-x8wmq\" (UID: \"ff7bc8c4-d331-46ec-a64e-f32d6ded6d9c\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-x8wmq" Jul 05 13:19:21.174214 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:19:21.174135 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/1c214417-9bef-4007-ab01-e61b84d1b35e-serving-cert\") pod \"console-operator-68b757fb67-tg2xx\" (UID: \"1c214417-9bef-4007-ab01-e61b84d1b35e\") " pod="openshift-console-operator/console-operator-68b757fb67-tg2xx" Jul 05 13:19:21.174214 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:19:21.174165 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dv4t9\" (UniqueName: \"kubernetes.io/projected/1c214417-9bef-4007-ab01-e61b84d1b35e-kube-api-access-dv4t9\") pod \"console-operator-68b757fb67-tg2xx\" (UID: \"1c214417-9bef-4007-ab01-e61b84d1b35e\") " pod="openshift-console-operator/console-operator-68b757fb67-tg2xx" Jul 05 13:19:21.174295 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:19:21.174220 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/beb3b580-1920-42d3-a152-ac42772476cf-tmp\") pod \"insights-operator-5c847dcfc4-7pd6v\" (UID: \"beb3b580-1920-42d3-a152-ac42772476cf\") " pod="openshift-insights/insights-operator-5c847dcfc4-7pd6v" Jul 05 13:19:21.174295 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:19:21.174246 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/b7b1094c-0b91-4c45-aec8-2c0e2d8aed07-config\") pod \"kube-storage-version-migrator-operator-846bf668f7-wgnw7\" (UID: \"b7b1094c-0b91-4c45-aec8-2c0e2d8aed07\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-wgnw7" Jul 05 13:19:21.174295 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:19:21.174265 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/1c214417-9bef-4007-ab01-e61b84d1b35e-trusted-ca\") pod \"console-operator-68b757fb67-tg2xx\" (UID: \"1c214417-9bef-4007-ab01-e61b84d1b35e\") " pod="openshift-console-operator/console-operator-68b757fb67-tg2xx" Jul 05 13:19:21.174295 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:19:21.174280 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/beb3b580-1920-42d3-a152-ac42772476cf-service-ca-bundle\") pod \"insights-operator-5c847dcfc4-7pd6v\" (UID: \"beb3b580-1920-42d3-a152-ac42772476cf\") " pod="openshift-insights/insights-operator-5c847dcfc4-7pd6v" Jul 05 13:19:21.174523 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:19:21.174296 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/1c214417-9bef-4007-ab01-e61b84d1b35e-config\") pod \"console-operator-68b757fb67-tg2xx\" (UID: \"1c214417-9bef-4007-ab01-e61b84d1b35e\") " pod="openshift-console-operator/console-operator-68b757fb67-tg2xx" Jul 05 13:19:21.174523 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:19:21.174351 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/b7b1094c-0b91-4c45-aec8-2c0e2d8aed07-serving-cert\") pod \"kube-storage-version-migrator-operator-846bf668f7-wgnw7\" (UID: \"b7b1094c-0b91-4c45-aec8-2c0e2d8aed07\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-wgnw7" Jul 05 13:19:21.174523 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:19:21.174383 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/beb3b580-1920-42d3-a152-ac42772476cf-serving-cert\") pod \"insights-operator-5c847dcfc4-7pd6v\" (UID: \"beb3b580-1920-42d3-a152-ac42772476cf\") " pod="openshift-insights/insights-operator-5c847dcfc4-7pd6v" Jul 05 13:19:21.174523 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:19:21.174416 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g6mbk\" (UniqueName: \"kubernetes.io/projected/beb3b580-1920-42d3-a152-ac42772476cf-kube-api-access-g6mbk\") pod \"insights-operator-5c847dcfc4-7pd6v\" (UID: \"beb3b580-1920-42d3-a152-ac42772476cf\") " pod="openshift-insights/insights-operator-5c847dcfc4-7pd6v" Jul 05 13:19:21.174523 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:19:21.174451 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zl988\" (UniqueName: \"kubernetes.io/projected/b7b1094c-0b91-4c45-aec8-2c0e2d8aed07-kube-api-access-zl988\") pod \"kube-storage-version-migrator-operator-846bf668f7-wgnw7\" (UID: \"b7b1094c-0b91-4c45-aec8-2c0e2d8aed07\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-wgnw7" Jul 05 13:19:21.182028 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:19:21.182009 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-64786d9cbb-d8wzk"] Jul 05 13:19:21.183813 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:19:21.183800 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-d8wzk" Jul 05 13:19:21.190345 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:19:21.190328 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\"" Jul 05 13:19:21.191332 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:19:21.191319 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 05 13:19:21.191423 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:19:21.191371 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-xv75j\"" Jul 05 13:19:21.193128 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:19:21.193105 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\"" Jul 05 13:19:21.193828 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:19:21.193813 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 05 13:19:21.226099 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:19:21.226076 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-64786d9cbb-d8wzk"] Jul 05 13:19:21.275228 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:19:21.275196 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/b7b1094c-0b91-4c45-aec8-2c0e2d8aed07-serving-cert\") pod \"kube-storage-version-migrator-operator-846bf668f7-wgnw7\" (UID: \"b7b1094c-0b91-4c45-aec8-2c0e2d8aed07\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-wgnw7" Jul 05 13:19:21.275228 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:19:21.275226 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/beb3b580-1920-42d3-a152-ac42772476cf-serving-cert\") pod \"insights-operator-5c847dcfc4-7pd6v\" (UID: \"beb3b580-1920-42d3-a152-ac42772476cf\") " pod="openshift-insights/insights-operator-5c847dcfc4-7pd6v" Jul 05 13:19:21.275460 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:19:21.275244 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g6mbk\" (UniqueName: \"kubernetes.io/projected/beb3b580-1920-42d3-a152-ac42772476cf-kube-api-access-g6mbk\") pod \"insights-operator-5c847dcfc4-7pd6v\" (UID: \"beb3b580-1920-42d3-a152-ac42772476cf\") " pod="openshift-insights/insights-operator-5c847dcfc4-7pd6v" Jul 05 13:19:21.275460 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:19:21.275262 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zl988\" (UniqueName: \"kubernetes.io/projected/b7b1094c-0b91-4c45-aec8-2c0e2d8aed07-kube-api-access-zl988\") pod \"kube-storage-version-migrator-operator-846bf668f7-wgnw7\" (UID: \"b7b1094c-0b91-4c45-aec8-2c0e2d8aed07\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-wgnw7" Jul 05 13:19:21.275460 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:19:21.275278 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/beb3b580-1920-42d3-a152-ac42772476cf-snapshots\") pod \"insights-operator-5c847dcfc4-7pd6v\" (UID: \"beb3b580-1920-42d3-a152-ac42772476cf\") " pod="openshift-insights/insights-operator-5c847dcfc4-7pd6v" Jul 05 13:19:21.275460 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:19:21.275292 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/beb3b580-1920-42d3-a152-ac42772476cf-trusted-ca-bundle\") pod \"insights-operator-5c847dcfc4-7pd6v\" (UID: \"beb3b580-1920-42d3-a152-ac42772476cf\") " pod="openshift-insights/insights-operator-5c847dcfc4-7pd6v" Jul 05 13:19:21.275460 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:19:21.275393 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pxqqv\" (UniqueName: \"kubernetes.io/projected/ff7bc8c4-d331-46ec-a64e-f32d6ded6d9c-kube-api-access-pxqqv\") pod \"volume-data-source-validator-5d456fdbcd-x8wmq\" (UID: \"ff7bc8c4-d331-46ec-a64e-f32d6ded6d9c\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-x8wmq" Jul 05 13:19:21.275460 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:19:21.275437 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/91869a1a-47e4-4b83-aac5-5c22528d0940-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-d8wzk\" (UID: \"91869a1a-47e4-4b83-aac5-5c22528d0940\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-d8wzk" Jul 05 13:19:21.275882 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:19:21.275504 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/1c214417-9bef-4007-ab01-e61b84d1b35e-serving-cert\") pod \"console-operator-68b757fb67-tg2xx\" (UID: \"1c214417-9bef-4007-ab01-e61b84d1b35e\") " pod="openshift-console-operator/console-operator-68b757fb67-tg2xx" Jul 05 13:19:21.275882 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:19:21.275543 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pjb8t\" (UniqueName: \"kubernetes.io/projected/91869a1a-47e4-4b83-aac5-5c22528d0940-kube-api-access-pjb8t\") pod \"cluster-monitoring-operator-64786d9cbb-d8wzk\" (UID: \"91869a1a-47e4-4b83-aac5-5c22528d0940\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-d8wzk" Jul 05 13:19:21.275882 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:19:21.275612 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dv4t9\" (UniqueName: \"kubernetes.io/projected/1c214417-9bef-4007-ab01-e61b84d1b35e-kube-api-access-dv4t9\") pod \"console-operator-68b757fb67-tg2xx\" (UID: \"1c214417-9bef-4007-ab01-e61b84d1b35e\") " pod="openshift-console-operator/console-operator-68b757fb67-tg2xx" Jul 05 13:19:21.275882 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:19:21.275642 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/91869a1a-47e4-4b83-aac5-5c22528d0940-telemetry-config\") pod \"cluster-monitoring-operator-64786d9cbb-d8wzk\" (UID: \"91869a1a-47e4-4b83-aac5-5c22528d0940\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-d8wzk" Jul 05 13:19:21.275882 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:19:21.275686 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/beb3b580-1920-42d3-a152-ac42772476cf-tmp\") pod \"insights-operator-5c847dcfc4-7pd6v\" (UID: \"beb3b580-1920-42d3-a152-ac42772476cf\") " pod="openshift-insights/insights-operator-5c847dcfc4-7pd6v" Jul 05 13:19:21.275882 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:19:21.275744 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/b7b1094c-0b91-4c45-aec8-2c0e2d8aed07-config\") pod \"kube-storage-version-migrator-operator-846bf668f7-wgnw7\" (UID: \"b7b1094c-0b91-4c45-aec8-2c0e2d8aed07\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-wgnw7" Jul 05 13:19:21.275882 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:19:21.275781 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/1c214417-9bef-4007-ab01-e61b84d1b35e-trusted-ca\") pod \"console-operator-68b757fb67-tg2xx\" (UID: \"1c214417-9bef-4007-ab01-e61b84d1b35e\") " pod="openshift-console-operator/console-operator-68b757fb67-tg2xx" Jul 05 13:19:21.275882 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:19:21.275811 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/beb3b580-1920-42d3-a152-ac42772476cf-service-ca-bundle\") pod \"insights-operator-5c847dcfc4-7pd6v\" (UID: \"beb3b580-1920-42d3-a152-ac42772476cf\") " pod="openshift-insights/insights-operator-5c847dcfc4-7pd6v" Jul 05 13:19:21.275882 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:19:21.275835 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/1c214417-9bef-4007-ab01-e61b84d1b35e-config\") pod \"console-operator-68b757fb67-tg2xx\" (UID: \"1c214417-9bef-4007-ab01-e61b84d1b35e\") " pod="openshift-console-operator/console-operator-68b757fb67-tg2xx" Jul 05 13:19:21.276348 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:19:21.276081 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/beb3b580-1920-42d3-a152-ac42772476cf-snapshots\") pod \"insights-operator-5c847dcfc4-7pd6v\" (UID: \"beb3b580-1920-42d3-a152-ac42772476cf\") " pod="openshift-insights/insights-operator-5c847dcfc4-7pd6v" Jul 05 13:19:21.276348 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:19:21.276131 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/beb3b580-1920-42d3-a152-ac42772476cf-tmp\") pod \"insights-operator-5c847dcfc4-7pd6v\" (UID: \"beb3b580-1920-42d3-a152-ac42772476cf\") " pod="openshift-insights/insights-operator-5c847dcfc4-7pd6v" Jul 05 13:19:21.276633 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:19:21.276610 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/1c214417-9bef-4007-ab01-e61b84d1b35e-config\") pod \"console-operator-68b757fb67-tg2xx\" (UID: \"1c214417-9bef-4007-ab01-e61b84d1b35e\") " pod="openshift-console-operator/console-operator-68b757fb67-tg2xx" Jul 05 13:19:21.276743 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:19:21.276725 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/b7b1094c-0b91-4c45-aec8-2c0e2d8aed07-config\") pod \"kube-storage-version-migrator-operator-846bf668f7-wgnw7\" (UID: \"b7b1094c-0b91-4c45-aec8-2c0e2d8aed07\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-wgnw7" Jul 05 13:19:21.277044 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:19:21.277022 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/beb3b580-1920-42d3-a152-ac42772476cf-service-ca-bundle\") pod \"insights-operator-5c847dcfc4-7pd6v\" (UID: \"beb3b580-1920-42d3-a152-ac42772476cf\") " pod="openshift-insights/insights-operator-5c847dcfc4-7pd6v" Jul 05 13:19:21.277146 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:19:21.277117 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/1c214417-9bef-4007-ab01-e61b84d1b35e-trusted-ca\") pod \"console-operator-68b757fb67-tg2xx\" (UID: \"1c214417-9bef-4007-ab01-e61b84d1b35e\") " pod="openshift-console-operator/console-operator-68b757fb67-tg2xx" Jul 05 13:19:21.277286 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:19:21.277268 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/beb3b580-1920-42d3-a152-ac42772476cf-trusted-ca-bundle\") pod \"insights-operator-5c847dcfc4-7pd6v\" (UID: \"beb3b580-1920-42d3-a152-ac42772476cf\") " pod="openshift-insights/insights-operator-5c847dcfc4-7pd6v" Jul 05 13:19:21.278385 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:19:21.278351 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/b7b1094c-0b91-4c45-aec8-2c0e2d8aed07-serving-cert\") pod \"kube-storage-version-migrator-operator-846bf668f7-wgnw7\" (UID: \"b7b1094c-0b91-4c45-aec8-2c0e2d8aed07\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-wgnw7" Jul 05 13:19:21.278493 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:19:21.278415 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/1c214417-9bef-4007-ab01-e61b84d1b35e-serving-cert\") pod \"console-operator-68b757fb67-tg2xx\" (UID: \"1c214417-9bef-4007-ab01-e61b84d1b35e\") " pod="openshift-console-operator/console-operator-68b757fb67-tg2xx" Jul 05 13:19:21.279547 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:19:21.279530 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-jzwj6"] Jul 05 13:19:21.279618 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:19:21.279529 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/beb3b580-1920-42d3-a152-ac42772476cf-serving-cert\") pod \"insights-operator-5c847dcfc4-7pd6v\" (UID: \"beb3b580-1920-42d3-a152-ac42772476cf\") " pod="openshift-insights/insights-operator-5c847dcfc4-7pd6v" Jul 05 13:19:21.282055 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:19:21.282039 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-jzwj6" Jul 05 13:19:21.283428 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:19:21.283408 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-989f8b8dc-6j68f"] Jul 05 13:19:21.285224 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:19:21.285207 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-6j68f" Jul 05 13:19:21.285723 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:19:21.285708 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\"" Jul 05 13:19:21.285927 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:19:21.285909 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-rwcgp\"" Jul 05 13:19:21.286067 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:19:21.285961 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\"" Jul 05 13:19:21.286067 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:19:21.285977 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\"" Jul 05 13:19:21.286067 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:19:21.285992 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\"" Jul 05 13:19:21.287967 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:19:21.287952 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-kvqcm\"" Jul 05 13:19:21.300273 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:19:21.300252 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g6mbk\" (UniqueName: \"kubernetes.io/projected/beb3b580-1920-42d3-a152-ac42772476cf-kube-api-access-g6mbk\") pod \"insights-operator-5c847dcfc4-7pd6v\" (UID: \"beb3b580-1920-42d3-a152-ac42772476cf\") " pod="openshift-insights/insights-operator-5c847dcfc4-7pd6v" Jul 05 13:19:21.300445 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:19:21.300426 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zl988\" (UniqueName: \"kubernetes.io/projected/b7b1094c-0b91-4c45-aec8-2c0e2d8aed07-kube-api-access-zl988\") pod \"kube-storage-version-migrator-operator-846bf668f7-wgnw7\" (UID: \"b7b1094c-0b91-4c45-aec8-2c0e2d8aed07\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-wgnw7" Jul 05 13:19:21.300615 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:19:21.300594 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dv4t9\" (UniqueName: \"kubernetes.io/projected/1c214417-9bef-4007-ab01-e61b84d1b35e-kube-api-access-dv4t9\") pod \"console-operator-68b757fb67-tg2xx\" (UID: \"1c214417-9bef-4007-ab01-e61b84d1b35e\") " pod="openshift-console-operator/console-operator-68b757fb67-tg2xx" Jul 05 13:19:21.300734 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:19:21.300716 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pxqqv\" (UniqueName: \"kubernetes.io/projected/ff7bc8c4-d331-46ec-a64e-f32d6ded6d9c-kube-api-access-pxqqv\") pod \"volume-data-source-validator-5d456fdbcd-x8wmq\" (UID: \"ff7bc8c4-d331-46ec-a64e-f32d6ded6d9c\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-x8wmq" Jul 05 13:19:21.310564 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:19:21.310546 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-989f8b8dc-6j68f"] Jul 05 13:19:21.311769 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:19:21.311751 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-jzwj6"] Jul 05 13:19:21.376216 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:19:21.376137 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/69911263-1cea-4a74-a4ff-1cc94312366b-config\") pod \"service-ca-operator-5d46d9ffd4-jzwj6\" (UID: \"69911263-1cea-4a74-a4ff-1cc94312366b\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-jzwj6" Jul 05 13:19:21.376216 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:19:21.376202 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/91869a1a-47e4-4b83-aac5-5c22528d0940-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-d8wzk\" (UID: \"91869a1a-47e4-4b83-aac5-5c22528d0940\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-d8wzk" Jul 05 13:19:21.376427 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:19:21.376233 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/69911263-1cea-4a74-a4ff-1cc94312366b-serving-cert\") pod \"service-ca-operator-5d46d9ffd4-jzwj6\" (UID: \"69911263-1cea-4a74-a4ff-1cc94312366b\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-jzwj6" Jul 05 13:19:21.376427 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:19:21.376261 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pjb8t\" (UniqueName: \"kubernetes.io/projected/91869a1a-47e4-4b83-aac5-5c22528d0940-kube-api-access-pjb8t\") pod \"cluster-monitoring-operator-64786d9cbb-d8wzk\" (UID: \"91869a1a-47e4-4b83-aac5-5c22528d0940\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-d8wzk" Jul 05 13:19:21.376427 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:19:21.376295 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/91869a1a-47e4-4b83-aac5-5c22528d0940-telemetry-config\") pod \"cluster-monitoring-operator-64786d9cbb-d8wzk\" (UID: \"91869a1a-47e4-4b83-aac5-5c22528d0940\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-d8wzk" Jul 05 13:19:21.376427 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:19:21.376320 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rldhb\" (UniqueName: \"kubernetes.io/projected/69911263-1cea-4a74-a4ff-1cc94312366b-kube-api-access-rldhb\") pod \"service-ca-operator-5d46d9ffd4-jzwj6\" (UID: \"69911263-1cea-4a74-a4ff-1cc94312366b\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-jzwj6" Jul 05 13:19:21.376427 ip-10-0-130-191 kubenswrapper[2578]: E0705 13:19:21.376346 2578 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 05 13:19:21.376427 ip-10-0-130-191 kubenswrapper[2578]: E0705 13:19:21.376424 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/91869a1a-47e4-4b83-aac5-5c22528d0940-cluster-monitoring-operator-tls podName:91869a1a-47e4-4b83-aac5-5c22528d0940 nodeName:}" failed. No retries permitted until 2026-07-05 13:19:21.876402313 +0000 UTC m=+246.334965402 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/91869a1a-47e4-4b83-aac5-5c22528d0940-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-d8wzk" (UID: "91869a1a-47e4-4b83-aac5-5c22528d0940") : secret "cluster-monitoring-operator-tls" not found Jul 05 13:19:21.376733 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:19:21.376347 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xpfgp\" (UniqueName: \"kubernetes.io/projected/2c8c2bde-471a-4c81-b4c3-76372d887d50-kube-api-access-xpfgp\") pod \"network-check-source-989f8b8dc-6j68f\" (UID: \"2c8c2bde-471a-4c81-b4c3-76372d887d50\") " pod="openshift-network-diagnostics/network-check-source-989f8b8dc-6j68f" Jul 05 13:19:21.376952 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:19:21.376934 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/91869a1a-47e4-4b83-aac5-5c22528d0940-telemetry-config\") pod \"cluster-monitoring-operator-64786d9cbb-d8wzk\" (UID: \"91869a1a-47e4-4b83-aac5-5c22528d0940\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-d8wzk" Jul 05 13:19:21.388974 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:19:21.388943 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pjb8t\" (UniqueName: \"kubernetes.io/projected/91869a1a-47e4-4b83-aac5-5c22528d0940-kube-api-access-pjb8t\") pod \"cluster-monitoring-operator-64786d9cbb-d8wzk\" (UID: \"91869a1a-47e4-4b83-aac5-5c22528d0940\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-d8wzk" Jul 05 13:19:21.390395 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:19:21.390374 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5c847dcfc4-7pd6v" Jul 05 13:19:21.396560 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:19:21.396463 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-x8wmq" Jul 05 13:19:21.414717 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:19:21.414693 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-68b757fb67-tg2xx" Jul 05 13:19:21.419901 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:19:21.419871 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-wgnw7" Jul 05 13:19:21.477403 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:19:21.476886 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/69911263-1cea-4a74-a4ff-1cc94312366b-config\") pod \"service-ca-operator-5d46d9ffd4-jzwj6\" (UID: \"69911263-1cea-4a74-a4ff-1cc94312366b\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-jzwj6" Jul 05 13:19:21.477403 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:19:21.476991 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/69911263-1cea-4a74-a4ff-1cc94312366b-serving-cert\") pod \"service-ca-operator-5d46d9ffd4-jzwj6\" (UID: \"69911263-1cea-4a74-a4ff-1cc94312366b\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-jzwj6" Jul 05 13:19:21.477403 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:19:21.477049 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rldhb\" (UniqueName: \"kubernetes.io/projected/69911263-1cea-4a74-a4ff-1cc94312366b-kube-api-access-rldhb\") pod \"service-ca-operator-5d46d9ffd4-jzwj6\" (UID: \"69911263-1cea-4a74-a4ff-1cc94312366b\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-jzwj6" Jul 05 13:19:21.477403 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:19:21.477098 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xpfgp\" (UniqueName: \"kubernetes.io/projected/2c8c2bde-471a-4c81-b4c3-76372d887d50-kube-api-access-xpfgp\") pod \"network-check-source-989f8b8dc-6j68f\" (UID: \"2c8c2bde-471a-4c81-b4c3-76372d887d50\") " pod="openshift-network-diagnostics/network-check-source-989f8b8dc-6j68f" Jul 05 13:19:21.478884 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:19:21.478847 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/69911263-1cea-4a74-a4ff-1cc94312366b-config\") pod \"service-ca-operator-5d46d9ffd4-jzwj6\" (UID: \"69911263-1cea-4a74-a4ff-1cc94312366b\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-jzwj6" Jul 05 13:19:21.480633 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:19:21.480566 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/69911263-1cea-4a74-a4ff-1cc94312366b-serving-cert\") pod \"service-ca-operator-5d46d9ffd4-jzwj6\" (UID: \"69911263-1cea-4a74-a4ff-1cc94312366b\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-jzwj6" Jul 05 13:19:21.492344 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:19:21.492163 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rldhb\" (UniqueName: \"kubernetes.io/projected/69911263-1cea-4a74-a4ff-1cc94312366b-kube-api-access-rldhb\") pod \"service-ca-operator-5d46d9ffd4-jzwj6\" (UID: \"69911263-1cea-4a74-a4ff-1cc94312366b\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-jzwj6" Jul 05 13:19:21.492502 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:19:21.492354 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xpfgp\" (UniqueName: \"kubernetes.io/projected/2c8c2bde-471a-4c81-b4c3-76372d887d50-kube-api-access-xpfgp\") pod \"network-check-source-989f8b8dc-6j68f\" (UID: \"2c8c2bde-471a-4c81-b4c3-76372d887d50\") " pod="openshift-network-diagnostics/network-check-source-989f8b8dc-6j68f" Jul 05 13:19:21.555981 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:19:21.555948 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5c847dcfc4-7pd6v"] Jul 05 13:19:21.559235 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:19:21.559203 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbeb3b580_1920_42d3_a152_ac42772476cf.slice/crio-1691aa5c4cf573d43934c18a6592c5835763a56f178c4b467c9748ce87b867d0 WatchSource:0}: Error finding container 1691aa5c4cf573d43934c18a6592c5835763a56f178c4b467c9748ce87b867d0: Status 404 returned error can't find the container with id 1691aa5c4cf573d43934c18a6592c5835763a56f178c4b467c9748ce87b867d0 Jul 05 13:19:21.582961 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:19:21.582938 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-x8wmq"] Jul 05 13:19:21.585671 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:19:21.585644 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podff7bc8c4_d331_46ec_a64e_f32d6ded6d9c.slice/crio-7529f6e179af31491d17c81310a132579d92769cb8caa6590d85f674b7ee006e WatchSource:0}: Error finding container 7529f6e179af31491d17c81310a132579d92769cb8caa6590d85f674b7ee006e: Status 404 returned error can't find the container with id 7529f6e179af31491d17c81310a132579d92769cb8caa6590d85f674b7ee006e Jul 05 13:19:21.596963 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:19:21.596945 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-jzwj6" Jul 05 13:19:21.608568 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:19:21.608539 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-6j68f" Jul 05 13:19:21.735533 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:19:21.735502 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-jzwj6"] Jul 05 13:19:21.738486 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:19:21.738441 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod69911263_1cea_4a74_a4ff_1cc94312366b.slice/crio-7b2ce7d9783b94d303afdf8913917d4b4f0da0e6d4739c7b4e82225443711a51 WatchSource:0}: Error finding container 7b2ce7d9783b94d303afdf8913917d4b4f0da0e6d4739c7b4e82225443711a51: Status 404 returned error can't find the container with id 7b2ce7d9783b94d303afdf8913917d4b4f0da0e6d4739c7b4e82225443711a51 Jul 05 13:19:21.750304 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:19:21.750202 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-989f8b8dc-6j68f"] Jul 05 13:19:21.752443 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:19:21.752410 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2c8c2bde_471a_4c81_b4c3_76372d887d50.slice/crio-39b42df99967c68a332f6a20fcb09f7f106c847cf102ab893d6b71559ede0b13 WatchSource:0}: Error finding container 39b42df99967c68a332f6a20fcb09f7f106c847cf102ab893d6b71559ede0b13: Status 404 returned error can't find the container with id 39b42df99967c68a332f6a20fcb09f7f106c847cf102ab893d6b71559ede0b13 Jul 05 13:19:21.764017 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:19:21.763967 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-6j68f" event={"ID":"2c8c2bde-471a-4c81-b4c3-76372d887d50","Type":"ContainerStarted","Data":"39b42df99967c68a332f6a20fcb09f7f106c847cf102ab893d6b71559ede0b13"} Jul 05 13:19:21.765267 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:19:21.765239 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-x8wmq" event={"ID":"ff7bc8c4-d331-46ec-a64e-f32d6ded6d9c","Type":"ContainerStarted","Data":"7529f6e179af31491d17c81310a132579d92769cb8caa6590d85f674b7ee006e"} Jul 05 13:19:21.766824 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:19:21.766772 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-jzwj6" event={"ID":"69911263-1cea-4a74-a4ff-1cc94312366b","Type":"ContainerStarted","Data":"7b2ce7d9783b94d303afdf8913917d4b4f0da0e6d4739c7b4e82225443711a51"} Jul 05 13:19:21.768216 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:19:21.768171 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5c847dcfc4-7pd6v" event={"ID":"beb3b580-1920-42d3-a152-ac42772476cf","Type":"ContainerStarted","Data":"1691aa5c4cf573d43934c18a6592c5835763a56f178c4b467c9748ce87b867d0"} Jul 05 13:19:21.798283 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:19:21.798258 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-wgnw7"] Jul 05 13:19:21.801158 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:19:21.801132 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb7b1094c_0b91_4c45_aec8_2c0e2d8aed07.slice/crio-2905a3c25c9b0026d6f3bae239664974a88bf1cd041bebc5600a26d63ff73cfd WatchSource:0}: Error finding container 2905a3c25c9b0026d6f3bae239664974a88bf1cd041bebc5600a26d63ff73cfd: Status 404 returned error can't find the container with id 2905a3c25c9b0026d6f3bae239664974a88bf1cd041bebc5600a26d63ff73cfd Jul 05 13:19:21.809270 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:19:21.809246 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-68b757fb67-tg2xx"] Jul 05 13:19:21.814382 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:19:21.814360 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1c214417_9bef_4007_ab01_e61b84d1b35e.slice/crio-479a992ba07fa436f39dfcb2d307e6b36c0ed134adf0857f7c835769ab8dac47 WatchSource:0}: Error finding container 479a992ba07fa436f39dfcb2d307e6b36c0ed134adf0857f7c835769ab8dac47: Status 404 returned error can't find the container with id 479a992ba07fa436f39dfcb2d307e6b36c0ed134adf0857f7c835769ab8dac47 Jul 05 13:19:21.881484 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:19:21.881438 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/91869a1a-47e4-4b83-aac5-5c22528d0940-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-d8wzk\" (UID: \"91869a1a-47e4-4b83-aac5-5c22528d0940\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-d8wzk" Jul 05 13:19:21.881681 ip-10-0-130-191 kubenswrapper[2578]: E0705 13:19:21.881606 2578 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 05 13:19:21.881681 ip-10-0-130-191 kubenswrapper[2578]: E0705 13:19:21.881669 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/91869a1a-47e4-4b83-aac5-5c22528d0940-cluster-monitoring-operator-tls podName:91869a1a-47e4-4b83-aac5-5c22528d0940 nodeName:}" failed. No retries permitted until 2026-07-05 13:19:22.881653508 +0000 UTC m=+247.340216591 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/91869a1a-47e4-4b83-aac5-5c22528d0940-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-d8wzk" (UID: "91869a1a-47e4-4b83-aac5-5c22528d0940") : secret "cluster-monitoring-operator-tls" not found Jul 05 13:19:22.776315 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:19:22.776196 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-wgnw7" event={"ID":"b7b1094c-0b91-4c45-aec8-2c0e2d8aed07","Type":"ContainerStarted","Data":"2905a3c25c9b0026d6f3bae239664974a88bf1cd041bebc5600a26d63ff73cfd"} Jul 05 13:19:22.778707 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:19:22.778618 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-tg2xx" event={"ID":"1c214417-9bef-4007-ab01-e61b84d1b35e","Type":"ContainerStarted","Data":"479a992ba07fa436f39dfcb2d307e6b36c0ed134adf0857f7c835769ab8dac47"} Jul 05 13:19:22.782197 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:19:22.782137 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-6j68f" event={"ID":"2c8c2bde-471a-4c81-b4c3-76372d887d50","Type":"ContainerStarted","Data":"1d856e28ea3205113e30d59b03df24029849224c0f17836660e418fe6e959923"} Jul 05 13:19:22.785135 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:19:22.785104 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-x8wmq" event={"ID":"ff7bc8c4-d331-46ec-a64e-f32d6ded6d9c","Type":"ContainerStarted","Data":"768c021288d35364d3f872a37987273dd34f90094ad651e115f0c648c85126b5"} Jul 05 13:19:22.800875 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:19:22.800828 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-6j68f" podStartSLOduration=1.8008099 podStartE2EDuration="1.8008099s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 13:19:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 13:19:22.799639517 +0000 UTC m=+247.258202623" watchObservedRunningTime="2026-07-05 13:19:22.8008099 +0000 UTC m=+247.259373005" Jul 05 13:19:22.818011 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:19:22.817960 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-x8wmq" podStartSLOduration=0.792127467 podStartE2EDuration="1.817942548s" totalImagesPullingTime="1.025815081s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 13:19:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 13:19:22.817216707 +0000 UTC m=+247.275779812" watchObservedRunningTime="2026-07-05 13:19:22.817942548 +0000 UTC m=+247.276505656" Jul 05 13:19:22.890650 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:19:22.890304 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/91869a1a-47e4-4b83-aac5-5c22528d0940-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-d8wzk\" (UID: \"91869a1a-47e4-4b83-aac5-5c22528d0940\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-d8wzk" Jul 05 13:19:22.890650 ip-10-0-130-191 kubenswrapper[2578]: E0705 13:19:22.890554 2578 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 05 13:19:22.890879 ip-10-0-130-191 kubenswrapper[2578]: E0705 13:19:22.890653 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/91869a1a-47e4-4b83-aac5-5c22528d0940-cluster-monitoring-operator-tls podName:91869a1a-47e4-4b83-aac5-5c22528d0940 nodeName:}" failed. No retries permitted until 2026-07-05 13:19:24.89063183 +0000 UTC m=+249.349194919 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/91869a1a-47e4-4b83-aac5-5c22528d0940-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-d8wzk" (UID: "91869a1a-47e4-4b83-aac5-5c22528d0940") : secret "cluster-monitoring-operator-tls" not found Jul 05 13:19:24.908357 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:19:24.908319 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/91869a1a-47e4-4b83-aac5-5c22528d0940-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-d8wzk\" (UID: \"91869a1a-47e4-4b83-aac5-5c22528d0940\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-d8wzk" Jul 05 13:19:24.908796 ip-10-0-130-191 kubenswrapper[2578]: E0705 13:19:24.908472 2578 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 05 13:19:24.908796 ip-10-0-130-191 kubenswrapper[2578]: E0705 13:19:24.908534 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/91869a1a-47e4-4b83-aac5-5c22528d0940-cluster-monitoring-operator-tls podName:91869a1a-47e4-4b83-aac5-5c22528d0940 nodeName:}" failed. No retries permitted until 2026-07-05 13:19:28.908518149 +0000 UTC m=+253.367081234 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/91869a1a-47e4-4b83-aac5-5c22528d0940-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-d8wzk" (UID: "91869a1a-47e4-4b83-aac5-5c22528d0940") : secret "cluster-monitoring-operator-tls" not found Jul 05 13:19:25.799000 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:19:25.798935 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5c847dcfc4-7pd6v" event={"ID":"beb3b580-1920-42d3-a152-ac42772476cf","Type":"ContainerStarted","Data":"dcfa576596b24838b8aacb55877ddab396f9d3dd87f776c88a491e3d3d249b0f"} Jul 05 13:19:25.800538 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:19:25.800511 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-wgnw7" event={"ID":"b7b1094c-0b91-4c45-aec8-2c0e2d8aed07","Type":"ContainerStarted","Data":"062e43acf6338d5a5034497387e37315825797286bc2c8f83737c8b34fa45506"} Jul 05 13:19:25.802237 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:19:25.802215 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-tg2xx_1c214417-9bef-4007-ab01-e61b84d1b35e/console-operator/0.log" Jul 05 13:19:25.802332 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:19:25.802261 2578 generic.go:356] "Generic (PLEG): container finished" podID="1c214417-9bef-4007-ab01-e61b84d1b35e" containerID="51650b70f72aa7169f681826a6a5fe5074c2b2f2732bbb4a97e599fb708e7f48" exitCode=255 Jul 05 13:19:25.802385 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:19:25.802356 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-tg2xx" event={"ID":"1c214417-9bef-4007-ab01-e61b84d1b35e","Type":"ContainerDied","Data":"51650b70f72aa7169f681826a6a5fe5074c2b2f2732bbb4a97e599fb708e7f48"} Jul 05 13:19:25.802519 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:19:25.802504 2578 scope.go:117] "RemoveContainer" containerID="51650b70f72aa7169f681826a6a5fe5074c2b2f2732bbb4a97e599fb708e7f48" Jul 05 13:19:25.803885 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:19:25.803859 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-jzwj6" event={"ID":"69911263-1cea-4a74-a4ff-1cc94312366b","Type":"ContainerStarted","Data":"c851e73ede4f94b99a8a0856f2828fab98eb5d441764a62da24a0ab3a41a172a"} Jul 05 13:19:25.831877 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:19:25.831832 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-5c847dcfc4-7pd6v" podStartSLOduration=1.106371705 podStartE2EDuration="4.831815197s" totalImagesPullingTime="3.725443492s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 13:19:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 13:19:25.830657401 +0000 UTC m=+250.289220508" watchObservedRunningTime="2026-07-05 13:19:25.831815197 +0000 UTC m=+250.290378341" Jul 05 13:19:25.918499 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:19:25.918437 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-wgnw7" podStartSLOduration=1.4331606510000001 podStartE2EDuration="4.918415023s" totalImagesPullingTime="3.485254372s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 13:19:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 13:19:25.917612888 +0000 UTC m=+250.376175995" watchObservedRunningTime="2026-07-05 13:19:25.918415023 +0000 UTC m=+250.376978125" Jul 05 13:19:25.919000 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:19:25.918944 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-jzwj6" podStartSLOduration=1.367208675 podStartE2EDuration="4.918936755s" totalImagesPullingTime="3.55172808s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 13:19:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 13:19:25.881639561 +0000 UTC m=+250.340202667" watchObservedRunningTime="2026-07-05 13:19:25.918936755 +0000 UTC m=+250.377499862" Jul 05 13:19:26.077183 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:19:26.077081 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-djwgd"] Jul 05 13:19:26.080959 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:19:26.080933 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-djwgd" Jul 05 13:19:26.084977 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:19:26.084955 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 05 13:19:26.085304 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:19:26.084963 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-ztcxg\"" Jul 05 13:19:26.085682 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:19:26.085666 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 05 13:19:26.108635 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:19:26.108574 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-djwgd"] Jul 05 13:19:26.120193 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:19:26.120162 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7bf56499-2b24-4899-9eab-8c809d555735-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-djwgd\" (UID: \"7bf56499-2b24-4899-9eab-8c809d555735\") " pod="openshift-insights/insights-runtime-extractor-djwgd" Jul 05 13:19:26.120350 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:19:26.120266 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/7bf56499-2b24-4899-9eab-8c809d555735-crio-socket\") pod \"insights-runtime-extractor-djwgd\" (UID: \"7bf56499-2b24-4899-9eab-8c809d555735\") " pod="openshift-insights/insights-runtime-extractor-djwgd" Jul 05 13:19:26.120350 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:19:26.120320 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/7bf56499-2b24-4899-9eab-8c809d555735-data-volume\") pod \"insights-runtime-extractor-djwgd\" (UID: \"7bf56499-2b24-4899-9eab-8c809d555735\") " pod="openshift-insights/insights-runtime-extractor-djwgd" Jul 05 13:19:26.120350 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:19:26.120345 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/7bf56499-2b24-4899-9eab-8c809d555735-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-djwgd\" (UID: \"7bf56499-2b24-4899-9eab-8c809d555735\") " pod="openshift-insights/insights-runtime-extractor-djwgd" Jul 05 13:19:26.120467 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:19:26.120372 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6vx2b\" (UniqueName: \"kubernetes.io/projected/7bf56499-2b24-4899-9eab-8c809d555735-kube-api-access-6vx2b\") pod \"insights-runtime-extractor-djwgd\" (UID: \"7bf56499-2b24-4899-9eab-8c809d555735\") " pod="openshift-insights/insights-runtime-extractor-djwgd" Jul 05 13:19:26.221736 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:19:26.221697 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7bf56499-2b24-4899-9eab-8c809d555735-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-djwgd\" (UID: \"7bf56499-2b24-4899-9eab-8c809d555735\") " pod="openshift-insights/insights-runtime-extractor-djwgd" Jul 05 13:19:26.221906 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:19:26.221796 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/7bf56499-2b24-4899-9eab-8c809d555735-crio-socket\") pod \"insights-runtime-extractor-djwgd\" (UID: \"7bf56499-2b24-4899-9eab-8c809d555735\") " pod="openshift-insights/insights-runtime-extractor-djwgd" Jul 05 13:19:26.221906 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:19:26.221853 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/7bf56499-2b24-4899-9eab-8c809d555735-data-volume\") pod \"insights-runtime-extractor-djwgd\" (UID: \"7bf56499-2b24-4899-9eab-8c809d555735\") " pod="openshift-insights/insights-runtime-extractor-djwgd" Jul 05 13:19:26.221906 ip-10-0-130-191 kubenswrapper[2578]: E0705 13:19:26.221863 2578 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 05 13:19:26.221906 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:19:26.221879 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/7bf56499-2b24-4899-9eab-8c809d555735-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-djwgd\" (UID: \"7bf56499-2b24-4899-9eab-8c809d555735\") " pod="openshift-insights/insights-runtime-extractor-djwgd" Jul 05 13:19:26.222124 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:19:26.221907 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6vx2b\" (UniqueName: \"kubernetes.io/projected/7bf56499-2b24-4899-9eab-8c809d555735-kube-api-access-6vx2b\") pod \"insights-runtime-extractor-djwgd\" (UID: \"7bf56499-2b24-4899-9eab-8c809d555735\") " pod="openshift-insights/insights-runtime-extractor-djwgd" Jul 05 13:19:26.222124 ip-10-0-130-191 kubenswrapper[2578]: E0705 13:19:26.221932 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7bf56499-2b24-4899-9eab-8c809d555735-insights-runtime-extractor-tls podName:7bf56499-2b24-4899-9eab-8c809d555735 nodeName:}" failed. No retries permitted until 2026-07-05 13:19:26.721912531 +0000 UTC m=+251.180475615 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/7bf56499-2b24-4899-9eab-8c809d555735-insights-runtime-extractor-tls") pod "insights-runtime-extractor-djwgd" (UID: "7bf56499-2b24-4899-9eab-8c809d555735") : secret "insights-runtime-extractor-tls" not found Jul 05 13:19:26.222124 ip-10-0-130-191 kubenswrapper[2578]: E0705 13:19:26.222065 2578 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 05 13:19:26.222124 ip-10-0-130-191 kubenswrapper[2578]: E0705 13:19:26.222113 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/7bf56499-2b24-4899-9eab-8c809d555735-kube-rbac-proxy-cm podName:7bf56499-2b24-4899-9eab-8c809d555735 nodeName:}" failed. No retries permitted until 2026-07-05 13:19:26.722100196 +0000 UTC m=+251.180663297 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/7bf56499-2b24-4899-9eab-8c809d555735-kube-rbac-proxy-cm") pod "insights-runtime-extractor-djwgd" (UID: "7bf56499-2b24-4899-9eab-8c809d555735") : configmap "kube-rbac-proxy" not found Jul 05 13:19:26.222317 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:19:26.222251 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/7bf56499-2b24-4899-9eab-8c809d555735-data-volume\") pod \"insights-runtime-extractor-djwgd\" (UID: \"7bf56499-2b24-4899-9eab-8c809d555735\") " pod="openshift-insights/insights-runtime-extractor-djwgd" Jul 05 13:19:26.222382 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:19:26.222322 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/7bf56499-2b24-4899-9eab-8c809d555735-crio-socket\") pod \"insights-runtime-extractor-djwgd\" (UID: \"7bf56499-2b24-4899-9eab-8c809d555735\") " pod="openshift-insights/insights-runtime-extractor-djwgd" Jul 05 13:19:26.234208 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:19:26.234168 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6vx2b\" (UniqueName: \"kubernetes.io/projected/7bf56499-2b24-4899-9eab-8c809d555735-kube-api-access-6vx2b\") pod \"insights-runtime-extractor-djwgd\" (UID: \"7bf56499-2b24-4899-9eab-8c809d555735\") " pod="openshift-insights/insights-runtime-extractor-djwgd" Jul 05 13:19:26.724758 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:19:26.724725 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/7bf56499-2b24-4899-9eab-8c809d555735-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-djwgd\" (UID: \"7bf56499-2b24-4899-9eab-8c809d555735\") " pod="openshift-insights/insights-runtime-extractor-djwgd" Jul 05 13:19:26.724932 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:19:26.724786 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7bf56499-2b24-4899-9eab-8c809d555735-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-djwgd\" (UID: \"7bf56499-2b24-4899-9eab-8c809d555735\") " pod="openshift-insights/insights-runtime-extractor-djwgd" Jul 05 13:19:26.724932 ip-10-0-130-191 kubenswrapper[2578]: E0705 13:19:26.724878 2578 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 05 13:19:26.724932 ip-10-0-130-191 kubenswrapper[2578]: E0705 13:19:26.724893 2578 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 05 13:19:26.724932 ip-10-0-130-191 kubenswrapper[2578]: E0705 13:19:26.724928 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7bf56499-2b24-4899-9eab-8c809d555735-insights-runtime-extractor-tls podName:7bf56499-2b24-4899-9eab-8c809d555735 nodeName:}" failed. No retries permitted until 2026-07-05 13:19:27.724913757 +0000 UTC m=+252.183476841 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/7bf56499-2b24-4899-9eab-8c809d555735-insights-runtime-extractor-tls") pod "insights-runtime-extractor-djwgd" (UID: "7bf56499-2b24-4899-9eab-8c809d555735") : secret "insights-runtime-extractor-tls" not found Jul 05 13:19:26.725118 ip-10-0-130-191 kubenswrapper[2578]: E0705 13:19:26.724963 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/7bf56499-2b24-4899-9eab-8c809d555735-kube-rbac-proxy-cm podName:7bf56499-2b24-4899-9eab-8c809d555735 nodeName:}" failed. No retries permitted until 2026-07-05 13:19:27.724944746 +0000 UTC m=+252.183507835 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/7bf56499-2b24-4899-9eab-8c809d555735-kube-rbac-proxy-cm") pod "insights-runtime-extractor-djwgd" (UID: "7bf56499-2b24-4899-9eab-8c809d555735") : configmap "kube-rbac-proxy" not found Jul 05 13:19:26.809508 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:19:26.809476 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-tg2xx_1c214417-9bef-4007-ab01-e61b84d1b35e/console-operator/1.log" Jul 05 13:19:26.809923 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:19:26.809906 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-tg2xx_1c214417-9bef-4007-ab01-e61b84d1b35e/console-operator/0.log" Jul 05 13:19:26.810015 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:19:26.809941 2578 generic.go:356] "Generic (PLEG): container finished" podID="1c214417-9bef-4007-ab01-e61b84d1b35e" containerID="e09be34c67b7af7889a5d298a92daa6f2258a98cf523c585a76e27ef1bb2897e" exitCode=255 Jul 05 13:19:26.810015 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:19:26.809973 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-tg2xx" event={"ID":"1c214417-9bef-4007-ab01-e61b84d1b35e","Type":"ContainerDied","Data":"e09be34c67b7af7889a5d298a92daa6f2258a98cf523c585a76e27ef1bb2897e"} Jul 05 13:19:26.810096 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:19:26.810074 2578 scope.go:117] "RemoveContainer" containerID="51650b70f72aa7169f681826a6a5fe5074c2b2f2732bbb4a97e599fb708e7f48" Jul 05 13:19:26.810300 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:19:26.810269 2578 scope.go:117] "RemoveContainer" containerID="e09be34c67b7af7889a5d298a92daa6f2258a98cf523c585a76e27ef1bb2897e" Jul 05 13:19:26.810474 ip-10-0-130-191 kubenswrapper[2578]: E0705 13:19:26.810449 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-68b757fb67-tg2xx_openshift-console-operator(1c214417-9bef-4007-ab01-e61b84d1b35e)\"" pod="openshift-console-operator/console-operator-68b757fb67-tg2xx" podUID="1c214417-9bef-4007-ab01-e61b84d1b35e" Jul 05 13:19:27.409109 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:19:27.409069 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-d57d58cc9-4thc8"] Jul 05 13:19:27.416044 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:19:27.416015 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-4thc8" Jul 05 13:19:27.419894 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:19:27.419868 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\"" Jul 05 13:19:27.420021 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:19:27.419868 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-qtnpx\"" Jul 05 13:19:27.420774 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:19:27.420757 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\"" Jul 05 13:19:27.440939 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:19:27.440902 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-d57d58cc9-4thc8"] Jul 05 13:19:27.532839 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:19:27.532801 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d8rzv\" (UniqueName: \"kubernetes.io/projected/7f87b4fa-b050-448a-a65e-883ab774a48c-kube-api-access-d8rzv\") pod \"migrator-d57d58cc9-4thc8\" (UID: \"7f87b4fa-b050-448a-a65e-883ab774a48c\") " pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-4thc8" Jul 05 13:19:27.634003 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:19:27.633960 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d8rzv\" (UniqueName: \"kubernetes.io/projected/7f87b4fa-b050-448a-a65e-883ab774a48c-kube-api-access-d8rzv\") pod \"migrator-d57d58cc9-4thc8\" (UID: \"7f87b4fa-b050-448a-a65e-883ab774a48c\") " pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-4thc8" Jul 05 13:19:27.648598 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:19:27.648548 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d8rzv\" (UniqueName: \"kubernetes.io/projected/7f87b4fa-b050-448a-a65e-883ab774a48c-kube-api-access-d8rzv\") pod \"migrator-d57d58cc9-4thc8\" (UID: \"7f87b4fa-b050-448a-a65e-883ab774a48c\") " pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-4thc8" Jul 05 13:19:27.725924 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:19:27.725827 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-4thc8" Jul 05 13:19:27.735132 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:19:27.735099 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7bf56499-2b24-4899-9eab-8c809d555735-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-djwgd\" (UID: \"7bf56499-2b24-4899-9eab-8c809d555735\") " pod="openshift-insights/insights-runtime-extractor-djwgd" Jul 05 13:19:27.735272 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:19:27.735231 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/7bf56499-2b24-4899-9eab-8c809d555735-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-djwgd\" (UID: \"7bf56499-2b24-4899-9eab-8c809d555735\") " pod="openshift-insights/insights-runtime-extractor-djwgd" Jul 05 13:19:27.735315 ip-10-0-130-191 kubenswrapper[2578]: E0705 13:19:27.735286 2578 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 05 13:19:27.735346 ip-10-0-130-191 kubenswrapper[2578]: E0705 13:19:27.735333 2578 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 05 13:19:27.735378 ip-10-0-130-191 kubenswrapper[2578]: E0705 13:19:27.735368 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7bf56499-2b24-4899-9eab-8c809d555735-insights-runtime-extractor-tls podName:7bf56499-2b24-4899-9eab-8c809d555735 nodeName:}" failed. No retries permitted until 2026-07-05 13:19:29.735348092 +0000 UTC m=+254.193911188 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/7bf56499-2b24-4899-9eab-8c809d555735-insights-runtime-extractor-tls") pod "insights-runtime-extractor-djwgd" (UID: "7bf56499-2b24-4899-9eab-8c809d555735") : secret "insights-runtime-extractor-tls" not found Jul 05 13:19:27.735492 ip-10-0-130-191 kubenswrapper[2578]: E0705 13:19:27.735385 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/7bf56499-2b24-4899-9eab-8c809d555735-kube-rbac-proxy-cm podName:7bf56499-2b24-4899-9eab-8c809d555735 nodeName:}" failed. No retries permitted until 2026-07-05 13:19:29.735378403 +0000 UTC m=+254.193941486 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/7bf56499-2b24-4899-9eab-8c809d555735-kube-rbac-proxy-cm") pod "insights-runtime-extractor-djwgd" (UID: "7bf56499-2b24-4899-9eab-8c809d555735") : configmap "kube-rbac-proxy" not found Jul 05 13:19:27.815706 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:19:27.815678 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-tg2xx_1c214417-9bef-4007-ab01-e61b84d1b35e/console-operator/1.log" Jul 05 13:19:27.816077 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:19:27.816051 2578 scope.go:117] "RemoveContainer" containerID="e09be34c67b7af7889a5d298a92daa6f2258a98cf523c585a76e27ef1bb2897e" Jul 05 13:19:27.816281 ip-10-0-130-191 kubenswrapper[2578]: E0705 13:19:27.816254 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-68b757fb67-tg2xx_openshift-console-operator(1c214417-9bef-4007-ab01-e61b84d1b35e)\"" pod="openshift-console-operator/console-operator-68b757fb67-tg2xx" podUID="1c214417-9bef-4007-ab01-e61b84d1b35e" Jul 05 13:19:27.884916 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:19:27.884882 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-d57d58cc9-4thc8"] Jul 05 13:19:27.888323 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:19:27.888292 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7f87b4fa_b050_448a_a65e_883ab774a48c.slice/crio-60bf935689b923c44fdddf24b186982f53d112c3939783ac8d32739e1b9c4a4e WatchSource:0}: Error finding container 60bf935689b923c44fdddf24b186982f53d112c3939783ac8d32739e1b9c4a4e: Status 404 returned error can't find the container with id 60bf935689b923c44fdddf24b186982f53d112c3939783ac8d32739e1b9c4a4e Jul 05 13:19:27.904091 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:19:27.904067 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-btxlt_c01b0489-2ef4-4fbe-9f9e-1870bc580ee9/dns-node-resolver/0.log" Jul 05 13:19:27.937769 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:19:27.937735 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5ad92660-5823-4f14-adcc-763d06385f45-metrics-certs\") pod \"network-metrics-daemon-dxwj6\" (UID: \"5ad92660-5823-4f14-adcc-763d06385f45\") " pod="openshift-multus/network-metrics-daemon-dxwj6" Jul 05 13:19:27.937931 ip-10-0-130-191 kubenswrapper[2578]: E0705 13:19:27.937882 2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 05 13:19:27.938001 ip-10-0-130-191 kubenswrapper[2578]: E0705 13:19:27.937942 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5ad92660-5823-4f14-adcc-763d06385f45-metrics-certs podName:5ad92660-5823-4f14-adcc-763d06385f45 nodeName:}" failed. No retries permitted until 2026-07-05 13:21:29.937926757 +0000 UTC m=+374.396489851 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5ad92660-5823-4f14-adcc-763d06385f45-metrics-certs") pod "network-metrics-daemon-dxwj6" (UID: "5ad92660-5823-4f14-adcc-763d06385f45") : secret "metrics-daemon-secret" not found Jul 05 13:19:28.820223 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:19:28.820192 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-4thc8" event={"ID":"7f87b4fa-b050-448a-a65e-883ab774a48c","Type":"ContainerStarted","Data":"60bf935689b923c44fdddf24b186982f53d112c3939783ac8d32739e1b9c4a4e"} Jul 05 13:19:28.894669 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:19:28.894646 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-g7nxb_b14dc74b-da46-4d14-8fd6-fd9357838fe2/node-ca/0.log" Jul 05 13:19:28.946001 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:19:28.945959 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/91869a1a-47e4-4b83-aac5-5c22528d0940-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-d8wzk\" (UID: \"91869a1a-47e4-4b83-aac5-5c22528d0940\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-d8wzk" Jul 05 13:19:28.946171 ip-10-0-130-191 kubenswrapper[2578]: E0705 13:19:28.946108 2578 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 05 13:19:28.946220 ip-10-0-130-191 kubenswrapper[2578]: E0705 13:19:28.946174 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/91869a1a-47e4-4b83-aac5-5c22528d0940-cluster-monitoring-operator-tls podName:91869a1a-47e4-4b83-aac5-5c22528d0940 nodeName:}" failed. No retries permitted until 2026-07-05 13:19:36.946158543 +0000 UTC m=+261.404721628 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/91869a1a-47e4-4b83-aac5-5c22528d0940-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-d8wzk" (UID: "91869a1a-47e4-4b83-aac5-5c22528d0940") : secret "cluster-monitoring-operator-tls" not found Jul 05 13:19:29.753045 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:19:29.753001 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/7bf56499-2b24-4899-9eab-8c809d555735-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-djwgd\" (UID: \"7bf56499-2b24-4899-9eab-8c809d555735\") " pod="openshift-insights/insights-runtime-extractor-djwgd" Jul 05 13:19:29.753255 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:19:29.753094 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7bf56499-2b24-4899-9eab-8c809d555735-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-djwgd\" (UID: \"7bf56499-2b24-4899-9eab-8c809d555735\") " pod="openshift-insights/insights-runtime-extractor-djwgd" Jul 05 13:19:29.753255 ip-10-0-130-191 kubenswrapper[2578]: E0705 13:19:29.753137 2578 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 05 13:19:29.753255 ip-10-0-130-191 kubenswrapper[2578]: E0705 13:19:29.753199 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/7bf56499-2b24-4899-9eab-8c809d555735-kube-rbac-proxy-cm podName:7bf56499-2b24-4899-9eab-8c809d555735 nodeName:}" failed. No retries permitted until 2026-07-05 13:19:33.753181166 +0000 UTC m=+258.211744249 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/7bf56499-2b24-4899-9eab-8c809d555735-kube-rbac-proxy-cm") pod "insights-runtime-extractor-djwgd" (UID: "7bf56499-2b24-4899-9eab-8c809d555735") : configmap "kube-rbac-proxy" not found Jul 05 13:19:29.753417 ip-10-0-130-191 kubenswrapper[2578]: E0705 13:19:29.753261 2578 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 05 13:19:29.753417 ip-10-0-130-191 kubenswrapper[2578]: E0705 13:19:29.753332 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7bf56499-2b24-4899-9eab-8c809d555735-insights-runtime-extractor-tls podName:7bf56499-2b24-4899-9eab-8c809d555735 nodeName:}" failed. No retries permitted until 2026-07-05 13:19:33.753314657 +0000 UTC m=+258.211877761 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/7bf56499-2b24-4899-9eab-8c809d555735-insights-runtime-extractor-tls") pod "insights-runtime-extractor-djwgd" (UID: "7bf56499-2b24-4899-9eab-8c809d555735") : secret "insights-runtime-extractor-tls" not found Jul 05 13:19:29.825635 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:19:29.825607 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-4thc8" event={"ID":"7f87b4fa-b050-448a-a65e-883ab774a48c","Type":"ContainerStarted","Data":"9f5c721f1f494223dcacf15e86645c815dd2e9c61b076a56c4671a7bae4cdfc0"} Jul 05 13:19:29.826019 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:19:29.825657 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-4thc8" event={"ID":"7f87b4fa-b050-448a-a65e-883ab774a48c","Type":"ContainerStarted","Data":"a60674f0ec254573be83aaaa799637adc24f71b5d89a19a93d767cfd5837a665"} Jul 05 13:19:29.858762 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:19:29.858711 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-4thc8" podStartSLOduration=1.9597870149999999 podStartE2EDuration="2.858693061s" totalImagesPullingTime="898.906046ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 13:19:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 13:19:29.857867691 +0000 UTC m=+254.316430788" watchObservedRunningTime="2026-07-05 13:19:29.858693061 +0000 UTC m=+254.317256165" Jul 05 13:19:30.094914 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:19:30.094834 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-qn45h_853f791d-d350-4626-989e-270b15b79453/serve-healthcheck-canary/0.log" Jul 05 13:19:31.415007 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:19:31.414965 2578 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-68b757fb67-tg2xx" Jul 05 13:19:31.415380 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:19:31.415022 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-68b757fb67-tg2xx" Jul 05 13:19:31.415380 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:19:31.415343 2578 scope.go:117] "RemoveContainer" containerID="e09be34c67b7af7889a5d298a92daa6f2258a98cf523c585a76e27ef1bb2897e" Jul 05 13:19:31.415493 ip-10-0-130-191 kubenswrapper[2578]: E0705 13:19:31.415475 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-68b757fb67-tg2xx_openshift-console-operator(1c214417-9bef-4007-ab01-e61b84d1b35e)\"" pod="openshift-console-operator/console-operator-68b757fb67-tg2xx" podUID="1c214417-9bef-4007-ab01-e61b84d1b35e" Jul 05 13:19:33.790278 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:19:33.790239 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7bf56499-2b24-4899-9eab-8c809d555735-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-djwgd\" (UID: \"7bf56499-2b24-4899-9eab-8c809d555735\") " pod="openshift-insights/insights-runtime-extractor-djwgd" Jul 05 13:19:33.790671 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:19:33.790337 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/7bf56499-2b24-4899-9eab-8c809d555735-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-djwgd\" (UID: \"7bf56499-2b24-4899-9eab-8c809d555735\") " pod="openshift-insights/insights-runtime-extractor-djwgd" Jul 05 13:19:33.790671 ip-10-0-130-191 kubenswrapper[2578]: E0705 13:19:33.790407 2578 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 05 13:19:33.790671 ip-10-0-130-191 kubenswrapper[2578]: E0705 13:19:33.790424 2578 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 05 13:19:33.790671 ip-10-0-130-191 kubenswrapper[2578]: E0705 13:19:33.790487 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7bf56499-2b24-4899-9eab-8c809d555735-insights-runtime-extractor-tls podName:7bf56499-2b24-4899-9eab-8c809d555735 nodeName:}" failed. No retries permitted until 2026-07-05 13:19:41.790465997 +0000 UTC m=+266.249029085 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/7bf56499-2b24-4899-9eab-8c809d555735-insights-runtime-extractor-tls") pod "insights-runtime-extractor-djwgd" (UID: "7bf56499-2b24-4899-9eab-8c809d555735") : secret "insights-runtime-extractor-tls" not found Jul 05 13:19:33.790671 ip-10-0-130-191 kubenswrapper[2578]: E0705 13:19:33.790509 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/7bf56499-2b24-4899-9eab-8c809d555735-kube-rbac-proxy-cm podName:7bf56499-2b24-4899-9eab-8c809d555735 nodeName:}" failed. No retries permitted until 2026-07-05 13:19:41.790499497 +0000 UTC m=+266.249062585 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/7bf56499-2b24-4899-9eab-8c809d555735-kube-rbac-proxy-cm") pod "insights-runtime-extractor-djwgd" (UID: "7bf56499-2b24-4899-9eab-8c809d555735") : configmap "kube-rbac-proxy" not found Jul 05 13:19:37.015672 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:19:37.015571 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/91869a1a-47e4-4b83-aac5-5c22528d0940-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-d8wzk\" (UID: \"91869a1a-47e4-4b83-aac5-5c22528d0940\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-d8wzk" Jul 05 13:19:37.016039 ip-10-0-130-191 kubenswrapper[2578]: E0705 13:19:37.015712 2578 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 05 13:19:37.016039 ip-10-0-130-191 kubenswrapper[2578]: E0705 13:19:37.015778 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/91869a1a-47e4-4b83-aac5-5c22528d0940-cluster-monitoring-operator-tls podName:91869a1a-47e4-4b83-aac5-5c22528d0940 nodeName:}" failed. No retries permitted until 2026-07-05 13:19:53.015763432 +0000 UTC m=+277.474326515 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/91869a1a-47e4-4b83-aac5-5c22528d0940-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-d8wzk" (UID: "91869a1a-47e4-4b83-aac5-5c22528d0940") : secret "cluster-monitoring-operator-tls" not found Jul 05 13:19:41.855917 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:19:41.855865 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7bf56499-2b24-4899-9eab-8c809d555735-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-djwgd\" (UID: \"7bf56499-2b24-4899-9eab-8c809d555735\") " pod="openshift-insights/insights-runtime-extractor-djwgd" Jul 05 13:19:41.856389 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:19:41.855975 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/7bf56499-2b24-4899-9eab-8c809d555735-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-djwgd\" (UID: \"7bf56499-2b24-4899-9eab-8c809d555735\") " pod="openshift-insights/insights-runtime-extractor-djwgd" Jul 05 13:19:41.856389 ip-10-0-130-191 kubenswrapper[2578]: E0705 13:19:41.856040 2578 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 05 13:19:41.856389 ip-10-0-130-191 kubenswrapper[2578]: E0705 13:19:41.856137 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7bf56499-2b24-4899-9eab-8c809d555735-insights-runtime-extractor-tls podName:7bf56499-2b24-4899-9eab-8c809d555735 nodeName:}" failed. No retries permitted until 2026-07-05 13:19:57.856115566 +0000 UTC m=+282.314678651 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/7bf56499-2b24-4899-9eab-8c809d555735-insights-runtime-extractor-tls") pod "insights-runtime-extractor-djwgd" (UID: "7bf56499-2b24-4899-9eab-8c809d555735") : secret "insights-runtime-extractor-tls" not found Jul 05 13:19:41.856601 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:19:41.856436 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/7bf56499-2b24-4899-9eab-8c809d555735-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-djwgd\" (UID: \"7bf56499-2b24-4899-9eab-8c809d555735\") " pod="openshift-insights/insights-runtime-extractor-djwgd" Jul 05 13:19:46.004528 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:19:46.004496 2578 scope.go:117] "RemoveContainer" containerID="e09be34c67b7af7889a5d298a92daa6f2258a98cf523c585a76e27ef1bb2897e" Jul 05 13:19:46.884980 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:19:46.884951 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-tg2xx_1c214417-9bef-4007-ab01-e61b84d1b35e/console-operator/1.log" Jul 05 13:19:46.885159 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:19:46.885027 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-tg2xx" event={"ID":"1c214417-9bef-4007-ab01-e61b84d1b35e","Type":"ContainerStarted","Data":"17acb2878b78c869359c0c0d42dca1b3c1a66d5547971e53fa1b4351188c9c92"} Jul 05 13:19:46.885360 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:19:46.885340 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-68b757fb67-tg2xx" Jul 05 13:19:46.907656 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:19:46.907601 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-68b757fb67-tg2xx" podStartSLOduration=22.431634834 podStartE2EDuration="25.907566329s" totalImagesPullingTime="3.475931495s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 13:19:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 13:19:46.906994193 +0000 UTC m=+271.365557299" watchObservedRunningTime="2026-07-05 13:19:46.907566329 +0000 UTC m=+271.366129435" Jul 05 13:19:47.020397 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:19:47.020358 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-68b757fb67-tg2xx" Jul 05 13:19:53.045494 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:19:53.045455 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/91869a1a-47e4-4b83-aac5-5c22528d0940-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-d8wzk\" (UID: \"91869a1a-47e4-4b83-aac5-5c22528d0940\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-d8wzk" Jul 05 13:19:53.048039 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:19:53.048016 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/91869a1a-47e4-4b83-aac5-5c22528d0940-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-d8wzk\" (UID: \"91869a1a-47e4-4b83-aac5-5c22528d0940\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-d8wzk" Jul 05 13:19:53.292009 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:19:53.291959 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-d8wzk" Jul 05 13:19:53.440729 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:19:53.440646 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-64786d9cbb-d8wzk"] Jul 05 13:19:53.443440 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:19:53.443413 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod91869a1a_47e4_4b83_aac5_5c22528d0940.slice/crio-35164d6e46660c7bd5dfa807507fb899c5ba61538d580939e70272b486b5bdb8 WatchSource:0}: Error finding container 35164d6e46660c7bd5dfa807507fb899c5ba61538d580939e70272b486b5bdb8: Status 404 returned error can't find the container with id 35164d6e46660c7bd5dfa807507fb899c5ba61538d580939e70272b486b5bdb8 Jul 05 13:19:53.908678 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:19:53.908631 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-d8wzk" event={"ID":"91869a1a-47e4-4b83-aac5-5c22528d0940","Type":"ContainerStarted","Data":"35164d6e46660c7bd5dfa807507fb899c5ba61538d580939e70272b486b5bdb8"} Jul 05 13:19:55.920020 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:19:55.919979 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-d8wzk" event={"ID":"91869a1a-47e4-4b83-aac5-5c22528d0940","Type":"ContainerStarted","Data":"9557c3f8860b316d7b29a5865c2b818d509f4e32d0b6793f9ba8e671261d7402"} Jul 05 13:19:55.940732 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:19:55.940680 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-d8wzk" podStartSLOduration=33.470598941 podStartE2EDuration="34.940665726s" totalImagesPullingTime="1.470066785s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 13:19:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 13:19:55.940507692 +0000 UTC m=+280.399070794" watchObservedRunningTime="2026-07-05 13:19:55.940665726 +0000 UTC m=+280.399228831" Jul 05 13:19:57.559381 ip-10-0-130-191 kubenswrapper[2578]: E0705 13:19:57.559332 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[networking-console-plugin-cert], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-network-console/networking-console-plugin-6bffdf9778-ljfps" podUID="30164fef-3f92-4b9d-8c8f-ea7e6a3f1047" Jul 05 13:19:57.559381 ip-10-0-130-191 kubenswrapper[2578]: E0705 13:19:57.559332 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-w8bmv" podUID="79875c5a-6f9e-4d4f-9de9-5d974249f92b" Jul 05 13:19:57.559874 ip-10-0-130-191 kubenswrapper[2578]: E0705 13:19:57.559488 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[registry-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-image-registry/image-registry-57b9f5fbdc-gsjvv" podUID="d933ee11-a974-467b-9dce-f898bcf3863b" Jul 05 13:19:57.887970 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:19:57.887859 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7bf56499-2b24-4899-9eab-8c809d555735-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-djwgd\" (UID: \"7bf56499-2b24-4899-9eab-8c809d555735\") " pod="openshift-insights/insights-runtime-extractor-djwgd" Jul 05 13:19:57.890404 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:19:57.890378 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7bf56499-2b24-4899-9eab-8c809d555735-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-djwgd\" (UID: \"7bf56499-2b24-4899-9eab-8c809d555735\") " pod="openshift-insights/insights-runtime-extractor-djwgd" Jul 05 13:19:57.893270 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:19:57.893251 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-djwgd" Jul 05 13:19:57.926448 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:19:57.926414 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-57b9f5fbdc-gsjvv" Jul 05 13:19:57.926673 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:19:57.926414 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6bffdf9778-ljfps" Jul 05 13:19:57.926673 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:19:57.926414 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-w8bmv" Jul 05 13:19:58.054842 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:19:58.054809 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-djwgd"] Jul 05 13:19:58.057727 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:19:58.057700 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7bf56499_2b24_4899_9eab_8c809d555735.slice/crio-86ee77abd95a9abe4fccde57616f24e42ce3d2cadd939765275a541b5ef0b187 WatchSource:0}: Error finding container 86ee77abd95a9abe4fccde57616f24e42ce3d2cadd939765275a541b5ef0b187: Status 404 returned error can't find the container with id 86ee77abd95a9abe4fccde57616f24e42ce3d2cadd939765275a541b5ef0b187 Jul 05 13:19:58.932013 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:19:58.931975 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-djwgd" event={"ID":"7bf56499-2b24-4899-9eab-8c809d555735","Type":"ContainerStarted","Data":"86ee77abd95a9abe4fccde57616f24e42ce3d2cadd939765275a541b5ef0b187"} Jul 05 13:19:59.985436 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:19:59.985400 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-76df58b848-5cbkg"] Jul 05 13:19:59.995629 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:19:59.995600 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-76df58b848-5cbkg" Jul 05 13:19:59.999107 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:19:59.999079 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 05 13:19:59.999238 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:19:59.999079 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-9x89p\"" Jul 05 13:19:59.999238 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:19:59.999079 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 05 13:20:00.000516 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:00.000292 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 05 13:20:00.000516 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:00.000334 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 05 13:20:00.000516 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:00.000335 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 05 13:20:00.000516 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:00.000471 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 05 13:20:00.007187 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:00.007164 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 05 13:20:00.020567 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:00.020538 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-76df58b848-5cbkg"] Jul 05 13:20:00.110208 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:00.110173 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b68e67e7-b24f-4823-93ce-b1a21d9997df-console-serving-cert\") pod \"console-76df58b848-5cbkg\" (UID: \"b68e67e7-b24f-4823-93ce-b1a21d9997df\") " pod="openshift-console/console-76df58b848-5cbkg" Jul 05 13:20:00.110208 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:00.110217 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b68e67e7-b24f-4823-93ce-b1a21d9997df-service-ca\") pod \"console-76df58b848-5cbkg\" (UID: \"b68e67e7-b24f-4823-93ce-b1a21d9997df\") " pod="openshift-console/console-76df58b848-5cbkg" Jul 05 13:20:00.110457 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:00.110305 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b68e67e7-b24f-4823-93ce-b1a21d9997df-console-config\") pod \"console-76df58b848-5cbkg\" (UID: \"b68e67e7-b24f-4823-93ce-b1a21d9997df\") " pod="openshift-console/console-76df58b848-5cbkg" Jul 05 13:20:00.110457 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:00.110421 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jwd82\" (UniqueName: \"kubernetes.io/projected/b68e67e7-b24f-4823-93ce-b1a21d9997df-kube-api-access-jwd82\") pod \"console-76df58b848-5cbkg\" (UID: \"b68e67e7-b24f-4823-93ce-b1a21d9997df\") " pod="openshift-console/console-76df58b848-5cbkg" Jul 05 13:20:00.110567 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:00.110502 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b68e67e7-b24f-4823-93ce-b1a21d9997df-oauth-serving-cert\") pod \"console-76df58b848-5cbkg\" (UID: \"b68e67e7-b24f-4823-93ce-b1a21d9997df\") " pod="openshift-console/console-76df58b848-5cbkg" Jul 05 13:20:00.110567 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:00.110552 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b68e67e7-b24f-4823-93ce-b1a21d9997df-console-oauth-config\") pod \"console-76df58b848-5cbkg\" (UID: \"b68e67e7-b24f-4823-93ce-b1a21d9997df\") " pod="openshift-console/console-76df58b848-5cbkg" Jul 05 13:20:00.211087 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:00.211055 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jwd82\" (UniqueName: \"kubernetes.io/projected/b68e67e7-b24f-4823-93ce-b1a21d9997df-kube-api-access-jwd82\") pod \"console-76df58b848-5cbkg\" (UID: \"b68e67e7-b24f-4823-93ce-b1a21d9997df\") " pod="openshift-console/console-76df58b848-5cbkg" Jul 05 13:20:00.211276 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:00.211111 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b68e67e7-b24f-4823-93ce-b1a21d9997df-oauth-serving-cert\") pod \"console-76df58b848-5cbkg\" (UID: \"b68e67e7-b24f-4823-93ce-b1a21d9997df\") " pod="openshift-console/console-76df58b848-5cbkg" Jul 05 13:20:00.211276 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:00.211150 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b68e67e7-b24f-4823-93ce-b1a21d9997df-console-oauth-config\") pod \"console-76df58b848-5cbkg\" (UID: \"b68e67e7-b24f-4823-93ce-b1a21d9997df\") " pod="openshift-console/console-76df58b848-5cbkg" Jul 05 13:20:00.211276 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:00.211176 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b68e67e7-b24f-4823-93ce-b1a21d9997df-console-serving-cert\") pod \"console-76df58b848-5cbkg\" (UID: \"b68e67e7-b24f-4823-93ce-b1a21d9997df\") " pod="openshift-console/console-76df58b848-5cbkg" Jul 05 13:20:00.211276 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:00.211194 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b68e67e7-b24f-4823-93ce-b1a21d9997df-service-ca\") pod \"console-76df58b848-5cbkg\" (UID: \"b68e67e7-b24f-4823-93ce-b1a21d9997df\") " pod="openshift-console/console-76df58b848-5cbkg" Jul 05 13:20:00.211276 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:00.211242 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b68e67e7-b24f-4823-93ce-b1a21d9997df-console-config\") pod \"console-76df58b848-5cbkg\" (UID: \"b68e67e7-b24f-4823-93ce-b1a21d9997df\") " pod="openshift-console/console-76df58b848-5cbkg" Jul 05 13:20:00.212068 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:00.212021 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b68e67e7-b24f-4823-93ce-b1a21d9997df-console-config\") pod \"console-76df58b848-5cbkg\" (UID: \"b68e67e7-b24f-4823-93ce-b1a21d9997df\") " pod="openshift-console/console-76df58b848-5cbkg" Jul 05 13:20:00.212068 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:00.212044 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b68e67e7-b24f-4823-93ce-b1a21d9997df-service-ca\") pod \"console-76df58b848-5cbkg\" (UID: \"b68e67e7-b24f-4823-93ce-b1a21d9997df\") " pod="openshift-console/console-76df58b848-5cbkg" Jul 05 13:20:00.212068 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:00.212024 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b68e67e7-b24f-4823-93ce-b1a21d9997df-oauth-serving-cert\") pod \"console-76df58b848-5cbkg\" (UID: \"b68e67e7-b24f-4823-93ce-b1a21d9997df\") " pod="openshift-console/console-76df58b848-5cbkg" Jul 05 13:20:00.214141 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:00.214112 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b68e67e7-b24f-4823-93ce-b1a21d9997df-console-serving-cert\") pod \"console-76df58b848-5cbkg\" (UID: \"b68e67e7-b24f-4823-93ce-b1a21d9997df\") " pod="openshift-console/console-76df58b848-5cbkg" Jul 05 13:20:00.214268 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:00.214160 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b68e67e7-b24f-4823-93ce-b1a21d9997df-console-oauth-config\") pod \"console-76df58b848-5cbkg\" (UID: \"b68e67e7-b24f-4823-93ce-b1a21d9997df\") " pod="openshift-console/console-76df58b848-5cbkg" Jul 05 13:20:00.221532 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:00.221462 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jwd82\" (UniqueName: \"kubernetes.io/projected/b68e67e7-b24f-4823-93ce-b1a21d9997df-kube-api-access-jwd82\") pod \"console-76df58b848-5cbkg\" (UID: \"b68e67e7-b24f-4823-93ce-b1a21d9997df\") " pod="openshift-console/console-76df58b848-5cbkg" Jul 05 13:20:00.306629 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:00.306574 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-76df58b848-5cbkg" Jul 05 13:20:01.422770 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:01.422731 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/79875c5a-6f9e-4d4f-9de9-5d974249f92b-metrics-tls\") pod \"dns-default-w8bmv\" (UID: \"79875c5a-6f9e-4d4f-9de9-5d974249f92b\") " pod="openshift-dns/dns-default-w8bmv" Jul 05 13:20:01.423215 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:01.422794 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/30164fef-3f92-4b9d-8c8f-ea7e6a3f1047-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-ljfps\" (UID: \"30164fef-3f92-4b9d-8c8f-ea7e6a3f1047\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-ljfps" Jul 05 13:20:01.423215 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:01.422843 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d933ee11-a974-467b-9dce-f898bcf3863b-registry-tls\") pod \"image-registry-57b9f5fbdc-gsjvv\" (UID: \"d933ee11-a974-467b-9dce-f898bcf3863b\") " pod="openshift-image-registry/image-registry-57b9f5fbdc-gsjvv" Jul 05 13:20:01.425623 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:01.425572 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/79875c5a-6f9e-4d4f-9de9-5d974249f92b-metrics-tls\") pod \"dns-default-w8bmv\" (UID: \"79875c5a-6f9e-4d4f-9de9-5d974249f92b\") " pod="openshift-dns/dns-default-w8bmv" Jul 05 13:20:01.425761 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:01.425678 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/30164fef-3f92-4b9d-8c8f-ea7e6a3f1047-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-ljfps\" (UID: \"30164fef-3f92-4b9d-8c8f-ea7e6a3f1047\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-ljfps" Jul 05 13:20:01.425824 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:01.425754 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d933ee11-a974-467b-9dce-f898bcf3863b-registry-tls\") pod \"image-registry-57b9f5fbdc-gsjvv\" (UID: \"d933ee11-a974-467b-9dce-f898bcf3863b\") " pod="openshift-image-registry/image-registry-57b9f5fbdc-gsjvv" Jul 05 13:20:01.532790 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:01.532755 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-6fcfs\"" Jul 05 13:20:01.532790 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:01.532759 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-9gzst\"" Jul 05 13:20:01.533025 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:01.532796 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-9lt9j\"" Jul 05 13:20:01.537934 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:01.537903 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6bffdf9778-ljfps" Jul 05 13:20:01.538055 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:01.537949 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-w8bmv" Jul 05 13:20:01.538055 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:01.538001 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-57b9f5fbdc-gsjvv" Jul 05 13:20:01.893280 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:01.893248 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6bffdf9778-ljfps"] Jul 05 13:20:01.896303 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:20:01.896272 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod30164fef_3f92_4b9d_8c8f_ea7e6a3f1047.slice/crio-72a4b8b3430a5e2247eaaa812252847097b8757132b14a5154006f2e0ade9221 WatchSource:0}: Error finding container 72a4b8b3430a5e2247eaaa812252847097b8757132b14a5154006f2e0ade9221: Status 404 returned error can't find the container with id 72a4b8b3430a5e2247eaaa812252847097b8757132b14a5154006f2e0ade9221 Jul 05 13:20:01.904573 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:01.904546 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-76df58b848-5cbkg"] Jul 05 13:20:01.907808 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:20:01.907782 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb68e67e7_b24f_4823_93ce_b1a21d9997df.slice/crio-15bad8e6611bd536809c17386cf9d27a64722a8c10dd92496befde3110adf46d WatchSource:0}: Error finding container 15bad8e6611bd536809c17386cf9d27a64722a8c10dd92496befde3110adf46d: Status 404 returned error can't find the container with id 15bad8e6611bd536809c17386cf9d27a64722a8c10dd92496befde3110adf46d Jul 05 13:20:01.946766 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:01.946735 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6bffdf9778-ljfps" event={"ID":"30164fef-3f92-4b9d-8c8f-ea7e6a3f1047","Type":"ContainerStarted","Data":"72a4b8b3430a5e2247eaaa812252847097b8757132b14a5154006f2e0ade9221"} Jul 05 13:20:01.947983 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:01.947955 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-76df58b848-5cbkg" event={"ID":"b68e67e7-b24f-4823-93ce-b1a21d9997df","Type":"ContainerStarted","Data":"15bad8e6611bd536809c17386cf9d27a64722a8c10dd92496befde3110adf46d"} Jul 05 13:20:01.949450 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:01.949431 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-djwgd" event={"ID":"7bf56499-2b24-4899-9eab-8c809d555735","Type":"ContainerStarted","Data":"689c21ad5f95c70d503f09c2a2aaca0bee2db2d8f3e9d4b0d6cebaca2017b631"} Jul 05 13:20:02.133817 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:02.133775 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-w8bmv"] Jul 05 13:20:02.136386 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:20:02.136361 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod79875c5a_6f9e_4d4f_9de9_5d974249f92b.slice/crio-47fcca64d064039fde11bca1b352f705e6564685de1cfe75618fca0c3ce1734d WatchSource:0}: Error finding container 47fcca64d064039fde11bca1b352f705e6564685de1cfe75618fca0c3ce1734d: Status 404 returned error can't find the container with id 47fcca64d064039fde11bca1b352f705e6564685de1cfe75618fca0c3ce1734d Jul 05 13:20:02.147761 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:02.147738 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-57b9f5fbdc-gsjvv"] Jul 05 13:20:02.148707 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:20:02.148679 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd933ee11_a974_467b_9dce_f898bcf3863b.slice/crio-7b3b180697c84ddc43330d235a858532f61011454ffdf3f69c2aa53cdbf8e1ef WatchSource:0}: Error finding container 7b3b180697c84ddc43330d235a858532f61011454ffdf3f69c2aa53cdbf8e1ef: Status 404 returned error can't find the container with id 7b3b180697c84ddc43330d235a858532f61011454ffdf3f69c2aa53cdbf8e1ef Jul 05 13:20:02.957529 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:02.957492 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-djwgd" event={"ID":"7bf56499-2b24-4899-9eab-8c809d555735","Type":"ContainerStarted","Data":"5a75a2bdf9f7ebdd326ffb6205679a1deff1791aabd8e1d40d9c18e688c460f2"} Jul 05 13:20:02.959937 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:02.959402 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-57b9f5fbdc-gsjvv" event={"ID":"d933ee11-a974-467b-9dce-f898bcf3863b","Type":"ContainerStarted","Data":"998daa2e6543e4ae237d93b5321ed62f4acaf3155781d814ce61b7b32b888434"} Jul 05 13:20:02.959937 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:02.959458 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-57b9f5fbdc-gsjvv" event={"ID":"d933ee11-a974-467b-9dce-f898bcf3863b","Type":"ContainerStarted","Data":"7b3b180697c84ddc43330d235a858532f61011454ffdf3f69c2aa53cdbf8e1ef"} Jul 05 13:20:02.959937 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:02.959511 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-57b9f5fbdc-gsjvv" Jul 05 13:20:02.960794 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:02.960738 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-w8bmv" event={"ID":"79875c5a-6f9e-4d4f-9de9-5d974249f92b","Type":"ContainerStarted","Data":"47fcca64d064039fde11bca1b352f705e6564685de1cfe75618fca0c3ce1734d"} Jul 05 13:20:02.993922 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:02.993809 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-57b9f5fbdc-gsjvv" podStartSLOduration=286.993786245 podStartE2EDuration="4m46.993786245s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 13:15:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 13:20:02.99027359 +0000 UTC m=+287.448836699" watchObservedRunningTime="2026-07-05 13:20:02.993786245 +0000 UTC m=+287.452349350" Jul 05 13:20:03.608139 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:03.608108 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-5f8697c67c-crgzp"] Jul 05 13:20:03.612736 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:03.612713 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5f8697c67c-crgzp" Jul 05 13:20:03.623785 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:03.623761 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 05 13:20:03.642379 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:03.642311 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5f8697c67c-crgzp"] Jul 05 13:20:03.743683 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:03.743442 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8cm7x\" (UniqueName: \"kubernetes.io/projected/3644ed9d-8c13-448b-98ab-4017dab5ea27-kube-api-access-8cm7x\") pod \"console-5f8697c67c-crgzp\" (UID: \"3644ed9d-8c13-448b-98ab-4017dab5ea27\") " pod="openshift-console/console-5f8697c67c-crgzp" Jul 05 13:20:03.743683 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:03.743499 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3644ed9d-8c13-448b-98ab-4017dab5ea27-console-serving-cert\") pod \"console-5f8697c67c-crgzp\" (UID: \"3644ed9d-8c13-448b-98ab-4017dab5ea27\") " pod="openshift-console/console-5f8697c67c-crgzp" Jul 05 13:20:03.743683 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:03.743536 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3644ed9d-8c13-448b-98ab-4017dab5ea27-console-config\") pod \"console-5f8697c67c-crgzp\" (UID: \"3644ed9d-8c13-448b-98ab-4017dab5ea27\") " pod="openshift-console/console-5f8697c67c-crgzp" Jul 05 13:20:03.743683 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:03.743594 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3644ed9d-8c13-448b-98ab-4017dab5ea27-console-oauth-config\") pod \"console-5f8697c67c-crgzp\" (UID: \"3644ed9d-8c13-448b-98ab-4017dab5ea27\") " pod="openshift-console/console-5f8697c67c-crgzp" Jul 05 13:20:03.743683 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:03.743627 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3644ed9d-8c13-448b-98ab-4017dab5ea27-oauth-serving-cert\") pod \"console-5f8697c67c-crgzp\" (UID: \"3644ed9d-8c13-448b-98ab-4017dab5ea27\") " pod="openshift-console/console-5f8697c67c-crgzp" Jul 05 13:20:03.744094 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:03.743709 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3644ed9d-8c13-448b-98ab-4017dab5ea27-trusted-ca-bundle\") pod \"console-5f8697c67c-crgzp\" (UID: \"3644ed9d-8c13-448b-98ab-4017dab5ea27\") " pod="openshift-console/console-5f8697c67c-crgzp" Jul 05 13:20:03.744094 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:03.743818 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3644ed9d-8c13-448b-98ab-4017dab5ea27-service-ca\") pod \"console-5f8697c67c-crgzp\" (UID: \"3644ed9d-8c13-448b-98ab-4017dab5ea27\") " pod="openshift-console/console-5f8697c67c-crgzp" Jul 05 13:20:03.845142 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:03.844988 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3644ed9d-8c13-448b-98ab-4017dab5ea27-trusted-ca-bundle\") pod \"console-5f8697c67c-crgzp\" (UID: \"3644ed9d-8c13-448b-98ab-4017dab5ea27\") " pod="openshift-console/console-5f8697c67c-crgzp" Jul 05 13:20:03.845142 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:03.845076 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3644ed9d-8c13-448b-98ab-4017dab5ea27-service-ca\") pod \"console-5f8697c67c-crgzp\" (UID: \"3644ed9d-8c13-448b-98ab-4017dab5ea27\") " pod="openshift-console/console-5f8697c67c-crgzp" Jul 05 13:20:03.846546 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:03.846514 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3644ed9d-8c13-448b-98ab-4017dab5ea27-trusted-ca-bundle\") pod \"console-5f8697c67c-crgzp\" (UID: \"3644ed9d-8c13-448b-98ab-4017dab5ea27\") " pod="openshift-console/console-5f8697c67c-crgzp" Jul 05 13:20:03.846736 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:03.846710 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8cm7x\" (UniqueName: \"kubernetes.io/projected/3644ed9d-8c13-448b-98ab-4017dab5ea27-kube-api-access-8cm7x\") pod \"console-5f8697c67c-crgzp\" (UID: \"3644ed9d-8c13-448b-98ab-4017dab5ea27\") " pod="openshift-console/console-5f8697c67c-crgzp" Jul 05 13:20:03.846850 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:03.846801 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3644ed9d-8c13-448b-98ab-4017dab5ea27-console-serving-cert\") pod \"console-5f8697c67c-crgzp\" (UID: \"3644ed9d-8c13-448b-98ab-4017dab5ea27\") " pod="openshift-console/console-5f8697c67c-crgzp" Jul 05 13:20:03.846850 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:03.846846 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3644ed9d-8c13-448b-98ab-4017dab5ea27-console-config\") pod \"console-5f8697c67c-crgzp\" (UID: \"3644ed9d-8c13-448b-98ab-4017dab5ea27\") " pod="openshift-console/console-5f8697c67c-crgzp" Jul 05 13:20:03.846963 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:03.846895 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3644ed9d-8c13-448b-98ab-4017dab5ea27-console-oauth-config\") pod \"console-5f8697c67c-crgzp\" (UID: \"3644ed9d-8c13-448b-98ab-4017dab5ea27\") " pod="openshift-console/console-5f8697c67c-crgzp" Jul 05 13:20:03.846963 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:03.846946 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3644ed9d-8c13-448b-98ab-4017dab5ea27-oauth-serving-cert\") pod \"console-5f8697c67c-crgzp\" (UID: \"3644ed9d-8c13-448b-98ab-4017dab5ea27\") " pod="openshift-console/console-5f8697c67c-crgzp" Jul 05 13:20:03.847718 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:03.847692 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3644ed9d-8c13-448b-98ab-4017dab5ea27-oauth-serving-cert\") pod \"console-5f8697c67c-crgzp\" (UID: \"3644ed9d-8c13-448b-98ab-4017dab5ea27\") " pod="openshift-console/console-5f8697c67c-crgzp" Jul 05 13:20:03.851610 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:03.848730 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3644ed9d-8c13-448b-98ab-4017dab5ea27-console-config\") pod \"console-5f8697c67c-crgzp\" (UID: \"3644ed9d-8c13-448b-98ab-4017dab5ea27\") " pod="openshift-console/console-5f8697c67c-crgzp" Jul 05 13:20:03.851610 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:03.848796 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3644ed9d-8c13-448b-98ab-4017dab5ea27-service-ca\") pod \"console-5f8697c67c-crgzp\" (UID: \"3644ed9d-8c13-448b-98ab-4017dab5ea27\") " pod="openshift-console/console-5f8697c67c-crgzp" Jul 05 13:20:03.855081 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:03.855053 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3644ed9d-8c13-448b-98ab-4017dab5ea27-console-serving-cert\") pod \"console-5f8697c67c-crgzp\" (UID: \"3644ed9d-8c13-448b-98ab-4017dab5ea27\") " pod="openshift-console/console-5f8697c67c-crgzp" Jul 05 13:20:03.855561 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:03.855538 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3644ed9d-8c13-448b-98ab-4017dab5ea27-console-oauth-config\") pod \"console-5f8697c67c-crgzp\" (UID: \"3644ed9d-8c13-448b-98ab-4017dab5ea27\") " pod="openshift-console/console-5f8697c67c-crgzp" Jul 05 13:20:03.864601 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:03.864514 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8cm7x\" (UniqueName: \"kubernetes.io/projected/3644ed9d-8c13-448b-98ab-4017dab5ea27-kube-api-access-8cm7x\") pod \"console-5f8697c67c-crgzp\" (UID: \"3644ed9d-8c13-448b-98ab-4017dab5ea27\") " pod="openshift-console/console-5f8697c67c-crgzp" Jul 05 13:20:03.925945 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:03.925910 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5f8697c67c-crgzp" Jul 05 13:20:04.779202 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:04.779165 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-567f7b8b7c-w4t2w"] Jul 05 13:20:04.782658 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:04.782637 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-w4t2w" Jul 05 13:20:04.787538 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:04.787516 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\"" Jul 05 13:20:04.789016 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:04.788826 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\"" Jul 05 13:20:04.789016 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:04.788858 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 05 13:20:04.789380 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:04.789352 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-j6mhl\"" Jul 05 13:20:04.819953 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:04.819919 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-567f7b8b7c-w4t2w"] Jul 05 13:20:04.957866 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:04.957830 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e45113b4-0332-41bf-9b40-5b3be8506ccc-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-567f7b8b7c-w4t2w\" (UID: \"e45113b4-0332-41bf-9b40-5b3be8506ccc\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-w4t2w" Jul 05 13:20:04.958039 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:04.957898 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e45113b4-0332-41bf-9b40-5b3be8506ccc-metrics-client-ca\") pod \"prometheus-operator-567f7b8b7c-w4t2w\" (UID: \"e45113b4-0332-41bf-9b40-5b3be8506ccc\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-w4t2w" Jul 05 13:20:04.958039 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:04.957962 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pbzhq\" (UniqueName: \"kubernetes.io/projected/e45113b4-0332-41bf-9b40-5b3be8506ccc-kube-api-access-pbzhq\") pod \"prometheus-operator-567f7b8b7c-w4t2w\" (UID: \"e45113b4-0332-41bf-9b40-5b3be8506ccc\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-w4t2w" Jul 05 13:20:04.958039 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:04.957996 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/e45113b4-0332-41bf-9b40-5b3be8506ccc-prometheus-operator-tls\") pod \"prometheus-operator-567f7b8b7c-w4t2w\" (UID: \"e45113b4-0332-41bf-9b40-5b3be8506ccc\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-w4t2w" Jul 05 13:20:05.058964 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:05.058889 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e45113b4-0332-41bf-9b40-5b3be8506ccc-metrics-client-ca\") pod \"prometheus-operator-567f7b8b7c-w4t2w\" (UID: \"e45113b4-0332-41bf-9b40-5b3be8506ccc\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-w4t2w" Jul 05 13:20:05.058964 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:05.058944 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pbzhq\" (UniqueName: \"kubernetes.io/projected/e45113b4-0332-41bf-9b40-5b3be8506ccc-kube-api-access-pbzhq\") pod \"prometheus-operator-567f7b8b7c-w4t2w\" (UID: \"e45113b4-0332-41bf-9b40-5b3be8506ccc\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-w4t2w" Jul 05 13:20:05.059192 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:05.058981 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/e45113b4-0332-41bf-9b40-5b3be8506ccc-prometheus-operator-tls\") pod \"prometheus-operator-567f7b8b7c-w4t2w\" (UID: \"e45113b4-0332-41bf-9b40-5b3be8506ccc\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-w4t2w" Jul 05 13:20:05.059192 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:05.059043 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e45113b4-0332-41bf-9b40-5b3be8506ccc-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-567f7b8b7c-w4t2w\" (UID: \"e45113b4-0332-41bf-9b40-5b3be8506ccc\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-w4t2w" Jul 05 13:20:05.059750 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:05.059717 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e45113b4-0332-41bf-9b40-5b3be8506ccc-metrics-client-ca\") pod \"prometheus-operator-567f7b8b7c-w4t2w\" (UID: \"e45113b4-0332-41bf-9b40-5b3be8506ccc\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-w4t2w" Jul 05 13:20:05.061925 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:05.061899 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e45113b4-0332-41bf-9b40-5b3be8506ccc-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-567f7b8b7c-w4t2w\" (UID: \"e45113b4-0332-41bf-9b40-5b3be8506ccc\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-w4t2w" Jul 05 13:20:05.062152 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:05.062123 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/e45113b4-0332-41bf-9b40-5b3be8506ccc-prometheus-operator-tls\") pod \"prometheus-operator-567f7b8b7c-w4t2w\" (UID: \"e45113b4-0332-41bf-9b40-5b3be8506ccc\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-w4t2w" Jul 05 13:20:05.072103 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:05.072076 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pbzhq\" (UniqueName: \"kubernetes.io/projected/e45113b4-0332-41bf-9b40-5b3be8506ccc-kube-api-access-pbzhq\") pod \"prometheus-operator-567f7b8b7c-w4t2w\" (UID: \"e45113b4-0332-41bf-9b40-5b3be8506ccc\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-w4t2w" Jul 05 13:20:05.094918 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:05.094886 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-w4t2w" Jul 05 13:20:06.792737 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:06.792707 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5f8697c67c-crgzp"] Jul 05 13:20:06.796013 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:20:06.795983 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3644ed9d_8c13_448b_98ab_4017dab5ea27.slice/crio-2f430fdb91722562cc05904927090e83118dd16c2f747aa78570a5a880d9503e WatchSource:0}: Error finding container 2f430fdb91722562cc05904927090e83118dd16c2f747aa78570a5a880d9503e: Status 404 returned error can't find the container with id 2f430fdb91722562cc05904927090e83118dd16c2f747aa78570a5a880d9503e Jul 05 13:20:06.830984 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:06.830910 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-567f7b8b7c-w4t2w"] Jul 05 13:20:06.841688 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:20:06.841644 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode45113b4_0332_41bf_9b40_5b3be8506ccc.slice/crio-00d67a41b1e29a77b053739cbc4c0bdb48c193217c1917d9453640ce4ce11c92 WatchSource:0}: Error finding container 00d67a41b1e29a77b053739cbc4c0bdb48c193217c1917d9453640ce4ce11c92: Status 404 returned error can't find the container with id 00d67a41b1e29a77b053739cbc4c0bdb48c193217c1917d9453640ce4ce11c92 Jul 05 13:20:06.978356 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:06.978321 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-76df58b848-5cbkg" event={"ID":"b68e67e7-b24f-4823-93ce-b1a21d9997df","Type":"ContainerStarted","Data":"667ab25291abd91a77282af694d4156f144d94c81ebdd67d9268376efda3624e"} Jul 05 13:20:06.980354 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:06.980325 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-djwgd" event={"ID":"7bf56499-2b24-4899-9eab-8c809d555735","Type":"ContainerStarted","Data":"5e41877e8282ca202cb87b74a4052c8f0406bc5d89c4b79e134d83343ea86c49"} Jul 05 13:20:06.981567 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:06.981545 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-w4t2w" event={"ID":"e45113b4-0332-41bf-9b40-5b3be8506ccc","Type":"ContainerStarted","Data":"00d67a41b1e29a77b053739cbc4c0bdb48c193217c1917d9453640ce4ce11c92"} Jul 05 13:20:06.983261 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:06.983243 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-w8bmv" event={"ID":"79875c5a-6f9e-4d4f-9de9-5d974249f92b","Type":"ContainerStarted","Data":"03604597af4b8a1e952d74c373631b7a0e51ab547fe94dedd58a10aa7b92c633"} Jul 05 13:20:06.983337 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:06.983275 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-w8bmv" event={"ID":"79875c5a-6f9e-4d4f-9de9-5d974249f92b","Type":"ContainerStarted","Data":"2f671a8723bcd33133edb7409b6f613d645923e8a02922e1c331eb8c70456527"} Jul 05 13:20:06.983403 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:06.983354 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-w8bmv" Jul 05 13:20:06.984712 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:06.984693 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5f8697c67c-crgzp" event={"ID":"3644ed9d-8c13-448b-98ab-4017dab5ea27","Type":"ContainerStarted","Data":"e0582c36d6621c6232af3416fcc858784b47ceb9f0556cfb0dea73177120ca2c"} Jul 05 13:20:06.984800 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:06.984735 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5f8697c67c-crgzp" event={"ID":"3644ed9d-8c13-448b-98ab-4017dab5ea27","Type":"ContainerStarted","Data":"2f430fdb91722562cc05904927090e83118dd16c2f747aa78570a5a880d9503e"} Jul 05 13:20:06.986022 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:06.986002 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6bffdf9778-ljfps" event={"ID":"30164fef-3f92-4b9d-8c8f-ea7e6a3f1047","Type":"ContainerStarted","Data":"09e06be3669226989219be96e093b979907fdcb7d378a24fffc236accf5101c1"} Jul 05 13:20:07.016955 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:07.016883 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-76df58b848-5cbkg" podStartSLOduration=3.326404949 podStartE2EDuration="8.016868296s" totalImagesPullingTime="4.690463347s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 13:19:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 13:20:07.01597644 +0000 UTC m=+291.474539545" watchObservedRunningTime="2026-07-05 13:20:07.016868296 +0000 UTC m=+291.475431379" Jul 05 13:20:07.068294 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:07.068241 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-5f8697c67c-crgzp" podStartSLOduration=4.06822543 podStartE2EDuration="4.06822543s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 13:20:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 13:20:07.0668069 +0000 UTC m=+291.525370008" watchObservedRunningTime="2026-07-05 13:20:07.06822543 +0000 UTC m=+291.526788535" Jul 05 13:20:07.107126 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:07.107015 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-6bffdf9778-ljfps" podStartSLOduration=280.412397695 podStartE2EDuration="4m45.106997652s" totalImagesPullingTime="4.694599957s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 13:15:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 13:20:07.105932194 +0000 UTC m=+291.564495299" watchObservedRunningTime="2026-07-05 13:20:07.106997652 +0000 UTC m=+291.565560981" Jul 05 13:20:07.146417 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:07.146364 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-djwgd" podStartSLOduration=32.739160959 podStartE2EDuration="41.146347072s" totalImagesPullingTime="8.407186113s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-05 13:19:26 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 13:20:07.14591224 +0000 UTC m=+291.604475346" watchObservedRunningTime="2026-07-05 13:20:07.146347072 +0000 UTC m=+291.604910176" Jul 05 13:20:07.187504 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:07.187453 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-w8bmv" podStartSLOduration=251.729173108 podStartE2EDuration="4m16.187434501s" totalImagesPullingTime="4.458261393s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 13:15:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 13:20:07.186502517 +0000 UTC m=+291.645065622" watchObservedRunningTime="2026-07-05 13:20:07.187434501 +0000 UTC m=+291.645997654" Jul 05 13:20:08.995877 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:08.995845 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-w4t2w" event={"ID":"e45113b4-0332-41bf-9b40-5b3be8506ccc","Type":"ContainerStarted","Data":"577c23e0dc1daf10bbb54524e201425313fee5d8ee6f95e67f5768f1c670ee2b"} Jul 05 13:20:08.996340 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:08.995896 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-w4t2w" event={"ID":"e45113b4-0332-41bf-9b40-5b3be8506ccc","Type":"ContainerStarted","Data":"8cf2c17a532f928bbb774fb105aeb4673884cd3fd2c05f751c17598d6574b769"} Jul 05 13:20:09.004978 ip-10-0-130-191 kubenswrapper[2578]: E0705 13:20:09.004948 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-dxwj6" podUID="5ad92660-5823-4f14-adcc-763d06385f45" Jul 05 13:20:09.038820 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:09.038772 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-w4t2w" podStartSLOduration=3.742905602 podStartE2EDuration="5.038754815s" totalImagesPullingTime="1.295849213s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 13:20:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 13:20:09.037911542 +0000 UTC m=+293.496474654" watchObservedRunningTime="2026-07-05 13:20:09.038754815 +0000 UTC m=+293.497317957" Jul 05 13:20:10.307861 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:10.307826 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-76df58b848-5cbkg" Jul 05 13:20:10.308283 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:10.307887 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-76df58b848-5cbkg" Jul 05 13:20:10.309227 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:10.309202 2578 patch_prober.go:28] interesting pod/console-76df58b848-5cbkg container/console namespace/openshift-console: Startup probe status=failure output="Get \"https://10.132.0.19:8443/health\": dial tcp 10.132.0.19:8443: connect: connection refused" start-of-body= Jul 05 13:20:10.309328 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:10.309249 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-console/console-76df58b848-5cbkg" podUID="b68e67e7-b24f-4823-93ce-b1a21d9997df" containerName="console" probeResult="failure" output="Get \"https://10.132.0.19:8443/health\": dial tcp 10.132.0.19:8443: connect: connection refused" Jul 05 13:20:11.676670 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:11.676637 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-kghbq"] Jul 05 13:20:11.679149 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:11.679131 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-kghbq" Jul 05 13:20:11.682562 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:11.682542 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 05 13:20:11.683001 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:11.682973 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-2xspd\"" Jul 05 13:20:11.683001 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:11.682997 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 05 13:20:11.683153 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:11.683130 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 05 13:20:11.724820 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:11.724778 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/c182307f-a4e2-4f81-97bc-fb7835a2a2ba-node-exporter-accelerators-collector-config\") pod \"node-exporter-kghbq\" (UID: \"c182307f-a4e2-4f81-97bc-fb7835a2a2ba\") " pod="openshift-monitoring/node-exporter-kghbq" Jul 05 13:20:11.725012 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:11.724849 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/c182307f-a4e2-4f81-97bc-fb7835a2a2ba-node-exporter-tls\") pod \"node-exporter-kghbq\" (UID: \"c182307f-a4e2-4f81-97bc-fb7835a2a2ba\") " pod="openshift-monitoring/node-exporter-kghbq" Jul 05 13:20:11.725012 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:11.724902 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c182307f-a4e2-4f81-97bc-fb7835a2a2ba-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-kghbq\" (UID: \"c182307f-a4e2-4f81-97bc-fb7835a2a2ba\") " pod="openshift-monitoring/node-exporter-kghbq" Jul 05 13:20:11.725012 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:11.724934 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c7jdn\" (UniqueName: \"kubernetes.io/projected/c182307f-a4e2-4f81-97bc-fb7835a2a2ba-kube-api-access-c7jdn\") pod \"node-exporter-kghbq\" (UID: \"c182307f-a4e2-4f81-97bc-fb7835a2a2ba\") " pod="openshift-monitoring/node-exporter-kghbq" Jul 05 13:20:11.725012 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:11.724996 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/c182307f-a4e2-4f81-97bc-fb7835a2a2ba-root\") pod \"node-exporter-kghbq\" (UID: \"c182307f-a4e2-4f81-97bc-fb7835a2a2ba\") " pod="openshift-monitoring/node-exporter-kghbq" Jul 05 13:20:11.725251 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:11.725025 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c182307f-a4e2-4f81-97bc-fb7835a2a2ba-sys\") pod \"node-exporter-kghbq\" (UID: \"c182307f-a4e2-4f81-97bc-fb7835a2a2ba\") " pod="openshift-monitoring/node-exporter-kghbq" Jul 05 13:20:11.725251 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:11.725049 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c182307f-a4e2-4f81-97bc-fb7835a2a2ba-metrics-client-ca\") pod \"node-exporter-kghbq\" (UID: \"c182307f-a4e2-4f81-97bc-fb7835a2a2ba\") " pod="openshift-monitoring/node-exporter-kghbq" Jul 05 13:20:11.725251 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:11.725175 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/c182307f-a4e2-4f81-97bc-fb7835a2a2ba-node-exporter-textfile\") pod \"node-exporter-kghbq\" (UID: \"c182307f-a4e2-4f81-97bc-fb7835a2a2ba\") " pod="openshift-monitoring/node-exporter-kghbq" Jul 05 13:20:11.725251 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:11.725221 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/c182307f-a4e2-4f81-97bc-fb7835a2a2ba-node-exporter-wtmp\") pod \"node-exporter-kghbq\" (UID: \"c182307f-a4e2-4f81-97bc-fb7835a2a2ba\") " pod="openshift-monitoring/node-exporter-kghbq" Jul 05 13:20:11.826275 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:11.826236 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/c182307f-a4e2-4f81-97bc-fb7835a2a2ba-node-exporter-textfile\") pod \"node-exporter-kghbq\" (UID: \"c182307f-a4e2-4f81-97bc-fb7835a2a2ba\") " pod="openshift-monitoring/node-exporter-kghbq" Jul 05 13:20:11.826275 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:11.826274 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/c182307f-a4e2-4f81-97bc-fb7835a2a2ba-node-exporter-wtmp\") pod \"node-exporter-kghbq\" (UID: \"c182307f-a4e2-4f81-97bc-fb7835a2a2ba\") " pod="openshift-monitoring/node-exporter-kghbq" Jul 05 13:20:11.826531 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:11.826296 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/c182307f-a4e2-4f81-97bc-fb7835a2a2ba-node-exporter-accelerators-collector-config\") pod \"node-exporter-kghbq\" (UID: \"c182307f-a4e2-4f81-97bc-fb7835a2a2ba\") " pod="openshift-monitoring/node-exporter-kghbq" Jul 05 13:20:11.826531 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:11.826324 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/c182307f-a4e2-4f81-97bc-fb7835a2a2ba-node-exporter-tls\") pod \"node-exporter-kghbq\" (UID: \"c182307f-a4e2-4f81-97bc-fb7835a2a2ba\") " pod="openshift-monitoring/node-exporter-kghbq" Jul 05 13:20:11.826531 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:11.826350 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c182307f-a4e2-4f81-97bc-fb7835a2a2ba-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-kghbq\" (UID: \"c182307f-a4e2-4f81-97bc-fb7835a2a2ba\") " pod="openshift-monitoring/node-exporter-kghbq" Jul 05 13:20:11.826531 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:11.826370 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c7jdn\" (UniqueName: \"kubernetes.io/projected/c182307f-a4e2-4f81-97bc-fb7835a2a2ba-kube-api-access-c7jdn\") pod \"node-exporter-kghbq\" (UID: \"c182307f-a4e2-4f81-97bc-fb7835a2a2ba\") " pod="openshift-monitoring/node-exporter-kghbq" Jul 05 13:20:11.826531 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:11.826393 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/c182307f-a4e2-4f81-97bc-fb7835a2a2ba-root\") pod \"node-exporter-kghbq\" (UID: \"c182307f-a4e2-4f81-97bc-fb7835a2a2ba\") " pod="openshift-monitoring/node-exporter-kghbq" Jul 05 13:20:11.826531 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:11.826421 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c182307f-a4e2-4f81-97bc-fb7835a2a2ba-sys\") pod \"node-exporter-kghbq\" (UID: \"c182307f-a4e2-4f81-97bc-fb7835a2a2ba\") " pod="openshift-monitoring/node-exporter-kghbq" Jul 05 13:20:11.826531 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:11.826436 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/c182307f-a4e2-4f81-97bc-fb7835a2a2ba-node-exporter-wtmp\") pod \"node-exporter-kghbq\" (UID: \"c182307f-a4e2-4f81-97bc-fb7835a2a2ba\") " pod="openshift-monitoring/node-exporter-kghbq" Jul 05 13:20:11.826531 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:11.826443 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c182307f-a4e2-4f81-97bc-fb7835a2a2ba-metrics-client-ca\") pod \"node-exporter-kghbq\" (UID: \"c182307f-a4e2-4f81-97bc-fb7835a2a2ba\") " pod="openshift-monitoring/node-exporter-kghbq" Jul 05 13:20:11.826531 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:11.826491 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c182307f-a4e2-4f81-97bc-fb7835a2a2ba-sys\") pod \"node-exporter-kghbq\" (UID: \"c182307f-a4e2-4f81-97bc-fb7835a2a2ba\") " pod="openshift-monitoring/node-exporter-kghbq" Jul 05 13:20:11.826531 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:11.826510 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/c182307f-a4e2-4f81-97bc-fb7835a2a2ba-root\") pod \"node-exporter-kghbq\" (UID: \"c182307f-a4e2-4f81-97bc-fb7835a2a2ba\") " pod="openshift-monitoring/node-exporter-kghbq" Jul 05 13:20:11.826991 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:11.826695 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/c182307f-a4e2-4f81-97bc-fb7835a2a2ba-node-exporter-textfile\") pod \"node-exporter-kghbq\" (UID: \"c182307f-a4e2-4f81-97bc-fb7835a2a2ba\") " pod="openshift-monitoring/node-exporter-kghbq" Jul 05 13:20:11.827068 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:11.827051 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c182307f-a4e2-4f81-97bc-fb7835a2a2ba-metrics-client-ca\") pod \"node-exporter-kghbq\" (UID: \"c182307f-a4e2-4f81-97bc-fb7835a2a2ba\") " pod="openshift-monitoring/node-exporter-kghbq" Jul 05 13:20:11.827141 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:11.827120 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/c182307f-a4e2-4f81-97bc-fb7835a2a2ba-node-exporter-accelerators-collector-config\") pod \"node-exporter-kghbq\" (UID: \"c182307f-a4e2-4f81-97bc-fb7835a2a2ba\") " pod="openshift-monitoring/node-exporter-kghbq" Jul 05 13:20:11.828848 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:11.828825 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c182307f-a4e2-4f81-97bc-fb7835a2a2ba-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-kghbq\" (UID: \"c182307f-a4e2-4f81-97bc-fb7835a2a2ba\") " pod="openshift-monitoring/node-exporter-kghbq" Jul 05 13:20:11.828957 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:11.828938 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/c182307f-a4e2-4f81-97bc-fb7835a2a2ba-node-exporter-tls\") pod \"node-exporter-kghbq\" (UID: \"c182307f-a4e2-4f81-97bc-fb7835a2a2ba\") " pod="openshift-monitoring/node-exporter-kghbq" Jul 05 13:20:11.856293 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:11.856258 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c7jdn\" (UniqueName: \"kubernetes.io/projected/c182307f-a4e2-4f81-97bc-fb7835a2a2ba-kube-api-access-c7jdn\") pod \"node-exporter-kghbq\" (UID: \"c182307f-a4e2-4f81-97bc-fb7835a2a2ba\") " pod="openshift-monitoring/node-exporter-kghbq" Jul 05 13:20:11.988708 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:11.988610 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-kghbq" Jul 05 13:20:11.999407 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:20:11.999374 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc182307f_a4e2_4f81_97bc_fb7835a2a2ba.slice/crio-678b0913681505a8a7ebf6749b18a32b2b1446a45ef8c34f373fee30d87c0115 WatchSource:0}: Error finding container 678b0913681505a8a7ebf6749b18a32b2b1446a45ef8c34f373fee30d87c0115: Status 404 returned error can't find the container with id 678b0913681505a8a7ebf6749b18a32b2b1446a45ef8c34f373fee30d87c0115 Jul 05 13:20:12.008922 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:12.008894 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-kghbq" event={"ID":"c182307f-a4e2-4f81-97bc-fb7835a2a2ba","Type":"ContainerStarted","Data":"678b0913681505a8a7ebf6749b18a32b2b1446a45ef8c34f373fee30d87c0115"} Jul 05 13:20:13.013812 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:13.013782 2578 generic.go:356] "Generic (PLEG): container finished" podID="c182307f-a4e2-4f81-97bc-fb7835a2a2ba" containerID="94480c3b4c42d5d7f83c567407d13c515a34a3492e20874a4b479ba6a468a244" exitCode=0 Jul 05 13:20:13.014172 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:13.013866 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-kghbq" event={"ID":"c182307f-a4e2-4f81-97bc-fb7835a2a2ba","Type":"ContainerDied","Data":"94480c3b4c42d5d7f83c567407d13c515a34a3492e20874a4b479ba6a468a244"} Jul 05 13:20:13.394216 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:13.394135 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 05 13:20:13.397396 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:13.397380 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 05 13:20:13.410042 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:13.410019 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jul 05 13:20:13.410195 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:13.410111 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jul 05 13:20:13.410195 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:13.410164 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jul 05 13:20:13.410336 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:13.410263 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jul 05 13:20:13.410527 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:13.410513 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-vg85r\"" Jul 05 13:20:13.410602 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:13.410562 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jul 05 13:20:13.423921 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:13.423885 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jul 05 13:20:13.423921 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:13.423885 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jul 05 13:20:13.424144 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:13.423886 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jul 05 13:20:13.429906 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:13.429884 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jul 05 13:20:13.442231 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:13.442198 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/aab13c90-c88f-42ce-83df-d29f7455f494-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"aab13c90-c88f-42ce-83df-d29f7455f494\") " pod="openshift-monitoring/alertmanager-main-0" Jul 05 13:20:13.442383 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:13.442241 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x2cb8\" (UniqueName: \"kubernetes.io/projected/aab13c90-c88f-42ce-83df-d29f7455f494-kube-api-access-x2cb8\") pod \"alertmanager-main-0\" (UID: \"aab13c90-c88f-42ce-83df-d29f7455f494\") " pod="openshift-monitoring/alertmanager-main-0" Jul 05 13:20:13.442383 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:13.442328 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/aab13c90-c88f-42ce-83df-d29f7455f494-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"aab13c90-c88f-42ce-83df-d29f7455f494\") " pod="openshift-monitoring/alertmanager-main-0" Jul 05 13:20:13.442383 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:13.442367 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/aab13c90-c88f-42ce-83df-d29f7455f494-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"aab13c90-c88f-42ce-83df-d29f7455f494\") " pod="openshift-monitoring/alertmanager-main-0" Jul 05 13:20:13.442533 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:13.442420 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/aab13c90-c88f-42ce-83df-d29f7455f494-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"aab13c90-c88f-42ce-83df-d29f7455f494\") " pod="openshift-monitoring/alertmanager-main-0" Jul 05 13:20:13.442533 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:13.442485 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/aab13c90-c88f-42ce-83df-d29f7455f494-tls-assets\") pod \"alertmanager-main-0\" (UID: \"aab13c90-c88f-42ce-83df-d29f7455f494\") " pod="openshift-monitoring/alertmanager-main-0" Jul 05 13:20:13.442533 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:13.442523 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/aab13c90-c88f-42ce-83df-d29f7455f494-config-volume\") pod \"alertmanager-main-0\" (UID: \"aab13c90-c88f-42ce-83df-d29f7455f494\") " pod="openshift-monitoring/alertmanager-main-0" Jul 05 13:20:13.442660 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:13.442563 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/aab13c90-c88f-42ce-83df-d29f7455f494-config-out\") pod \"alertmanager-main-0\" (UID: \"aab13c90-c88f-42ce-83df-d29f7455f494\") " pod="openshift-monitoring/alertmanager-main-0" Jul 05 13:20:13.442660 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:13.442614 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/aab13c90-c88f-42ce-83df-d29f7455f494-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"aab13c90-c88f-42ce-83df-d29f7455f494\") " pod="openshift-monitoring/alertmanager-main-0" Jul 05 13:20:13.442660 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:13.442639 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/aab13c90-c88f-42ce-83df-d29f7455f494-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"aab13c90-c88f-42ce-83df-d29f7455f494\") " pod="openshift-monitoring/alertmanager-main-0" Jul 05 13:20:13.442752 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:13.442687 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/aab13c90-c88f-42ce-83df-d29f7455f494-web-config\") pod \"alertmanager-main-0\" (UID: \"aab13c90-c88f-42ce-83df-d29f7455f494\") " pod="openshift-monitoring/alertmanager-main-0" Jul 05 13:20:13.442752 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:13.442726 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/aab13c90-c88f-42ce-83df-d29f7455f494-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"aab13c90-c88f-42ce-83df-d29f7455f494\") " pod="openshift-monitoring/alertmanager-main-0" Jul 05 13:20:13.442838 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:13.442758 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/aab13c90-c88f-42ce-83df-d29f7455f494-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"aab13c90-c88f-42ce-83df-d29f7455f494\") " pod="openshift-monitoring/alertmanager-main-0" Jul 05 13:20:13.544091 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:13.544049 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/aab13c90-c88f-42ce-83df-d29f7455f494-config-out\") pod \"alertmanager-main-0\" (UID: \"aab13c90-c88f-42ce-83df-d29f7455f494\") " pod="openshift-monitoring/alertmanager-main-0" Jul 05 13:20:13.544251 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:13.544107 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/aab13c90-c88f-42ce-83df-d29f7455f494-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"aab13c90-c88f-42ce-83df-d29f7455f494\") " pod="openshift-monitoring/alertmanager-main-0" Jul 05 13:20:13.544251 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:13.544131 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/aab13c90-c88f-42ce-83df-d29f7455f494-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"aab13c90-c88f-42ce-83df-d29f7455f494\") " pod="openshift-monitoring/alertmanager-main-0" Jul 05 13:20:13.544251 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:13.544166 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/aab13c90-c88f-42ce-83df-d29f7455f494-web-config\") pod \"alertmanager-main-0\" (UID: \"aab13c90-c88f-42ce-83df-d29f7455f494\") " pod="openshift-monitoring/alertmanager-main-0" Jul 05 13:20:13.544428 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:13.544295 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/aab13c90-c88f-42ce-83df-d29f7455f494-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"aab13c90-c88f-42ce-83df-d29f7455f494\") " pod="openshift-monitoring/alertmanager-main-0" Jul 05 13:20:13.544428 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:13.544346 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/aab13c90-c88f-42ce-83df-d29f7455f494-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"aab13c90-c88f-42ce-83df-d29f7455f494\") " pod="openshift-monitoring/alertmanager-main-0" Jul 05 13:20:13.544428 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:13.544391 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/aab13c90-c88f-42ce-83df-d29f7455f494-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"aab13c90-c88f-42ce-83df-d29f7455f494\") " pod="openshift-monitoring/alertmanager-main-0" Jul 05 13:20:13.544428 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:13.544417 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x2cb8\" (UniqueName: \"kubernetes.io/projected/aab13c90-c88f-42ce-83df-d29f7455f494-kube-api-access-x2cb8\") pod \"alertmanager-main-0\" (UID: \"aab13c90-c88f-42ce-83df-d29f7455f494\") " pod="openshift-monitoring/alertmanager-main-0" Jul 05 13:20:13.544575 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:13.544480 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/aab13c90-c88f-42ce-83df-d29f7455f494-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"aab13c90-c88f-42ce-83df-d29f7455f494\") " pod="openshift-monitoring/alertmanager-main-0" Jul 05 13:20:13.544575 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:13.544506 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/aab13c90-c88f-42ce-83df-d29f7455f494-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"aab13c90-c88f-42ce-83df-d29f7455f494\") " pod="openshift-monitoring/alertmanager-main-0" Jul 05 13:20:13.544575 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:13.544556 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/aab13c90-c88f-42ce-83df-d29f7455f494-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"aab13c90-c88f-42ce-83df-d29f7455f494\") " pod="openshift-monitoring/alertmanager-main-0" Jul 05 13:20:13.544752 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:13.544631 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/aab13c90-c88f-42ce-83df-d29f7455f494-tls-assets\") pod \"alertmanager-main-0\" (UID: \"aab13c90-c88f-42ce-83df-d29f7455f494\") " pod="openshift-monitoring/alertmanager-main-0" Jul 05 13:20:13.544752 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:13.544648 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/aab13c90-c88f-42ce-83df-d29f7455f494-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"aab13c90-c88f-42ce-83df-d29f7455f494\") " pod="openshift-monitoring/alertmanager-main-0" Jul 05 13:20:13.544752 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:13.544664 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/aab13c90-c88f-42ce-83df-d29f7455f494-config-volume\") pod \"alertmanager-main-0\" (UID: \"aab13c90-c88f-42ce-83df-d29f7455f494\") " pod="openshift-monitoring/alertmanager-main-0" Jul 05 13:20:13.546002 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:13.545649 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/aab13c90-c88f-42ce-83df-d29f7455f494-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"aab13c90-c88f-42ce-83df-d29f7455f494\") " pod="openshift-monitoring/alertmanager-main-0" Jul 05 13:20:13.546138 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:13.546098 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/aab13c90-c88f-42ce-83df-d29f7455f494-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"aab13c90-c88f-42ce-83df-d29f7455f494\") " pod="openshift-monitoring/alertmanager-main-0" Jul 05 13:20:13.547499 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:13.547425 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/aab13c90-c88f-42ce-83df-d29f7455f494-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"aab13c90-c88f-42ce-83df-d29f7455f494\") " pod="openshift-monitoring/alertmanager-main-0" Jul 05 13:20:13.548353 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:13.548318 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/aab13c90-c88f-42ce-83df-d29f7455f494-config-volume\") pod \"alertmanager-main-0\" (UID: \"aab13c90-c88f-42ce-83df-d29f7455f494\") " pod="openshift-monitoring/alertmanager-main-0" Jul 05 13:20:13.548516 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:13.548493 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/aab13c90-c88f-42ce-83df-d29f7455f494-web-config\") pod \"alertmanager-main-0\" (UID: \"aab13c90-c88f-42ce-83df-d29f7455f494\") " pod="openshift-monitoring/alertmanager-main-0" Jul 05 13:20:13.548516 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:13.548500 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/aab13c90-c88f-42ce-83df-d29f7455f494-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"aab13c90-c88f-42ce-83df-d29f7455f494\") " pod="openshift-monitoring/alertmanager-main-0" Jul 05 13:20:13.548808 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:13.548789 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/aab13c90-c88f-42ce-83df-d29f7455f494-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"aab13c90-c88f-42ce-83df-d29f7455f494\") " pod="openshift-monitoring/alertmanager-main-0" Jul 05 13:20:13.548965 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:13.548949 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/aab13c90-c88f-42ce-83df-d29f7455f494-config-out\") pod \"alertmanager-main-0\" (UID: \"aab13c90-c88f-42ce-83df-d29f7455f494\") " pod="openshift-monitoring/alertmanager-main-0" Jul 05 13:20:13.549067 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:13.549041 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/aab13c90-c88f-42ce-83df-d29f7455f494-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"aab13c90-c88f-42ce-83df-d29f7455f494\") " pod="openshift-monitoring/alertmanager-main-0" Jul 05 13:20:13.549225 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:13.549209 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/aab13c90-c88f-42ce-83df-d29f7455f494-tls-assets\") pod \"alertmanager-main-0\" (UID: \"aab13c90-c88f-42ce-83df-d29f7455f494\") " pod="openshift-monitoring/alertmanager-main-0" Jul 05 13:20:13.549872 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:13.549851 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/aab13c90-c88f-42ce-83df-d29f7455f494-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"aab13c90-c88f-42ce-83df-d29f7455f494\") " pod="openshift-monitoring/alertmanager-main-0" Jul 05 13:20:13.584864 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:13.584826 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x2cb8\" (UniqueName: \"kubernetes.io/projected/aab13c90-c88f-42ce-83df-d29f7455f494-kube-api-access-x2cb8\") pod \"alertmanager-main-0\" (UID: \"aab13c90-c88f-42ce-83df-d29f7455f494\") " pod="openshift-monitoring/alertmanager-main-0" Jul 05 13:20:13.620522 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:13.620490 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 05 13:20:13.706609 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:13.706504 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 05 13:20:13.926816 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:13.926772 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-5f8697c67c-crgzp" Jul 05 13:20:13.926969 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:13.926852 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5f8697c67c-crgzp" Jul 05 13:20:13.928160 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:13.928132 2578 patch_prober.go:28] interesting pod/console-5f8697c67c-crgzp container/console namespace/openshift-console: Startup probe status=failure output="Get \"https://10.132.0.20:8443/health\": dial tcp 10.132.0.20:8443: connect: connection refused" start-of-body= Jul 05 13:20:13.928270 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:13.928181 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-console/console-5f8697c67c-crgzp" podUID="3644ed9d-8c13-448b-98ab-4017dab5ea27" containerName="console" probeResult="failure" output="Get \"https://10.132.0.20:8443/health\": dial tcp 10.132.0.20:8443: connect: connection refused" Jul 05 13:20:13.937943 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:13.937915 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 05 13:20:13.942604 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:20:13.942561 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaab13c90_c88f_42ce_83df_d29f7455f494.slice/crio-0c7a2bfee9ef67b79b56999948b8f540d26ca30b4c731b0b6ff351536a3e2dea WatchSource:0}: Error finding container 0c7a2bfee9ef67b79b56999948b8f540d26ca30b4c731b0b6ff351536a3e2dea: Status 404 returned error can't find the container with id 0c7a2bfee9ef67b79b56999948b8f540d26ca30b4c731b0b6ff351536a3e2dea Jul 05 13:20:14.020270 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:14.020222 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-kghbq" event={"ID":"c182307f-a4e2-4f81-97bc-fb7835a2a2ba","Type":"ContainerStarted","Data":"53a4c983ba799d274d77fa164fba6842fc86b644b8ff8fc19e143b8302cf6ab7"} Jul 05 13:20:14.020726 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:14.020303 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-kghbq" event={"ID":"c182307f-a4e2-4f81-97bc-fb7835a2a2ba","Type":"ContainerStarted","Data":"8627c08dfc106a2935cd6f7121d6bdf8e9a4e238a1912340ef566007f6c575ad"} Jul 05 13:20:14.021506 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:14.021484 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"aab13c90-c88f-42ce-83df-d29f7455f494","Type":"ContainerStarted","Data":"0c7a2bfee9ef67b79b56999948b8f540d26ca30b4c731b0b6ff351536a3e2dea"} Jul 05 13:20:14.076208 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:14.076152 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-kghbq" podStartSLOduration=2.186282997 podStartE2EDuration="3.076136234s" totalImagesPullingTime="817.03758ms" totalInitContainerRuntime="72.815657ms" isStatefulPod=true podCreationTimestamp="2026-07-05 13:20:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 13:20:14.071413211 +0000 UTC m=+298.529976329" watchObservedRunningTime="2026-07-05 13:20:14.076136234 +0000 UTC m=+298.534699338" Jul 05 13:20:15.902417 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:15.902389 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-tg2xx_1c214417-9bef-4007-ab01-e61b84d1b35e/console-operator/1.log" Jul 05 13:20:15.902864 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:15.902389 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-tg2xx_1c214417-9bef-4007-ab01-e61b84d1b35e/console-operator/1.log" Jul 05 13:20:15.912735 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:15.912710 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-656m6_7bdbd204-0fb7-4ebf-af05-432c370a814b/ovn-acl-logging/0.log" Jul 05 13:20:15.913055 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:15.913032 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-656m6_7bdbd204-0fb7-4ebf-af05-432c370a814b/ovn-acl-logging/0.log" Jul 05 13:20:15.917496 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:15.917475 2578 kubelet.go:1648] "Image garbage collection succeeded" Jul 05 13:20:16.032101 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:16.031965 2578 generic.go:356] "Generic (PLEG): container finished" podID="aab13c90-c88f-42ce-83df-d29f7455f494" containerID="98dbc87ce520e3f234985438447150204b58f75df3310ed3b3b036fe14a81486" exitCode=0 Jul 05 13:20:16.032101 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:16.032001 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"aab13c90-c88f-42ce-83df-d29f7455f494","Type":"ContainerDied","Data":"98dbc87ce520e3f234985438447150204b58f75df3310ed3b3b036fe14a81486"} Jul 05 13:20:16.055204 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:16.049245 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-76df58b848-5cbkg"] Jul 05 13:20:16.127175 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:16.127135 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-74cf497c56-t99zs"] Jul 05 13:20:16.130290 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:16.130271 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-74cf497c56-t99zs" Jul 05 13:20:16.156639 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:16.156540 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-74cf497c56-t99zs"] Jul 05 13:20:16.176278 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:16.176243 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/943ce8b6-c20c-4d0a-8c56-29a8aa714643-trusted-ca-bundle\") pod \"console-74cf497c56-t99zs\" (UID: \"943ce8b6-c20c-4d0a-8c56-29a8aa714643\") " pod="openshift-console/console-74cf497c56-t99zs" Jul 05 13:20:16.176456 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:16.176300 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/943ce8b6-c20c-4d0a-8c56-29a8aa714643-console-oauth-config\") pod \"console-74cf497c56-t99zs\" (UID: \"943ce8b6-c20c-4d0a-8c56-29a8aa714643\") " pod="openshift-console/console-74cf497c56-t99zs" Jul 05 13:20:16.176456 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:16.176353 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/943ce8b6-c20c-4d0a-8c56-29a8aa714643-service-ca\") pod \"console-74cf497c56-t99zs\" (UID: \"943ce8b6-c20c-4d0a-8c56-29a8aa714643\") " pod="openshift-console/console-74cf497c56-t99zs" Jul 05 13:20:16.176456 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:16.176399 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-phwkx\" (UniqueName: \"kubernetes.io/projected/943ce8b6-c20c-4d0a-8c56-29a8aa714643-kube-api-access-phwkx\") pod \"console-74cf497c56-t99zs\" (UID: \"943ce8b6-c20c-4d0a-8c56-29a8aa714643\") " pod="openshift-console/console-74cf497c56-t99zs" Jul 05 13:20:16.176456 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:16.176432 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/943ce8b6-c20c-4d0a-8c56-29a8aa714643-console-config\") pod \"console-74cf497c56-t99zs\" (UID: \"943ce8b6-c20c-4d0a-8c56-29a8aa714643\") " pod="openshift-console/console-74cf497c56-t99zs" Jul 05 13:20:16.176643 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:16.176492 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/943ce8b6-c20c-4d0a-8c56-29a8aa714643-console-serving-cert\") pod \"console-74cf497c56-t99zs\" (UID: \"943ce8b6-c20c-4d0a-8c56-29a8aa714643\") " pod="openshift-console/console-74cf497c56-t99zs" Jul 05 13:20:16.176643 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:16.176512 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/943ce8b6-c20c-4d0a-8c56-29a8aa714643-oauth-serving-cert\") pod \"console-74cf497c56-t99zs\" (UID: \"943ce8b6-c20c-4d0a-8c56-29a8aa714643\") " pod="openshift-console/console-74cf497c56-t99zs" Jul 05 13:20:16.277865 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:16.277826 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/943ce8b6-c20c-4d0a-8c56-29a8aa714643-trusted-ca-bundle\") pod \"console-74cf497c56-t99zs\" (UID: \"943ce8b6-c20c-4d0a-8c56-29a8aa714643\") " pod="openshift-console/console-74cf497c56-t99zs" Jul 05 13:20:16.278071 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:16.277914 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/943ce8b6-c20c-4d0a-8c56-29a8aa714643-console-oauth-config\") pod \"console-74cf497c56-t99zs\" (UID: \"943ce8b6-c20c-4d0a-8c56-29a8aa714643\") " pod="openshift-console/console-74cf497c56-t99zs" Jul 05 13:20:16.278071 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:16.277951 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/943ce8b6-c20c-4d0a-8c56-29a8aa714643-service-ca\") pod \"console-74cf497c56-t99zs\" (UID: \"943ce8b6-c20c-4d0a-8c56-29a8aa714643\") " pod="openshift-console/console-74cf497c56-t99zs" Jul 05 13:20:16.278071 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:16.277984 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-phwkx\" (UniqueName: \"kubernetes.io/projected/943ce8b6-c20c-4d0a-8c56-29a8aa714643-kube-api-access-phwkx\") pod \"console-74cf497c56-t99zs\" (UID: \"943ce8b6-c20c-4d0a-8c56-29a8aa714643\") " pod="openshift-console/console-74cf497c56-t99zs" Jul 05 13:20:16.278071 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:16.278021 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/943ce8b6-c20c-4d0a-8c56-29a8aa714643-console-config\") pod \"console-74cf497c56-t99zs\" (UID: \"943ce8b6-c20c-4d0a-8c56-29a8aa714643\") " pod="openshift-console/console-74cf497c56-t99zs" Jul 05 13:20:16.278071 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:16.278063 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/943ce8b6-c20c-4d0a-8c56-29a8aa714643-console-serving-cert\") pod \"console-74cf497c56-t99zs\" (UID: \"943ce8b6-c20c-4d0a-8c56-29a8aa714643\") " pod="openshift-console/console-74cf497c56-t99zs" Jul 05 13:20:16.278311 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:16.278086 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/943ce8b6-c20c-4d0a-8c56-29a8aa714643-oauth-serving-cert\") pod \"console-74cf497c56-t99zs\" (UID: \"943ce8b6-c20c-4d0a-8c56-29a8aa714643\") " pod="openshift-console/console-74cf497c56-t99zs" Jul 05 13:20:16.278818 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:16.278793 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/943ce8b6-c20c-4d0a-8c56-29a8aa714643-service-ca\") pod \"console-74cf497c56-t99zs\" (UID: \"943ce8b6-c20c-4d0a-8c56-29a8aa714643\") " pod="openshift-console/console-74cf497c56-t99zs" Jul 05 13:20:16.278934 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:16.278863 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/943ce8b6-c20c-4d0a-8c56-29a8aa714643-trusted-ca-bundle\") pod \"console-74cf497c56-t99zs\" (UID: \"943ce8b6-c20c-4d0a-8c56-29a8aa714643\") " pod="openshift-console/console-74cf497c56-t99zs" Jul 05 13:20:16.278934 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:16.278803 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/943ce8b6-c20c-4d0a-8c56-29a8aa714643-console-config\") pod \"console-74cf497c56-t99zs\" (UID: \"943ce8b6-c20c-4d0a-8c56-29a8aa714643\") " pod="openshift-console/console-74cf497c56-t99zs" Jul 05 13:20:16.279077 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:16.279054 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/943ce8b6-c20c-4d0a-8c56-29a8aa714643-oauth-serving-cert\") pod \"console-74cf497c56-t99zs\" (UID: \"943ce8b6-c20c-4d0a-8c56-29a8aa714643\") " pod="openshift-console/console-74cf497c56-t99zs" Jul 05 13:20:16.280614 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:16.280572 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/943ce8b6-c20c-4d0a-8c56-29a8aa714643-console-serving-cert\") pod \"console-74cf497c56-t99zs\" (UID: \"943ce8b6-c20c-4d0a-8c56-29a8aa714643\") " pod="openshift-console/console-74cf497c56-t99zs" Jul 05 13:20:16.280696 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:16.280673 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/943ce8b6-c20c-4d0a-8c56-29a8aa714643-console-oauth-config\") pod \"console-74cf497c56-t99zs\" (UID: \"943ce8b6-c20c-4d0a-8c56-29a8aa714643\") " pod="openshift-console/console-74cf497c56-t99zs" Jul 05 13:20:16.293862 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:16.293827 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-phwkx\" (UniqueName: \"kubernetes.io/projected/943ce8b6-c20c-4d0a-8c56-29a8aa714643-kube-api-access-phwkx\") pod \"console-74cf497c56-t99zs\" (UID: \"943ce8b6-c20c-4d0a-8c56-29a8aa714643\") " pod="openshift-console/console-74cf497c56-t99zs" Jul 05 13:20:16.439830 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:16.439746 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-74cf497c56-t99zs" Jul 05 13:20:16.635766 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:16.635731 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-74cf497c56-t99zs"] Jul 05 13:20:16.638690 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:20:16.638660 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod943ce8b6_c20c_4d0a_8c56_29a8aa714643.slice/crio-46de71c939d74962ecdd985921360036106a7268244e506aaea4142d7b197637 WatchSource:0}: Error finding container 46de71c939d74962ecdd985921360036106a7268244e506aaea4142d7b197637: Status 404 returned error can't find the container with id 46de71c939d74962ecdd985921360036106a7268244e506aaea4142d7b197637 Jul 05 13:20:16.992829 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:16.992751 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-w8bmv" Jul 05 13:20:17.038436 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:17.038397 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-74cf497c56-t99zs" event={"ID":"943ce8b6-c20c-4d0a-8c56-29a8aa714643","Type":"ContainerStarted","Data":"1ae92f2ca099200b2a990c5db72508e076e608cb6f1e1fa928481d33042af96c"} Jul 05 13:20:17.038670 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:17.038454 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-74cf497c56-t99zs" event={"ID":"943ce8b6-c20c-4d0a-8c56-29a8aa714643","Type":"ContainerStarted","Data":"46de71c939d74962ecdd985921360036106a7268244e506aaea4142d7b197637"} Jul 05 13:20:17.079271 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:17.079215 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-74cf497c56-t99zs" podStartSLOduration=1.079192114 podStartE2EDuration="1.079192114s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 13:20:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 13:20:17.078971067 +0000 UTC m=+301.537534197" watchObservedRunningTime="2026-07-05 13:20:17.079192114 +0000 UTC m=+301.537755221" Jul 05 13:20:17.971687 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:17.971663 2578 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 05 13:20:18.046163 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:18.046132 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"aab13c90-c88f-42ce-83df-d29f7455f494","Type":"ContainerStarted","Data":"5fc68338561ed75b256f01710e7f644946c1cfd924e68973497293ab1d770997"} Jul 05 13:20:18.046538 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:18.046198 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"aab13c90-c88f-42ce-83df-d29f7455f494","Type":"ContainerStarted","Data":"cd065603214df93760be671adf6fd2e1a3e6b1a4ea44cb42b67b45b9ab86ce1c"} Jul 05 13:20:18.046538 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:18.046225 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"aab13c90-c88f-42ce-83df-d29f7455f494","Type":"ContainerStarted","Data":"74f22890bc99dd19e25f16a7280f4399b3ce05fe371f15dc91366bae401aecd5"} Jul 05 13:20:18.046538 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:18.046253 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"aab13c90-c88f-42ce-83df-d29f7455f494","Type":"ContainerStarted","Data":"9276413d588f97b8b2cdb52c516c3ff63177291a1f4074f567d84bca2aca0e62"} Jul 05 13:20:18.046538 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:18.046277 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"aab13c90-c88f-42ce-83df-d29f7455f494","Type":"ContainerStarted","Data":"08aae034773cfa532c5bb6cee4b9b3a85b29ba87002feece5f2ddeaab3130849"} Jul 05 13:20:19.053411 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:19.053377 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"aab13c90-c88f-42ce-83df-d29f7455f494","Type":"ContainerStarted","Data":"5df4853441335824507dd3abb0d6aaceeb94a92d28da466ef5f25b503991bc0b"} Jul 05 13:20:19.101528 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:19.099948 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.753192633 podStartE2EDuration="6.099927957s" totalImagesPullingTime="3.329033886s" totalInitContainerRuntime="17.701438ms" isStatefulPod=false podCreationTimestamp="2026-07-05 13:20:13 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 13:20:19.096909665 +0000 UTC m=+303.555472765" watchObservedRunningTime="2026-07-05 13:20:19.099927957 +0000 UTC m=+303.558491063" Jul 05 13:20:20.003462 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:20.003423 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dxwj6" Jul 05 13:20:21.543283 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:21.543250 2578 patch_prober.go:28] interesting pod/image-registry-57b9f5fbdc-gsjvv container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jul 05 13:20:21.543674 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:21.543305 2578 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-57b9f5fbdc-gsjvv" podUID="d933ee11-a974-467b-9dce-f898bcf3863b" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 05 13:20:23.930864 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:23.930834 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-5f8697c67c-crgzp" Jul 05 13:20:23.934536 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:23.934503 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-5f8697c67c-crgzp" Jul 05 13:20:23.971054 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:23.971023 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-57b9f5fbdc-gsjvv" Jul 05 13:20:25.234730 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:25.234696 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-57b9f5fbdc-gsjvv"] Jul 05 13:20:26.187217 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:26.187156 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-74cf497c56-t99zs"] Jul 05 13:20:26.284635 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:26.284607 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-66955d844b-9pds8"] Jul 05 13:20:26.508283 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:26.508254 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-74cf497c56-t99zs" Jul 05 13:20:26.508283 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:26.508290 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-66955d844b-9pds8"] Jul 05 13:20:26.508481 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:26.508380 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-66955d844b-9pds8" Jul 05 13:20:26.577771 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:26.577738 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8435ac41-be8e-4ab7-8f3a-8f50f3b02416-console-oauth-config\") pod \"console-66955d844b-9pds8\" (UID: \"8435ac41-be8e-4ab7-8f3a-8f50f3b02416\") " pod="openshift-console/console-66955d844b-9pds8" Jul 05 13:20:26.577928 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:26.577781 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pjbvs\" (UniqueName: \"kubernetes.io/projected/8435ac41-be8e-4ab7-8f3a-8f50f3b02416-kube-api-access-pjbvs\") pod \"console-66955d844b-9pds8\" (UID: \"8435ac41-be8e-4ab7-8f3a-8f50f3b02416\") " pod="openshift-console/console-66955d844b-9pds8" Jul 05 13:20:26.577928 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:26.577807 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8435ac41-be8e-4ab7-8f3a-8f50f3b02416-console-serving-cert\") pod \"console-66955d844b-9pds8\" (UID: \"8435ac41-be8e-4ab7-8f3a-8f50f3b02416\") " pod="openshift-console/console-66955d844b-9pds8" Jul 05 13:20:26.577928 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:26.577852 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8435ac41-be8e-4ab7-8f3a-8f50f3b02416-service-ca\") pod \"console-66955d844b-9pds8\" (UID: \"8435ac41-be8e-4ab7-8f3a-8f50f3b02416\") " pod="openshift-console/console-66955d844b-9pds8" Jul 05 13:20:26.578086 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:26.577980 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8435ac41-be8e-4ab7-8f3a-8f50f3b02416-trusted-ca-bundle\") pod \"console-66955d844b-9pds8\" (UID: \"8435ac41-be8e-4ab7-8f3a-8f50f3b02416\") " pod="openshift-console/console-66955d844b-9pds8" Jul 05 13:20:26.578086 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:26.578014 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8435ac41-be8e-4ab7-8f3a-8f50f3b02416-oauth-serving-cert\") pod \"console-66955d844b-9pds8\" (UID: \"8435ac41-be8e-4ab7-8f3a-8f50f3b02416\") " pod="openshift-console/console-66955d844b-9pds8" Jul 05 13:20:26.578086 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:26.578046 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8435ac41-be8e-4ab7-8f3a-8f50f3b02416-console-config\") pod \"console-66955d844b-9pds8\" (UID: \"8435ac41-be8e-4ab7-8f3a-8f50f3b02416\") " pod="openshift-console/console-66955d844b-9pds8" Jul 05 13:20:26.679234 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:26.679193 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8435ac41-be8e-4ab7-8f3a-8f50f3b02416-trusted-ca-bundle\") pod \"console-66955d844b-9pds8\" (UID: \"8435ac41-be8e-4ab7-8f3a-8f50f3b02416\") " pod="openshift-console/console-66955d844b-9pds8" Jul 05 13:20:26.679234 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:26.679233 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8435ac41-be8e-4ab7-8f3a-8f50f3b02416-oauth-serving-cert\") pod \"console-66955d844b-9pds8\" (UID: \"8435ac41-be8e-4ab7-8f3a-8f50f3b02416\") " pod="openshift-console/console-66955d844b-9pds8" Jul 05 13:20:26.679497 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:26.679256 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8435ac41-be8e-4ab7-8f3a-8f50f3b02416-console-config\") pod \"console-66955d844b-9pds8\" (UID: \"8435ac41-be8e-4ab7-8f3a-8f50f3b02416\") " pod="openshift-console/console-66955d844b-9pds8" Jul 05 13:20:26.679497 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:26.679278 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8435ac41-be8e-4ab7-8f3a-8f50f3b02416-console-oauth-config\") pod \"console-66955d844b-9pds8\" (UID: \"8435ac41-be8e-4ab7-8f3a-8f50f3b02416\") " pod="openshift-console/console-66955d844b-9pds8" Jul 05 13:20:26.679497 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:26.679400 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pjbvs\" (UniqueName: \"kubernetes.io/projected/8435ac41-be8e-4ab7-8f3a-8f50f3b02416-kube-api-access-pjbvs\") pod \"console-66955d844b-9pds8\" (UID: \"8435ac41-be8e-4ab7-8f3a-8f50f3b02416\") " pod="openshift-console/console-66955d844b-9pds8" Jul 05 13:20:26.679497 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:26.679447 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8435ac41-be8e-4ab7-8f3a-8f50f3b02416-console-serving-cert\") pod \"console-66955d844b-9pds8\" (UID: \"8435ac41-be8e-4ab7-8f3a-8f50f3b02416\") " pod="openshift-console/console-66955d844b-9pds8" Jul 05 13:20:26.679497 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:26.679466 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8435ac41-be8e-4ab7-8f3a-8f50f3b02416-service-ca\") pod \"console-66955d844b-9pds8\" (UID: \"8435ac41-be8e-4ab7-8f3a-8f50f3b02416\") " pod="openshift-console/console-66955d844b-9pds8" Jul 05 13:20:26.680063 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:26.680037 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8435ac41-be8e-4ab7-8f3a-8f50f3b02416-oauth-serving-cert\") pod \"console-66955d844b-9pds8\" (UID: \"8435ac41-be8e-4ab7-8f3a-8f50f3b02416\") " pod="openshift-console/console-66955d844b-9pds8" Jul 05 13:20:26.680247 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:26.680228 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8435ac41-be8e-4ab7-8f3a-8f50f3b02416-console-config\") pod \"console-66955d844b-9pds8\" (UID: \"8435ac41-be8e-4ab7-8f3a-8f50f3b02416\") " pod="openshift-console/console-66955d844b-9pds8" Jul 05 13:20:26.680329 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:26.680308 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8435ac41-be8e-4ab7-8f3a-8f50f3b02416-trusted-ca-bundle\") pod \"console-66955d844b-9pds8\" (UID: \"8435ac41-be8e-4ab7-8f3a-8f50f3b02416\") " pod="openshift-console/console-66955d844b-9pds8" Jul 05 13:20:26.680366 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:26.680350 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8435ac41-be8e-4ab7-8f3a-8f50f3b02416-service-ca\") pod \"console-66955d844b-9pds8\" (UID: \"8435ac41-be8e-4ab7-8f3a-8f50f3b02416\") " pod="openshift-console/console-66955d844b-9pds8" Jul 05 13:20:26.681966 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:26.681935 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8435ac41-be8e-4ab7-8f3a-8f50f3b02416-console-oauth-config\") pod \"console-66955d844b-9pds8\" (UID: \"8435ac41-be8e-4ab7-8f3a-8f50f3b02416\") " pod="openshift-console/console-66955d844b-9pds8" Jul 05 13:20:26.682065 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:26.682031 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8435ac41-be8e-4ab7-8f3a-8f50f3b02416-console-serving-cert\") pod \"console-66955d844b-9pds8\" (UID: \"8435ac41-be8e-4ab7-8f3a-8f50f3b02416\") " pod="openshift-console/console-66955d844b-9pds8" Jul 05 13:20:26.688743 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:26.688723 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pjbvs\" (UniqueName: \"kubernetes.io/projected/8435ac41-be8e-4ab7-8f3a-8f50f3b02416-kube-api-access-pjbvs\") pod \"console-66955d844b-9pds8\" (UID: \"8435ac41-be8e-4ab7-8f3a-8f50f3b02416\") " pod="openshift-console/console-66955d844b-9pds8" Jul 05 13:20:26.818300 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:26.818216 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-66955d844b-9pds8" Jul 05 13:20:26.974893 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:26.974767 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-66955d844b-9pds8"] Jul 05 13:20:26.977306 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:20:26.977280 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8435ac41_be8e_4ab7_8f3a_8f50f3b02416.slice/crio-6c1ded61734e038cbf904fb03228d3940b23daa4fe0ccca37a577b26b2966b95 WatchSource:0}: Error finding container 6c1ded61734e038cbf904fb03228d3940b23daa4fe0ccca37a577b26b2966b95: Status 404 returned error can't find the container with id 6c1ded61734e038cbf904fb03228d3940b23daa4fe0ccca37a577b26b2966b95 Jul 05 13:20:27.091630 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:27.091597 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-66955d844b-9pds8" event={"ID":"8435ac41-be8e-4ab7-8f3a-8f50f3b02416","Type":"ContainerStarted","Data":"3bf0f0b8df333e8ea7c4a7275680ab1bd40c8197c1aabbfa9045a3493fcc3706"} Jul 05 13:20:27.091779 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:27.091667 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-66955d844b-9pds8" event={"ID":"8435ac41-be8e-4ab7-8f3a-8f50f3b02416","Type":"ContainerStarted","Data":"6c1ded61734e038cbf904fb03228d3940b23daa4fe0ccca37a577b26b2966b95"} Jul 05 13:20:28.116537 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:28.116483 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-66955d844b-9pds8" podStartSLOduration=2.1164685 podStartE2EDuration="2.1164685s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 13:20:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 13:20:28.114084226 +0000 UTC m=+312.572647332" watchObservedRunningTime="2026-07-05 13:20:28.1164685 +0000 UTC m=+312.575031661" Jul 05 13:20:36.818599 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:36.818554 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-66955d844b-9pds8" Jul 05 13:20:36.818968 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:36.818625 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-66955d844b-9pds8" Jul 05 13:20:36.823113 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:36.823083 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-66955d844b-9pds8" Jul 05 13:20:37.134502 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:37.134426 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-66955d844b-9pds8" Jul 05 13:20:37.183300 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:37.183265 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5f8697c67c-crgzp"] Jul 05 13:20:41.070996 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:41.070947 2578 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-76df58b848-5cbkg" podUID="b68e67e7-b24f-4823-93ce-b1a21d9997df" containerName="console" containerID="cri-o://667ab25291abd91a77282af694d4156f144d94c81ebdd67d9268376efda3624e" gracePeriod=15 Jul 05 13:20:41.147437 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:41.147406 2578 generic.go:356] "Generic (PLEG): container finished" podID="beb3b580-1920-42d3-a152-ac42772476cf" containerID="dcfa576596b24838b8aacb55877ddab396f9d3dd87f776c88a491e3d3d249b0f" exitCode=0 Jul 05 13:20:41.147607 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:41.147455 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5c847dcfc4-7pd6v" event={"ID":"beb3b580-1920-42d3-a152-ac42772476cf","Type":"ContainerDied","Data":"dcfa576596b24838b8aacb55877ddab396f9d3dd87f776c88a491e3d3d249b0f"} Jul 05 13:20:41.147825 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:41.147810 2578 scope.go:117] "RemoveContainer" containerID="dcfa576596b24838b8aacb55877ddab396f9d3dd87f776c88a491e3d3d249b0f" Jul 05 13:20:41.329794 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:41.329769 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-76df58b848-5cbkg" Jul 05 13:20:41.510228 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:41.510199 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b68e67e7-b24f-4823-93ce-b1a21d9997df-oauth-serving-cert\") pod \"b68e67e7-b24f-4823-93ce-b1a21d9997df\" (UID: \"b68e67e7-b24f-4823-93ce-b1a21d9997df\") " Jul 05 13:20:41.510406 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:41.510241 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b68e67e7-b24f-4823-93ce-b1a21d9997df-console-config\") pod \"b68e67e7-b24f-4823-93ce-b1a21d9997df\" (UID: \"b68e67e7-b24f-4823-93ce-b1a21d9997df\") " Jul 05 13:20:41.510406 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:41.510288 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b68e67e7-b24f-4823-93ce-b1a21d9997df-console-oauth-config\") pod \"b68e67e7-b24f-4823-93ce-b1a21d9997df\" (UID: \"b68e67e7-b24f-4823-93ce-b1a21d9997df\") " Jul 05 13:20:41.510406 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:41.510336 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b68e67e7-b24f-4823-93ce-b1a21d9997df-service-ca\") pod \"b68e67e7-b24f-4823-93ce-b1a21d9997df\" (UID: \"b68e67e7-b24f-4823-93ce-b1a21d9997df\") " Jul 05 13:20:41.510406 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:41.510366 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b68e67e7-b24f-4823-93ce-b1a21d9997df-console-serving-cert\") pod \"b68e67e7-b24f-4823-93ce-b1a21d9997df\" (UID: \"b68e67e7-b24f-4823-93ce-b1a21d9997df\") " Jul 05 13:20:41.510633 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:41.510440 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jwd82\" (UniqueName: \"kubernetes.io/projected/b68e67e7-b24f-4823-93ce-b1a21d9997df-kube-api-access-jwd82\") pod \"b68e67e7-b24f-4823-93ce-b1a21d9997df\" (UID: \"b68e67e7-b24f-4823-93ce-b1a21d9997df\") " Jul 05 13:20:41.510736 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:41.510713 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b68e67e7-b24f-4823-93ce-b1a21d9997df-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "b68e67e7-b24f-4823-93ce-b1a21d9997df" (UID: "b68e67e7-b24f-4823-93ce-b1a21d9997df"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 05 13:20:41.510798 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:41.510754 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b68e67e7-b24f-4823-93ce-b1a21d9997df-console-config" (OuterVolumeSpecName: "console-config") pod "b68e67e7-b24f-4823-93ce-b1a21d9997df" (UID: "b68e67e7-b24f-4823-93ce-b1a21d9997df"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 05 13:20:41.510852 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:41.510827 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b68e67e7-b24f-4823-93ce-b1a21d9997df-service-ca" (OuterVolumeSpecName: "service-ca") pod "b68e67e7-b24f-4823-93ce-b1a21d9997df" (UID: "b68e67e7-b24f-4823-93ce-b1a21d9997df"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 05 13:20:41.512882 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:41.512857 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b68e67e7-b24f-4823-93ce-b1a21d9997df-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "b68e67e7-b24f-4823-93ce-b1a21d9997df" (UID: "b68e67e7-b24f-4823-93ce-b1a21d9997df"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 05 13:20:41.512882 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:41.512874 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b68e67e7-b24f-4823-93ce-b1a21d9997df-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "b68e67e7-b24f-4823-93ce-b1a21d9997df" (UID: "b68e67e7-b24f-4823-93ce-b1a21d9997df"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 05 13:20:41.513013 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:41.512905 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b68e67e7-b24f-4823-93ce-b1a21d9997df-kube-api-access-jwd82" (OuterVolumeSpecName: "kube-api-access-jwd82") pod "b68e67e7-b24f-4823-93ce-b1a21d9997df" (UID: "b68e67e7-b24f-4823-93ce-b1a21d9997df"). InnerVolumeSpecName "kube-api-access-jwd82". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 05 13:20:41.611745 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:41.611711 2578 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b68e67e7-b24f-4823-93ce-b1a21d9997df-console-oauth-config\") on node \"ip-10-0-130-191.ec2.internal\" DevicePath \"\"" Jul 05 13:20:41.611745 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:41.611740 2578 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b68e67e7-b24f-4823-93ce-b1a21d9997df-service-ca\") on node \"ip-10-0-130-191.ec2.internal\" DevicePath \"\"" Jul 05 13:20:41.611948 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:41.611753 2578 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b68e67e7-b24f-4823-93ce-b1a21d9997df-console-serving-cert\") on node \"ip-10-0-130-191.ec2.internal\" DevicePath \"\"" Jul 05 13:20:41.611948 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:41.611766 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jwd82\" (UniqueName: \"kubernetes.io/projected/b68e67e7-b24f-4823-93ce-b1a21d9997df-kube-api-access-jwd82\") on node \"ip-10-0-130-191.ec2.internal\" DevicePath \"\"" Jul 05 13:20:41.611948 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:41.611779 2578 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b68e67e7-b24f-4823-93ce-b1a21d9997df-oauth-serving-cert\") on node \"ip-10-0-130-191.ec2.internal\" DevicePath \"\"" Jul 05 13:20:41.611948 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:41.611790 2578 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b68e67e7-b24f-4823-93ce-b1a21d9997df-console-config\") on node \"ip-10-0-130-191.ec2.internal\" DevicePath \"\"" Jul 05 13:20:42.152480 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:42.152390 2578 generic.go:356] "Generic (PLEG): container finished" podID="b68e67e7-b24f-4823-93ce-b1a21d9997df" containerID="667ab25291abd91a77282af694d4156f144d94c81ebdd67d9268376efda3624e" exitCode=0 Jul 05 13:20:42.152480 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:42.152445 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-76df58b848-5cbkg" Jul 05 13:20:42.152480 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:42.152464 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-76df58b848-5cbkg" event={"ID":"b68e67e7-b24f-4823-93ce-b1a21d9997df","Type":"ContainerDied","Data":"667ab25291abd91a77282af694d4156f144d94c81ebdd67d9268376efda3624e"} Jul 05 13:20:42.153020 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:42.152515 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-76df58b848-5cbkg" event={"ID":"b68e67e7-b24f-4823-93ce-b1a21d9997df","Type":"ContainerDied","Data":"15bad8e6611bd536809c17386cf9d27a64722a8c10dd92496befde3110adf46d"} Jul 05 13:20:42.153020 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:42.152541 2578 scope.go:117] "RemoveContainer" containerID="667ab25291abd91a77282af694d4156f144d94c81ebdd67d9268376efda3624e" Jul 05 13:20:42.154819 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:42.154733 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5c847dcfc4-7pd6v" event={"ID":"beb3b580-1920-42d3-a152-ac42772476cf","Type":"ContainerStarted","Data":"babd388ce3cf01a5658fbd970adbb3aad4b194706b04810bf8d5e444dd8d4130"} Jul 05 13:20:42.165757 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:42.165737 2578 scope.go:117] "RemoveContainer" containerID="667ab25291abd91a77282af694d4156f144d94c81ebdd67d9268376efda3624e" Jul 05 13:20:42.166097 ip-10-0-130-191 kubenswrapper[2578]: E0705 13:20:42.166077 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"667ab25291abd91a77282af694d4156f144d94c81ebdd67d9268376efda3624e\": container with ID starting with 667ab25291abd91a77282af694d4156f144d94c81ebdd67d9268376efda3624e not found: ID does not exist" containerID="667ab25291abd91a77282af694d4156f144d94c81ebdd67d9268376efda3624e" Jul 05 13:20:42.166150 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:42.166106 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"667ab25291abd91a77282af694d4156f144d94c81ebdd67d9268376efda3624e"} err="failed to get container status \"667ab25291abd91a77282af694d4156f144d94c81ebdd67d9268376efda3624e\": rpc error: code = NotFound desc = could not find container \"667ab25291abd91a77282af694d4156f144d94c81ebdd67d9268376efda3624e\": container with ID starting with 667ab25291abd91a77282af694d4156f144d94c81ebdd67d9268376efda3624e not found: ID does not exist" Jul 05 13:20:42.196804 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:42.196773 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-76df58b848-5cbkg"] Jul 05 13:20:42.206221 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:42.206192 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-76df58b848-5cbkg"] Jul 05 13:20:44.008977 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:44.008942 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b68e67e7-b24f-4823-93ce-b1a21d9997df" path="/var/lib/kubelet/pods/b68e67e7-b24f-4823-93ce-b1a21d9997df/volumes" Jul 05 13:20:50.254821 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:50.254754 2578 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-image-registry/image-registry-57b9f5fbdc-gsjvv" podUID="d933ee11-a974-467b-9dce-f898bcf3863b" containerName="registry" containerID="cri-o://998daa2e6543e4ae237d93b5321ed62f4acaf3155781d814ce61b7b32b888434" gracePeriod=30 Jul 05 13:20:50.494598 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:50.494556 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-57b9f5fbdc-gsjvv" Jul 05 13:20:50.595945 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:50.595848 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/d933ee11-a974-467b-9dce-f898bcf3863b-image-registry-private-configuration\") pod \"d933ee11-a974-467b-9dce-f898bcf3863b\" (UID: \"d933ee11-a974-467b-9dce-f898bcf3863b\") " Jul 05 13:20:50.596090 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:50.595961 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d933ee11-a974-467b-9dce-f898bcf3863b-bound-sa-token\") pod \"d933ee11-a974-467b-9dce-f898bcf3863b\" (UID: \"d933ee11-a974-467b-9dce-f898bcf3863b\") " Jul 05 13:20:50.596090 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:50.595992 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d933ee11-a974-467b-9dce-f898bcf3863b-registry-tls\") pod \"d933ee11-a974-467b-9dce-f898bcf3863b\" (UID: \"d933ee11-a974-467b-9dce-f898bcf3863b\") " Jul 05 13:20:50.596090 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:50.596025 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/d933ee11-a974-467b-9dce-f898bcf3863b-registry-certificates\") pod \"d933ee11-a974-467b-9dce-f898bcf3863b\" (UID: \"d933ee11-a974-467b-9dce-f898bcf3863b\") " Jul 05 13:20:50.596191 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:50.596170 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/d933ee11-a974-467b-9dce-f898bcf3863b-ca-trust-extracted\") pod \"d933ee11-a974-467b-9dce-f898bcf3863b\" (UID: \"d933ee11-a974-467b-9dce-f898bcf3863b\") " Jul 05 13:20:50.596240 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:50.596225 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/d933ee11-a974-467b-9dce-f898bcf3863b-installation-pull-secrets\") pod \"d933ee11-a974-467b-9dce-f898bcf3863b\" (UID: \"d933ee11-a974-467b-9dce-f898bcf3863b\") " Jul 05 13:20:50.596286 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:50.596264 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k2cfn\" (UniqueName: \"kubernetes.io/projected/d933ee11-a974-467b-9dce-f898bcf3863b-kube-api-access-k2cfn\") pod \"d933ee11-a974-467b-9dce-f898bcf3863b\" (UID: \"d933ee11-a974-467b-9dce-f898bcf3863b\") " Jul 05 13:20:50.596335 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:50.596315 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/d933ee11-a974-467b-9dce-f898bcf3863b-trusted-ca\") pod \"d933ee11-a974-467b-9dce-f898bcf3863b\" (UID: \"d933ee11-a974-467b-9dce-f898bcf3863b\") " Jul 05 13:20:50.596641 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:50.596472 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d933ee11-a974-467b-9dce-f898bcf3863b-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "d933ee11-a974-467b-9dce-f898bcf3863b" (UID: "d933ee11-a974-467b-9dce-f898bcf3863b"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 05 13:20:50.596839 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:50.596700 2578 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/d933ee11-a974-467b-9dce-f898bcf3863b-registry-certificates\") on node \"ip-10-0-130-191.ec2.internal\" DevicePath \"\"" Jul 05 13:20:50.597170 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:50.597137 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d933ee11-a974-467b-9dce-f898bcf3863b-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "d933ee11-a974-467b-9dce-f898bcf3863b" (UID: "d933ee11-a974-467b-9dce-f898bcf3863b"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 05 13:20:50.598494 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:50.598453 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d933ee11-a974-467b-9dce-f898bcf3863b-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "d933ee11-a974-467b-9dce-f898bcf3863b" (UID: "d933ee11-a974-467b-9dce-f898bcf3863b"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 05 13:20:50.598744 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:50.598526 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d933ee11-a974-467b-9dce-f898bcf3863b-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "d933ee11-a974-467b-9dce-f898bcf3863b" (UID: "d933ee11-a974-467b-9dce-f898bcf3863b"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 05 13:20:50.598824 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:50.598737 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d933ee11-a974-467b-9dce-f898bcf3863b-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "d933ee11-a974-467b-9dce-f898bcf3863b" (UID: "d933ee11-a974-467b-9dce-f898bcf3863b"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 05 13:20:50.599038 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:50.599007 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d933ee11-a974-467b-9dce-f898bcf3863b-kube-api-access-k2cfn" (OuterVolumeSpecName: "kube-api-access-k2cfn") pod "d933ee11-a974-467b-9dce-f898bcf3863b" (UID: "d933ee11-a974-467b-9dce-f898bcf3863b"). InnerVolumeSpecName "kube-api-access-k2cfn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 05 13:20:50.599152 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:50.599129 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d933ee11-a974-467b-9dce-f898bcf3863b-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "d933ee11-a974-467b-9dce-f898bcf3863b" (UID: "d933ee11-a974-467b-9dce-f898bcf3863b"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 05 13:20:50.605816 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:50.605791 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d933ee11-a974-467b-9dce-f898bcf3863b-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "d933ee11-a974-467b-9dce-f898bcf3863b" (UID: "d933ee11-a974-467b-9dce-f898bcf3863b"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 05 13:20:50.697558 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:50.697512 2578 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d933ee11-a974-467b-9dce-f898bcf3863b-bound-sa-token\") on node \"ip-10-0-130-191.ec2.internal\" DevicePath \"\"" Jul 05 13:20:50.697558 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:50.697552 2578 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d933ee11-a974-467b-9dce-f898bcf3863b-registry-tls\") on node \"ip-10-0-130-191.ec2.internal\" DevicePath \"\"" Jul 05 13:20:50.697558 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:50.697562 2578 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/d933ee11-a974-467b-9dce-f898bcf3863b-ca-trust-extracted\") on node \"ip-10-0-130-191.ec2.internal\" DevicePath \"\"" Jul 05 13:20:50.697558 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:50.697572 2578 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/d933ee11-a974-467b-9dce-f898bcf3863b-installation-pull-secrets\") on node \"ip-10-0-130-191.ec2.internal\" DevicePath \"\"" Jul 05 13:20:50.697813 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:50.697606 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k2cfn\" (UniqueName: \"kubernetes.io/projected/d933ee11-a974-467b-9dce-f898bcf3863b-kube-api-access-k2cfn\") on node \"ip-10-0-130-191.ec2.internal\" DevicePath \"\"" Jul 05 13:20:50.697813 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:50.697615 2578 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/d933ee11-a974-467b-9dce-f898bcf3863b-trusted-ca\") on node \"ip-10-0-130-191.ec2.internal\" DevicePath \"\"" Jul 05 13:20:50.697813 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:50.697624 2578 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/d933ee11-a974-467b-9dce-f898bcf3863b-image-registry-private-configuration\") on node \"ip-10-0-130-191.ec2.internal\" DevicePath \"\"" Jul 05 13:20:51.192395 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:51.192356 2578 generic.go:356] "Generic (PLEG): container finished" podID="d933ee11-a974-467b-9dce-f898bcf3863b" containerID="998daa2e6543e4ae237d93b5321ed62f4acaf3155781d814ce61b7b32b888434" exitCode=0 Jul 05 13:20:51.192622 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:51.192403 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-57b9f5fbdc-gsjvv" Jul 05 13:20:51.192622 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:51.192443 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-57b9f5fbdc-gsjvv" event={"ID":"d933ee11-a974-467b-9dce-f898bcf3863b","Type":"ContainerDied","Data":"998daa2e6543e4ae237d93b5321ed62f4acaf3155781d814ce61b7b32b888434"} Jul 05 13:20:51.192622 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:51.192503 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-57b9f5fbdc-gsjvv" event={"ID":"d933ee11-a974-467b-9dce-f898bcf3863b","Type":"ContainerDied","Data":"7b3b180697c84ddc43330d235a858532f61011454ffdf3f69c2aa53cdbf8e1ef"} Jul 05 13:20:51.192622 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:51.192531 2578 scope.go:117] "RemoveContainer" containerID="998daa2e6543e4ae237d93b5321ed62f4acaf3155781d814ce61b7b32b888434" Jul 05 13:20:51.201304 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:51.201285 2578 scope.go:117] "RemoveContainer" containerID="998daa2e6543e4ae237d93b5321ed62f4acaf3155781d814ce61b7b32b888434" Jul 05 13:20:51.201661 ip-10-0-130-191 kubenswrapper[2578]: E0705 13:20:51.201638 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"998daa2e6543e4ae237d93b5321ed62f4acaf3155781d814ce61b7b32b888434\": container with ID starting with 998daa2e6543e4ae237d93b5321ed62f4acaf3155781d814ce61b7b32b888434 not found: ID does not exist" containerID="998daa2e6543e4ae237d93b5321ed62f4acaf3155781d814ce61b7b32b888434" Jul 05 13:20:51.201728 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:51.201671 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"998daa2e6543e4ae237d93b5321ed62f4acaf3155781d814ce61b7b32b888434"} err="failed to get container status \"998daa2e6543e4ae237d93b5321ed62f4acaf3155781d814ce61b7b32b888434\": rpc error: code = NotFound desc = could not find container \"998daa2e6543e4ae237d93b5321ed62f4acaf3155781d814ce61b7b32b888434\": container with ID starting with 998daa2e6543e4ae237d93b5321ed62f4acaf3155781d814ce61b7b32b888434 not found: ID does not exist" Jul 05 13:20:51.207951 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:51.207905 2578 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-74cf497c56-t99zs" podUID="943ce8b6-c20c-4d0a-8c56-29a8aa714643" containerName="console" containerID="cri-o://1ae92f2ca099200b2a990c5db72508e076e608cb6f1e1fa928481d33042af96c" gracePeriod=15 Jul 05 13:20:51.214539 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:51.214513 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-57b9f5fbdc-gsjvv"] Jul 05 13:20:51.220484 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:51.220463 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-57b9f5fbdc-gsjvv"] Jul 05 13:20:51.454647 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:51.454571 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-74cf497c56-t99zs" Jul 05 13:20:51.504030 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:51.504001 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/943ce8b6-c20c-4d0a-8c56-29a8aa714643-console-serving-cert\") pod \"943ce8b6-c20c-4d0a-8c56-29a8aa714643\" (UID: \"943ce8b6-c20c-4d0a-8c56-29a8aa714643\") " Jul 05 13:20:51.504030 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:51.504032 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/943ce8b6-c20c-4d0a-8c56-29a8aa714643-oauth-serving-cert\") pod \"943ce8b6-c20c-4d0a-8c56-29a8aa714643\" (UID: \"943ce8b6-c20c-4d0a-8c56-29a8aa714643\") " Jul 05 13:20:51.504258 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:51.504056 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-phwkx\" (UniqueName: \"kubernetes.io/projected/943ce8b6-c20c-4d0a-8c56-29a8aa714643-kube-api-access-phwkx\") pod \"943ce8b6-c20c-4d0a-8c56-29a8aa714643\" (UID: \"943ce8b6-c20c-4d0a-8c56-29a8aa714643\") " Jul 05 13:20:51.504258 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:51.504099 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/943ce8b6-c20c-4d0a-8c56-29a8aa714643-console-oauth-config\") pod \"943ce8b6-c20c-4d0a-8c56-29a8aa714643\" (UID: \"943ce8b6-c20c-4d0a-8c56-29a8aa714643\") " Jul 05 13:20:51.504258 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:51.504135 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/943ce8b6-c20c-4d0a-8c56-29a8aa714643-trusted-ca-bundle\") pod \"943ce8b6-c20c-4d0a-8c56-29a8aa714643\" (UID: \"943ce8b6-c20c-4d0a-8c56-29a8aa714643\") " Jul 05 13:20:51.504258 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:51.504182 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/943ce8b6-c20c-4d0a-8c56-29a8aa714643-service-ca\") pod \"943ce8b6-c20c-4d0a-8c56-29a8aa714643\" (UID: \"943ce8b6-c20c-4d0a-8c56-29a8aa714643\") " Jul 05 13:20:51.504258 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:51.504204 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/943ce8b6-c20c-4d0a-8c56-29a8aa714643-console-config\") pod \"943ce8b6-c20c-4d0a-8c56-29a8aa714643\" (UID: \"943ce8b6-c20c-4d0a-8c56-29a8aa714643\") " Jul 05 13:20:51.504497 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:51.504451 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/943ce8b6-c20c-4d0a-8c56-29a8aa714643-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "943ce8b6-c20c-4d0a-8c56-29a8aa714643" (UID: "943ce8b6-c20c-4d0a-8c56-29a8aa714643"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 05 13:20:51.504695 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:51.504563 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/943ce8b6-c20c-4d0a-8c56-29a8aa714643-service-ca" (OuterVolumeSpecName: "service-ca") pod "943ce8b6-c20c-4d0a-8c56-29a8aa714643" (UID: "943ce8b6-c20c-4d0a-8c56-29a8aa714643"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 05 13:20:51.504695 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:51.504680 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/943ce8b6-c20c-4d0a-8c56-29a8aa714643-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "943ce8b6-c20c-4d0a-8c56-29a8aa714643" (UID: "943ce8b6-c20c-4d0a-8c56-29a8aa714643"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 05 13:20:51.504840 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:51.504767 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/943ce8b6-c20c-4d0a-8c56-29a8aa714643-console-config" (OuterVolumeSpecName: "console-config") pod "943ce8b6-c20c-4d0a-8c56-29a8aa714643" (UID: "943ce8b6-c20c-4d0a-8c56-29a8aa714643"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 05 13:20:51.506339 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:51.506316 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/943ce8b6-c20c-4d0a-8c56-29a8aa714643-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "943ce8b6-c20c-4d0a-8c56-29a8aa714643" (UID: "943ce8b6-c20c-4d0a-8c56-29a8aa714643"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 05 13:20:51.506460 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:51.506412 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/943ce8b6-c20c-4d0a-8c56-29a8aa714643-kube-api-access-phwkx" (OuterVolumeSpecName: "kube-api-access-phwkx") pod "943ce8b6-c20c-4d0a-8c56-29a8aa714643" (UID: "943ce8b6-c20c-4d0a-8c56-29a8aa714643"). InnerVolumeSpecName "kube-api-access-phwkx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 05 13:20:51.506460 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:51.506418 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/943ce8b6-c20c-4d0a-8c56-29a8aa714643-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "943ce8b6-c20c-4d0a-8c56-29a8aa714643" (UID: "943ce8b6-c20c-4d0a-8c56-29a8aa714643"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 05 13:20:51.605463 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:51.605430 2578 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/943ce8b6-c20c-4d0a-8c56-29a8aa714643-console-oauth-config\") on node \"ip-10-0-130-191.ec2.internal\" DevicePath \"\"" Jul 05 13:20:51.605463 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:51.605458 2578 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/943ce8b6-c20c-4d0a-8c56-29a8aa714643-trusted-ca-bundle\") on node \"ip-10-0-130-191.ec2.internal\" DevicePath \"\"" Jul 05 13:20:51.605463 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:51.605467 2578 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/943ce8b6-c20c-4d0a-8c56-29a8aa714643-service-ca\") on node \"ip-10-0-130-191.ec2.internal\" DevicePath \"\"" Jul 05 13:20:51.605716 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:51.605476 2578 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/943ce8b6-c20c-4d0a-8c56-29a8aa714643-console-config\") on node \"ip-10-0-130-191.ec2.internal\" DevicePath \"\"" Jul 05 13:20:51.605716 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:51.605484 2578 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/943ce8b6-c20c-4d0a-8c56-29a8aa714643-console-serving-cert\") on node \"ip-10-0-130-191.ec2.internal\" DevicePath \"\"" Jul 05 13:20:51.605716 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:51.605492 2578 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/943ce8b6-c20c-4d0a-8c56-29a8aa714643-oauth-serving-cert\") on node \"ip-10-0-130-191.ec2.internal\" DevicePath \"\"" Jul 05 13:20:51.605716 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:51.605500 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-phwkx\" (UniqueName: \"kubernetes.io/projected/943ce8b6-c20c-4d0a-8c56-29a8aa714643-kube-api-access-phwkx\") on node \"ip-10-0-130-191.ec2.internal\" DevicePath \"\"" Jul 05 13:20:52.007823 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:52.007786 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d933ee11-a974-467b-9dce-f898bcf3863b" path="/var/lib/kubelet/pods/d933ee11-a974-467b-9dce-f898bcf3863b/volumes" Jul 05 13:20:52.198465 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:52.198414 2578 generic.go:356] "Generic (PLEG): container finished" podID="943ce8b6-c20c-4d0a-8c56-29a8aa714643" containerID="1ae92f2ca099200b2a990c5db72508e076e608cb6f1e1fa928481d33042af96c" exitCode=0 Jul 05 13:20:52.198632 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:52.198541 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-74cf497c56-t99zs" Jul 05 13:20:52.198632 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:52.198553 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-74cf497c56-t99zs" event={"ID":"943ce8b6-c20c-4d0a-8c56-29a8aa714643","Type":"ContainerDied","Data":"1ae92f2ca099200b2a990c5db72508e076e608cb6f1e1fa928481d33042af96c"} Jul 05 13:20:52.198632 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:52.198625 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-74cf497c56-t99zs" event={"ID":"943ce8b6-c20c-4d0a-8c56-29a8aa714643","Type":"ContainerDied","Data":"46de71c939d74962ecdd985921360036106a7268244e506aaea4142d7b197637"} Jul 05 13:20:52.198798 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:52.198646 2578 scope.go:117] "RemoveContainer" containerID="1ae92f2ca099200b2a990c5db72508e076e608cb6f1e1fa928481d33042af96c" Jul 05 13:20:52.200499 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:52.200469 2578 generic.go:356] "Generic (PLEG): container finished" podID="69911263-1cea-4a74-a4ff-1cc94312366b" containerID="c851e73ede4f94b99a8a0856f2828fab98eb5d441764a62da24a0ab3a41a172a" exitCode=0 Jul 05 13:20:52.200633 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:52.200504 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-jzwj6" event={"ID":"69911263-1cea-4a74-a4ff-1cc94312366b","Type":"ContainerDied","Data":"c851e73ede4f94b99a8a0856f2828fab98eb5d441764a62da24a0ab3a41a172a"} Jul 05 13:20:52.200925 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:52.200887 2578 scope.go:117] "RemoveContainer" containerID="c851e73ede4f94b99a8a0856f2828fab98eb5d441764a62da24a0ab3a41a172a" Jul 05 13:20:52.208152 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:52.208131 2578 scope.go:117] "RemoveContainer" containerID="1ae92f2ca099200b2a990c5db72508e076e608cb6f1e1fa928481d33042af96c" Jul 05 13:20:52.208467 ip-10-0-130-191 kubenswrapper[2578]: E0705 13:20:52.208441 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1ae92f2ca099200b2a990c5db72508e076e608cb6f1e1fa928481d33042af96c\": container with ID starting with 1ae92f2ca099200b2a990c5db72508e076e608cb6f1e1fa928481d33042af96c not found: ID does not exist" containerID="1ae92f2ca099200b2a990c5db72508e076e608cb6f1e1fa928481d33042af96c" Jul 05 13:20:52.208549 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:52.208475 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1ae92f2ca099200b2a990c5db72508e076e608cb6f1e1fa928481d33042af96c"} err="failed to get container status \"1ae92f2ca099200b2a990c5db72508e076e608cb6f1e1fa928481d33042af96c\": rpc error: code = NotFound desc = could not find container \"1ae92f2ca099200b2a990c5db72508e076e608cb6f1e1fa928481d33042af96c\": container with ID starting with 1ae92f2ca099200b2a990c5db72508e076e608cb6f1e1fa928481d33042af96c not found: ID does not exist" Jul 05 13:20:52.217359 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:52.217340 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-74cf497c56-t99zs"] Jul 05 13:20:52.222123 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:52.222103 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-74cf497c56-t99zs"] Jul 05 13:20:53.208430 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:53.208396 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-jzwj6" event={"ID":"69911263-1cea-4a74-a4ff-1cc94312366b","Type":"ContainerStarted","Data":"51b374abcb895c4787d85e7f67ac87acb1957f06c40cf11824c5af633c4568d2"} Jul 05 13:20:54.009005 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:54.008974 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="943ce8b6-c20c-4d0a-8c56-29a8aa714643" path="/var/lib/kubelet/pods/943ce8b6-c20c-4d0a-8c56-29a8aa714643/volumes" Jul 05 13:20:56.227519 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:56.227440 2578 generic.go:356] "Generic (PLEG): container finished" podID="b7b1094c-0b91-4c45-aec8-2c0e2d8aed07" containerID="062e43acf6338d5a5034497387e37315825797286bc2c8f83737c8b34fa45506" exitCode=0 Jul 05 13:20:56.227905 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:56.227528 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-wgnw7" event={"ID":"b7b1094c-0b91-4c45-aec8-2c0e2d8aed07","Type":"ContainerDied","Data":"062e43acf6338d5a5034497387e37315825797286bc2c8f83737c8b34fa45506"} Jul 05 13:20:56.227905 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:56.227867 2578 scope.go:117] "RemoveContainer" containerID="062e43acf6338d5a5034497387e37315825797286bc2c8f83737c8b34fa45506" Jul 05 13:20:57.233705 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:20:57.233666 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-wgnw7" event={"ID":"b7b1094c-0b91-4c45-aec8-2c0e2d8aed07","Type":"ContainerStarted","Data":"e70d8b554b531379445b69bff0c234f06798449518715bcd245156e572008b84"} Jul 05 13:21:02.204877 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:21:02.204831 2578 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-5f8697c67c-crgzp" podUID="3644ed9d-8c13-448b-98ab-4017dab5ea27" containerName="console" containerID="cri-o://e0582c36d6621c6232af3416fcc858784b47ceb9f0556cfb0dea73177120ca2c" gracePeriod=15 Jul 05 13:21:02.452689 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:21:02.452661 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5f8697c67c-crgzp" Jul 05 13:21:02.503023 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:21:02.502988 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3644ed9d-8c13-448b-98ab-4017dab5ea27-console-oauth-config\") pod \"3644ed9d-8c13-448b-98ab-4017dab5ea27\" (UID: \"3644ed9d-8c13-448b-98ab-4017dab5ea27\") " Jul 05 13:21:02.503203 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:21:02.503049 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3644ed9d-8c13-448b-98ab-4017dab5ea27-console-serving-cert\") pod \"3644ed9d-8c13-448b-98ab-4017dab5ea27\" (UID: \"3644ed9d-8c13-448b-98ab-4017dab5ea27\") " Jul 05 13:21:02.503203 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:21:02.503075 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3644ed9d-8c13-448b-98ab-4017dab5ea27-oauth-serving-cert\") pod \"3644ed9d-8c13-448b-98ab-4017dab5ea27\" (UID: \"3644ed9d-8c13-448b-98ab-4017dab5ea27\") " Jul 05 13:21:02.503318 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:21:02.503264 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3644ed9d-8c13-448b-98ab-4017dab5ea27-service-ca\") pod \"3644ed9d-8c13-448b-98ab-4017dab5ea27\" (UID: \"3644ed9d-8c13-448b-98ab-4017dab5ea27\") " Jul 05 13:21:02.503375 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:21:02.503320 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3644ed9d-8c13-448b-98ab-4017dab5ea27-console-config\") pod \"3644ed9d-8c13-448b-98ab-4017dab5ea27\" (UID: \"3644ed9d-8c13-448b-98ab-4017dab5ea27\") " Jul 05 13:21:02.503427 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:21:02.503371 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8cm7x\" (UniqueName: \"kubernetes.io/projected/3644ed9d-8c13-448b-98ab-4017dab5ea27-kube-api-access-8cm7x\") pod \"3644ed9d-8c13-448b-98ab-4017dab5ea27\" (UID: \"3644ed9d-8c13-448b-98ab-4017dab5ea27\") " Jul 05 13:21:02.503427 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:21:02.503408 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3644ed9d-8c13-448b-98ab-4017dab5ea27-trusted-ca-bundle\") pod \"3644ed9d-8c13-448b-98ab-4017dab5ea27\" (UID: \"3644ed9d-8c13-448b-98ab-4017dab5ea27\") " Jul 05 13:21:02.503571 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:21:02.503550 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3644ed9d-8c13-448b-98ab-4017dab5ea27-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "3644ed9d-8c13-448b-98ab-4017dab5ea27" (UID: "3644ed9d-8c13-448b-98ab-4017dab5ea27"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 05 13:21:02.503666 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:21:02.503647 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3644ed9d-8c13-448b-98ab-4017dab5ea27-service-ca" (OuterVolumeSpecName: "service-ca") pod "3644ed9d-8c13-448b-98ab-4017dab5ea27" (UID: "3644ed9d-8c13-448b-98ab-4017dab5ea27"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 05 13:21:02.503718 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:21:02.503699 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3644ed9d-8c13-448b-98ab-4017dab5ea27-console-config" (OuterVolumeSpecName: "console-config") pod "3644ed9d-8c13-448b-98ab-4017dab5ea27" (UID: "3644ed9d-8c13-448b-98ab-4017dab5ea27"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 05 13:21:02.503768 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:21:02.503724 2578 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3644ed9d-8c13-448b-98ab-4017dab5ea27-oauth-serving-cert\") on node \"ip-10-0-130-191.ec2.internal\" DevicePath \"\"" Jul 05 13:21:02.503768 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:21:02.503745 2578 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3644ed9d-8c13-448b-98ab-4017dab5ea27-service-ca\") on node \"ip-10-0-130-191.ec2.internal\" DevicePath \"\"" Jul 05 13:21:02.503911 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:21:02.503892 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3644ed9d-8c13-448b-98ab-4017dab5ea27-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "3644ed9d-8c13-448b-98ab-4017dab5ea27" (UID: "3644ed9d-8c13-448b-98ab-4017dab5ea27"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 05 13:21:02.505363 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:21:02.505338 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3644ed9d-8c13-448b-98ab-4017dab5ea27-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "3644ed9d-8c13-448b-98ab-4017dab5ea27" (UID: "3644ed9d-8c13-448b-98ab-4017dab5ea27"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 05 13:21:02.505772 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:21:02.505755 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3644ed9d-8c13-448b-98ab-4017dab5ea27-kube-api-access-8cm7x" (OuterVolumeSpecName: "kube-api-access-8cm7x") pod "3644ed9d-8c13-448b-98ab-4017dab5ea27" (UID: "3644ed9d-8c13-448b-98ab-4017dab5ea27"). InnerVolumeSpecName "kube-api-access-8cm7x". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 05 13:21:02.505839 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:21:02.505818 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3644ed9d-8c13-448b-98ab-4017dab5ea27-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "3644ed9d-8c13-448b-98ab-4017dab5ea27" (UID: "3644ed9d-8c13-448b-98ab-4017dab5ea27"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 05 13:21:02.604392 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:21:02.604343 2578 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3644ed9d-8c13-448b-98ab-4017dab5ea27-trusted-ca-bundle\") on node \"ip-10-0-130-191.ec2.internal\" DevicePath \"\"" Jul 05 13:21:02.604392 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:21:02.604382 2578 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3644ed9d-8c13-448b-98ab-4017dab5ea27-console-oauth-config\") on node \"ip-10-0-130-191.ec2.internal\" DevicePath \"\"" Jul 05 13:21:02.604392 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:21:02.604395 2578 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3644ed9d-8c13-448b-98ab-4017dab5ea27-console-serving-cert\") on node \"ip-10-0-130-191.ec2.internal\" DevicePath \"\"" Jul 05 13:21:02.604663 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:21:02.604407 2578 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3644ed9d-8c13-448b-98ab-4017dab5ea27-console-config\") on node \"ip-10-0-130-191.ec2.internal\" DevicePath \"\"" Jul 05 13:21:02.604663 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:21:02.604419 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8cm7x\" (UniqueName: \"kubernetes.io/projected/3644ed9d-8c13-448b-98ab-4017dab5ea27-kube-api-access-8cm7x\") on node \"ip-10-0-130-191.ec2.internal\" DevicePath \"\"" Jul 05 13:21:03.258973 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:21:03.258940 2578 generic.go:356] "Generic (PLEG): container finished" podID="3644ed9d-8c13-448b-98ab-4017dab5ea27" containerID="e0582c36d6621c6232af3416fcc858784b47ceb9f0556cfb0dea73177120ca2c" exitCode=0 Jul 05 13:21:03.259368 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:21:03.258985 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5f8697c67c-crgzp" Jul 05 13:21:03.259368 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:21:03.258994 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5f8697c67c-crgzp" event={"ID":"3644ed9d-8c13-448b-98ab-4017dab5ea27","Type":"ContainerDied","Data":"e0582c36d6621c6232af3416fcc858784b47ceb9f0556cfb0dea73177120ca2c"} Jul 05 13:21:03.259368 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:21:03.259033 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5f8697c67c-crgzp" event={"ID":"3644ed9d-8c13-448b-98ab-4017dab5ea27","Type":"ContainerDied","Data":"2f430fdb91722562cc05904927090e83118dd16c2f747aa78570a5a880d9503e"} Jul 05 13:21:03.259368 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:21:03.259053 2578 scope.go:117] "RemoveContainer" containerID="e0582c36d6621c6232af3416fcc858784b47ceb9f0556cfb0dea73177120ca2c" Jul 05 13:21:03.268288 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:21:03.268269 2578 scope.go:117] "RemoveContainer" containerID="e0582c36d6621c6232af3416fcc858784b47ceb9f0556cfb0dea73177120ca2c" Jul 05 13:21:03.268559 ip-10-0-130-191 kubenswrapper[2578]: E0705 13:21:03.268541 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e0582c36d6621c6232af3416fcc858784b47ceb9f0556cfb0dea73177120ca2c\": container with ID starting with e0582c36d6621c6232af3416fcc858784b47ceb9f0556cfb0dea73177120ca2c not found: ID does not exist" containerID="e0582c36d6621c6232af3416fcc858784b47ceb9f0556cfb0dea73177120ca2c" Jul 05 13:21:03.268682 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:21:03.268570 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e0582c36d6621c6232af3416fcc858784b47ceb9f0556cfb0dea73177120ca2c"} err="failed to get container status \"e0582c36d6621c6232af3416fcc858784b47ceb9f0556cfb0dea73177120ca2c\": rpc error: code = NotFound desc = could not find container \"e0582c36d6621c6232af3416fcc858784b47ceb9f0556cfb0dea73177120ca2c\": container with ID starting with e0582c36d6621c6232af3416fcc858784b47ceb9f0556cfb0dea73177120ca2c not found: ID does not exist" Jul 05 13:21:03.280568 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:21:03.280535 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5f8697c67c-crgzp"] Jul 05 13:21:03.284366 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:21:03.284344 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-5f8697c67c-crgzp"] Jul 05 13:21:04.008254 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:21:04.008219 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3644ed9d-8c13-448b-98ab-4017dab5ea27" path="/var/lib/kubelet/pods/3644ed9d-8c13-448b-98ab-4017dab5ea27/volumes" Jul 05 13:21:29.693214 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:21:29.693177 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-75cb8f6bcc-dbrfg"] Jul 05 13:21:29.693671 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:21:29.693471 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d933ee11-a974-467b-9dce-f898bcf3863b" containerName="registry" Jul 05 13:21:29.693671 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:21:29.693484 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="d933ee11-a974-467b-9dce-f898bcf3863b" containerName="registry" Jul 05 13:21:29.693671 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:21:29.693498 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="943ce8b6-c20c-4d0a-8c56-29a8aa714643" containerName="console" Jul 05 13:21:29.693671 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:21:29.693503 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="943ce8b6-c20c-4d0a-8c56-29a8aa714643" containerName="console" Jul 05 13:21:29.693671 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:21:29.693511 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b68e67e7-b24f-4823-93ce-b1a21d9997df" containerName="console" Jul 05 13:21:29.693671 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:21:29.693516 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="b68e67e7-b24f-4823-93ce-b1a21d9997df" containerName="console" Jul 05 13:21:29.693671 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:21:29.693522 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3644ed9d-8c13-448b-98ab-4017dab5ea27" containerName="console" Jul 05 13:21:29.693671 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:21:29.693527 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="3644ed9d-8c13-448b-98ab-4017dab5ea27" containerName="console" Jul 05 13:21:29.693671 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:21:29.693605 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="3644ed9d-8c13-448b-98ab-4017dab5ea27" containerName="console" Jul 05 13:21:29.693671 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:21:29.693616 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="943ce8b6-c20c-4d0a-8c56-29a8aa714643" containerName="console" Jul 05 13:21:29.693671 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:21:29.693626 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="d933ee11-a974-467b-9dce-f898bcf3863b" containerName="registry" Jul 05 13:21:29.693671 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:21:29.693632 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="b68e67e7-b24f-4823-93ce-b1a21d9997df" containerName="console" Jul 05 13:21:29.698208 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:21:29.698179 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-75cb8f6bcc-dbrfg" Jul 05 13:21:29.710462 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:21:29.710432 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-75cb8f6bcc-dbrfg"] Jul 05 13:21:29.736302 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:21:29.736261 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bd354538-c932-466f-ae54-9575b7c11f52-trusted-ca-bundle\") pod \"console-75cb8f6bcc-dbrfg\" (UID: \"bd354538-c932-466f-ae54-9575b7c11f52\") " pod="openshift-console/console-75cb8f6bcc-dbrfg" Jul 05 13:21:29.736467 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:21:29.736371 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/bd354538-c932-466f-ae54-9575b7c11f52-console-oauth-config\") pod \"console-75cb8f6bcc-dbrfg\" (UID: \"bd354538-c932-466f-ae54-9575b7c11f52\") " pod="openshift-console/console-75cb8f6bcc-dbrfg" Jul 05 13:21:29.736467 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:21:29.736397 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/bd354538-c932-466f-ae54-9575b7c11f52-oauth-serving-cert\") pod \"console-75cb8f6bcc-dbrfg\" (UID: \"bd354538-c932-466f-ae54-9575b7c11f52\") " pod="openshift-console/console-75cb8f6bcc-dbrfg" Jul 05 13:21:29.736467 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:21:29.736423 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/bd354538-c932-466f-ae54-9575b7c11f52-console-config\") pod \"console-75cb8f6bcc-dbrfg\" (UID: \"bd354538-c932-466f-ae54-9575b7c11f52\") " pod="openshift-console/console-75cb8f6bcc-dbrfg" Jul 05 13:21:29.736467 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:21:29.736442 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qkfpl\" (UniqueName: \"kubernetes.io/projected/bd354538-c932-466f-ae54-9575b7c11f52-kube-api-access-qkfpl\") pod \"console-75cb8f6bcc-dbrfg\" (UID: \"bd354538-c932-466f-ae54-9575b7c11f52\") " pod="openshift-console/console-75cb8f6bcc-dbrfg" Jul 05 13:21:29.736467 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:21:29.736461 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/bd354538-c932-466f-ae54-9575b7c11f52-console-serving-cert\") pod \"console-75cb8f6bcc-dbrfg\" (UID: \"bd354538-c932-466f-ae54-9575b7c11f52\") " pod="openshift-console/console-75cb8f6bcc-dbrfg" Jul 05 13:21:29.736682 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:21:29.736481 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/bd354538-c932-466f-ae54-9575b7c11f52-service-ca\") pod \"console-75cb8f6bcc-dbrfg\" (UID: \"bd354538-c932-466f-ae54-9575b7c11f52\") " pod="openshift-console/console-75cb8f6bcc-dbrfg" Jul 05 13:21:29.837419 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:21:29.837386 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/bd354538-c932-466f-ae54-9575b7c11f52-oauth-serving-cert\") pod \"console-75cb8f6bcc-dbrfg\" (UID: \"bd354538-c932-466f-ae54-9575b7c11f52\") " pod="openshift-console/console-75cb8f6bcc-dbrfg" Jul 05 13:21:29.837419 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:21:29.837434 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/bd354538-c932-466f-ae54-9575b7c11f52-console-config\") pod \"console-75cb8f6bcc-dbrfg\" (UID: \"bd354538-c932-466f-ae54-9575b7c11f52\") " pod="openshift-console/console-75cb8f6bcc-dbrfg" Jul 05 13:21:29.837680 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:21:29.837458 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qkfpl\" (UniqueName: \"kubernetes.io/projected/bd354538-c932-466f-ae54-9575b7c11f52-kube-api-access-qkfpl\") pod \"console-75cb8f6bcc-dbrfg\" (UID: \"bd354538-c932-466f-ae54-9575b7c11f52\") " pod="openshift-console/console-75cb8f6bcc-dbrfg" Jul 05 13:21:29.837680 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:21:29.837483 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/bd354538-c932-466f-ae54-9575b7c11f52-console-serving-cert\") pod \"console-75cb8f6bcc-dbrfg\" (UID: \"bd354538-c932-466f-ae54-9575b7c11f52\") " pod="openshift-console/console-75cb8f6bcc-dbrfg" Jul 05 13:21:29.837680 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:21:29.837511 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/bd354538-c932-466f-ae54-9575b7c11f52-service-ca\") pod \"console-75cb8f6bcc-dbrfg\" (UID: \"bd354538-c932-466f-ae54-9575b7c11f52\") " pod="openshift-console/console-75cb8f6bcc-dbrfg" Jul 05 13:21:29.837680 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:21:29.837574 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bd354538-c932-466f-ae54-9575b7c11f52-trusted-ca-bundle\") pod \"console-75cb8f6bcc-dbrfg\" (UID: \"bd354538-c932-466f-ae54-9575b7c11f52\") " pod="openshift-console/console-75cb8f6bcc-dbrfg" Jul 05 13:21:29.837680 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:21:29.837655 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/bd354538-c932-466f-ae54-9575b7c11f52-console-oauth-config\") pod \"console-75cb8f6bcc-dbrfg\" (UID: \"bd354538-c932-466f-ae54-9575b7c11f52\") " pod="openshift-console/console-75cb8f6bcc-dbrfg" Jul 05 13:21:29.838343 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:21:29.838306 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/bd354538-c932-466f-ae54-9575b7c11f52-service-ca\") pod \"console-75cb8f6bcc-dbrfg\" (UID: \"bd354538-c932-466f-ae54-9575b7c11f52\") " pod="openshift-console/console-75cb8f6bcc-dbrfg" Jul 05 13:21:29.838447 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:21:29.838338 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/bd354538-c932-466f-ae54-9575b7c11f52-console-config\") pod \"console-75cb8f6bcc-dbrfg\" (UID: \"bd354538-c932-466f-ae54-9575b7c11f52\") " pod="openshift-console/console-75cb8f6bcc-dbrfg" Jul 05 13:21:29.838503 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:21:29.838442 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bd354538-c932-466f-ae54-9575b7c11f52-trusted-ca-bundle\") pod \"console-75cb8f6bcc-dbrfg\" (UID: \"bd354538-c932-466f-ae54-9575b7c11f52\") " pod="openshift-console/console-75cb8f6bcc-dbrfg" Jul 05 13:21:29.838539 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:21:29.838513 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/bd354538-c932-466f-ae54-9575b7c11f52-oauth-serving-cert\") pod \"console-75cb8f6bcc-dbrfg\" (UID: \"bd354538-c932-466f-ae54-9575b7c11f52\") " pod="openshift-console/console-75cb8f6bcc-dbrfg" Jul 05 13:21:29.840162 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:21:29.840144 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/bd354538-c932-466f-ae54-9575b7c11f52-console-oauth-config\") pod \"console-75cb8f6bcc-dbrfg\" (UID: \"bd354538-c932-466f-ae54-9575b7c11f52\") " pod="openshift-console/console-75cb8f6bcc-dbrfg" Jul 05 13:21:29.840257 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:21:29.840239 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/bd354538-c932-466f-ae54-9575b7c11f52-console-serving-cert\") pod \"console-75cb8f6bcc-dbrfg\" (UID: \"bd354538-c932-466f-ae54-9575b7c11f52\") " pod="openshift-console/console-75cb8f6bcc-dbrfg" Jul 05 13:21:29.847269 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:21:29.847235 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qkfpl\" (UniqueName: \"kubernetes.io/projected/bd354538-c932-466f-ae54-9575b7c11f52-kube-api-access-qkfpl\") pod \"console-75cb8f6bcc-dbrfg\" (UID: \"bd354538-c932-466f-ae54-9575b7c11f52\") " pod="openshift-console/console-75cb8f6bcc-dbrfg" Jul 05 13:21:29.938633 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:21:29.938564 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5ad92660-5823-4f14-adcc-763d06385f45-metrics-certs\") pod \"network-metrics-daemon-dxwj6\" (UID: \"5ad92660-5823-4f14-adcc-763d06385f45\") " pod="openshift-multus/network-metrics-daemon-dxwj6" Jul 05 13:21:29.941006 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:21:29.940984 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5ad92660-5823-4f14-adcc-763d06385f45-metrics-certs\") pod \"network-metrics-daemon-dxwj6\" (UID: \"5ad92660-5823-4f14-adcc-763d06385f45\") " pod="openshift-multus/network-metrics-daemon-dxwj6" Jul 05 13:21:30.008468 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:21:30.008443 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-75cb8f6bcc-dbrfg" Jul 05 13:21:30.146565 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:21:30.146539 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-75cb8f6bcc-dbrfg"] Jul 05 13:21:30.148695 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:21:30.148662 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbd354538_c932_466f_ae54_9575b7c11f52.slice/crio-6322269673a65c4a7fd63f427953014d3de334c2078a6191afecdc96596cae82 WatchSource:0}: Error finding container 6322269673a65c4a7fd63f427953014d3de334c2078a6191afecdc96596cae82: Status 404 returned error can't find the container with id 6322269673a65c4a7fd63f427953014d3de334c2078a6191afecdc96596cae82 Jul 05 13:21:30.206836 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:21:30.206813 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-jgr78\"" Jul 05 13:21:30.215080 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:21:30.215054 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dxwj6" Jul 05 13:21:30.366314 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:21:30.366277 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-75cb8f6bcc-dbrfg" event={"ID":"bd354538-c932-466f-ae54-9575b7c11f52","Type":"ContainerStarted","Data":"6c867577e39196148bb42c817fc740f50ee9770a18274d0f8f7939c41067a7c9"} Jul 05 13:21:30.366484 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:21:30.366337 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-75cb8f6bcc-dbrfg" event={"ID":"bd354538-c932-466f-ae54-9575b7c11f52","Type":"ContainerStarted","Data":"6322269673a65c4a7fd63f427953014d3de334c2078a6191afecdc96596cae82"} Jul 05 13:21:30.372432 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:21:30.372389 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-dxwj6"] Jul 05 13:21:30.374545 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:21:30.374520 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5ad92660_5823_4f14_adcc_763d06385f45.slice/crio-f8673411f7e70535596f83915959d348fcd3f391fb4e53367f6ab11f84d001cb WatchSource:0}: Error finding container f8673411f7e70535596f83915959d348fcd3f391fb4e53367f6ab11f84d001cb: Status 404 returned error can't find the container with id f8673411f7e70535596f83915959d348fcd3f391fb4e53367f6ab11f84d001cb Jul 05 13:21:30.389106 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:21:30.389050 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-75cb8f6bcc-dbrfg" podStartSLOduration=1.389032266 podStartE2EDuration="1.389032266s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 13:21:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 13:21:30.387106864 +0000 UTC m=+374.845669977" watchObservedRunningTime="2026-07-05 13:21:30.389032266 +0000 UTC m=+374.847595371" Jul 05 13:21:31.371261 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:21:31.371215 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-dxwj6" event={"ID":"5ad92660-5823-4f14-adcc-763d06385f45","Type":"ContainerStarted","Data":"f8673411f7e70535596f83915959d348fcd3f391fb4e53367f6ab11f84d001cb"} Jul 05 13:21:32.376570 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:21:32.376534 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-dxwj6" event={"ID":"5ad92660-5823-4f14-adcc-763d06385f45","Type":"ContainerStarted","Data":"e82beb2aaa101c5fc647bab119cc1fb6384ec8342ab64bcd869b413a7c2d5165"} Jul 05 13:21:32.376570 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:21:32.376604 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-dxwj6" event={"ID":"5ad92660-5823-4f14-adcc-763d06385f45","Type":"ContainerStarted","Data":"ad9393f10e5b14bde61cd32511d8ad959ad364629bf1f963903dc7b2d0cda4b2"} Jul 05 13:21:32.395082 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:21:32.395031 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-dxwj6" podStartSLOduration=375.279640178 podStartE2EDuration="6m16.395016491s" totalImagesPullingTime="1.115376313s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 13:15:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 13:21:32.39364819 +0000 UTC m=+376.852211295" watchObservedRunningTime="2026-07-05 13:21:32.395016491 +0000 UTC m=+376.853579597" Jul 05 13:21:40.009483 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:21:40.009455 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-75cb8f6bcc-dbrfg" Jul 05 13:21:40.009860 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:21:40.009504 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-75cb8f6bcc-dbrfg" Jul 05 13:21:40.014161 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:21:40.014141 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-75cb8f6bcc-dbrfg" Jul 05 13:21:40.412525 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:21:40.412446 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-75cb8f6bcc-dbrfg" Jul 05 13:21:40.475726 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:21:40.475688 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-66955d844b-9pds8"] Jul 05 13:22:05.498078 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:22:05.498034 2578 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-66955d844b-9pds8" podUID="8435ac41-be8e-4ab7-8f3a-8f50f3b02416" containerName="console" containerID="cri-o://3bf0f0b8df333e8ea7c4a7275680ab1bd40c8197c1aabbfa9045a3493fcc3706" gracePeriod=15 Jul 05 13:22:05.755898 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:22:05.755836 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-66955d844b-9pds8" Jul 05 13:22:05.848178 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:22:05.848142 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8435ac41-be8e-4ab7-8f3a-8f50f3b02416-console-config\") pod \"8435ac41-be8e-4ab7-8f3a-8f50f3b02416\" (UID: \"8435ac41-be8e-4ab7-8f3a-8f50f3b02416\") " Jul 05 13:22:05.848345 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:22:05.848206 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8435ac41-be8e-4ab7-8f3a-8f50f3b02416-trusted-ca-bundle\") pod \"8435ac41-be8e-4ab7-8f3a-8f50f3b02416\" (UID: \"8435ac41-be8e-4ab7-8f3a-8f50f3b02416\") " Jul 05 13:22:05.848345 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:22:05.848228 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pjbvs\" (UniqueName: \"kubernetes.io/projected/8435ac41-be8e-4ab7-8f3a-8f50f3b02416-kube-api-access-pjbvs\") pod \"8435ac41-be8e-4ab7-8f3a-8f50f3b02416\" (UID: \"8435ac41-be8e-4ab7-8f3a-8f50f3b02416\") " Jul 05 13:22:05.848345 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:22:05.848249 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8435ac41-be8e-4ab7-8f3a-8f50f3b02416-oauth-serving-cert\") pod \"8435ac41-be8e-4ab7-8f3a-8f50f3b02416\" (UID: \"8435ac41-be8e-4ab7-8f3a-8f50f3b02416\") " Jul 05 13:22:05.848345 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:22:05.848274 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8435ac41-be8e-4ab7-8f3a-8f50f3b02416-service-ca\") pod \"8435ac41-be8e-4ab7-8f3a-8f50f3b02416\" (UID: \"8435ac41-be8e-4ab7-8f3a-8f50f3b02416\") " Jul 05 13:22:05.848345 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:22:05.848296 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8435ac41-be8e-4ab7-8f3a-8f50f3b02416-console-serving-cert\") pod \"8435ac41-be8e-4ab7-8f3a-8f50f3b02416\" (UID: \"8435ac41-be8e-4ab7-8f3a-8f50f3b02416\") " Jul 05 13:22:05.848345 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:22:05.848339 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8435ac41-be8e-4ab7-8f3a-8f50f3b02416-console-oauth-config\") pod \"8435ac41-be8e-4ab7-8f3a-8f50f3b02416\" (UID: \"8435ac41-be8e-4ab7-8f3a-8f50f3b02416\") " Jul 05 13:22:05.848776 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:22:05.848742 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8435ac41-be8e-4ab7-8f3a-8f50f3b02416-console-config" (OuterVolumeSpecName: "console-config") pod "8435ac41-be8e-4ab7-8f3a-8f50f3b02416" (UID: "8435ac41-be8e-4ab7-8f3a-8f50f3b02416"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 05 13:22:05.848895 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:22:05.848826 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8435ac41-be8e-4ab7-8f3a-8f50f3b02416-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "8435ac41-be8e-4ab7-8f3a-8f50f3b02416" (UID: "8435ac41-be8e-4ab7-8f3a-8f50f3b02416"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 05 13:22:05.848895 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:22:05.848822 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8435ac41-be8e-4ab7-8f3a-8f50f3b02416-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "8435ac41-be8e-4ab7-8f3a-8f50f3b02416" (UID: "8435ac41-be8e-4ab7-8f3a-8f50f3b02416"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 05 13:22:05.848895 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:22:05.848847 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8435ac41-be8e-4ab7-8f3a-8f50f3b02416-service-ca" (OuterVolumeSpecName: "service-ca") pod "8435ac41-be8e-4ab7-8f3a-8f50f3b02416" (UID: "8435ac41-be8e-4ab7-8f3a-8f50f3b02416"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 05 13:22:05.850762 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:22:05.850741 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8435ac41-be8e-4ab7-8f3a-8f50f3b02416-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "8435ac41-be8e-4ab7-8f3a-8f50f3b02416" (UID: "8435ac41-be8e-4ab7-8f3a-8f50f3b02416"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 05 13:22:05.850840 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:22:05.850811 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8435ac41-be8e-4ab7-8f3a-8f50f3b02416-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "8435ac41-be8e-4ab7-8f3a-8f50f3b02416" (UID: "8435ac41-be8e-4ab7-8f3a-8f50f3b02416"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 05 13:22:05.850878 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:22:05.850854 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8435ac41-be8e-4ab7-8f3a-8f50f3b02416-kube-api-access-pjbvs" (OuterVolumeSpecName: "kube-api-access-pjbvs") pod "8435ac41-be8e-4ab7-8f3a-8f50f3b02416" (UID: "8435ac41-be8e-4ab7-8f3a-8f50f3b02416"). InnerVolumeSpecName "kube-api-access-pjbvs". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 05 13:22:05.949348 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:22:05.949306 2578 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8435ac41-be8e-4ab7-8f3a-8f50f3b02416-console-oauth-config\") on node \"ip-10-0-130-191.ec2.internal\" DevicePath \"\"" Jul 05 13:22:05.949348 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:22:05.949340 2578 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8435ac41-be8e-4ab7-8f3a-8f50f3b02416-console-config\") on node \"ip-10-0-130-191.ec2.internal\" DevicePath \"\"" Jul 05 13:22:05.949348 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:22:05.949349 2578 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8435ac41-be8e-4ab7-8f3a-8f50f3b02416-trusted-ca-bundle\") on node \"ip-10-0-130-191.ec2.internal\" DevicePath \"\"" Jul 05 13:22:05.949348 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:22:05.949359 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pjbvs\" (UniqueName: \"kubernetes.io/projected/8435ac41-be8e-4ab7-8f3a-8f50f3b02416-kube-api-access-pjbvs\") on node \"ip-10-0-130-191.ec2.internal\" DevicePath \"\"" Jul 05 13:22:05.949657 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:22:05.949371 2578 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8435ac41-be8e-4ab7-8f3a-8f50f3b02416-oauth-serving-cert\") on node \"ip-10-0-130-191.ec2.internal\" DevicePath \"\"" Jul 05 13:22:05.949657 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:22:05.949380 2578 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8435ac41-be8e-4ab7-8f3a-8f50f3b02416-service-ca\") on node \"ip-10-0-130-191.ec2.internal\" DevicePath \"\"" Jul 05 13:22:05.949657 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:22:05.949388 2578 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8435ac41-be8e-4ab7-8f3a-8f50f3b02416-console-serving-cert\") on node \"ip-10-0-130-191.ec2.internal\" DevicePath \"\"" Jul 05 13:22:06.511168 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:22:06.511131 2578 generic.go:356] "Generic (PLEG): container finished" podID="8435ac41-be8e-4ab7-8f3a-8f50f3b02416" containerID="3bf0f0b8df333e8ea7c4a7275680ab1bd40c8197c1aabbfa9045a3493fcc3706" exitCode=0 Jul 05 13:22:06.511643 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:22:06.511178 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-66955d844b-9pds8" Jul 05 13:22:06.511643 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:22:06.511188 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-66955d844b-9pds8" event={"ID":"8435ac41-be8e-4ab7-8f3a-8f50f3b02416","Type":"ContainerDied","Data":"3bf0f0b8df333e8ea7c4a7275680ab1bd40c8197c1aabbfa9045a3493fcc3706"} Jul 05 13:22:06.511643 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:22:06.511243 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-66955d844b-9pds8" event={"ID":"8435ac41-be8e-4ab7-8f3a-8f50f3b02416","Type":"ContainerDied","Data":"6c1ded61734e038cbf904fb03228d3940b23daa4fe0ccca37a577b26b2966b95"} Jul 05 13:22:06.511643 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:22:06.511270 2578 scope.go:117] "RemoveContainer" containerID="3bf0f0b8df333e8ea7c4a7275680ab1bd40c8197c1aabbfa9045a3493fcc3706" Jul 05 13:22:06.521780 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:22:06.521757 2578 scope.go:117] "RemoveContainer" containerID="3bf0f0b8df333e8ea7c4a7275680ab1bd40c8197c1aabbfa9045a3493fcc3706" Jul 05 13:22:06.522119 ip-10-0-130-191 kubenswrapper[2578]: E0705 13:22:06.522094 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3bf0f0b8df333e8ea7c4a7275680ab1bd40c8197c1aabbfa9045a3493fcc3706\": container with ID starting with 3bf0f0b8df333e8ea7c4a7275680ab1bd40c8197c1aabbfa9045a3493fcc3706 not found: ID does not exist" containerID="3bf0f0b8df333e8ea7c4a7275680ab1bd40c8197c1aabbfa9045a3493fcc3706" Jul 05 13:22:06.522177 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:22:06.522134 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3bf0f0b8df333e8ea7c4a7275680ab1bd40c8197c1aabbfa9045a3493fcc3706"} err="failed to get container status \"3bf0f0b8df333e8ea7c4a7275680ab1bd40c8197c1aabbfa9045a3493fcc3706\": rpc error: code = NotFound desc = could not find container \"3bf0f0b8df333e8ea7c4a7275680ab1bd40c8197c1aabbfa9045a3493fcc3706\": container with ID starting with 3bf0f0b8df333e8ea7c4a7275680ab1bd40c8197c1aabbfa9045a3493fcc3706 not found: ID does not exist" Jul 05 13:22:06.530751 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:22:06.530724 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-66955d844b-9pds8"] Jul 05 13:22:06.534537 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:22:06.534510 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-66955d844b-9pds8"] Jul 05 13:22:08.009174 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:22:08.009141 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8435ac41-be8e-4ab7-8f3a-8f50f3b02416" path="/var/lib/kubelet/pods/8435ac41-be8e-4ab7-8f3a-8f50f3b02416/volumes" Jul 05 13:22:24.408041 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:22:24.408001 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675406w4sf"] Jul 05 13:22:24.408411 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:22:24.408330 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8435ac41-be8e-4ab7-8f3a-8f50f3b02416" containerName="console" Jul 05 13:22:24.408411 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:22:24.408343 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="8435ac41-be8e-4ab7-8f3a-8f50f3b02416" containerName="console" Jul 05 13:22:24.408411 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:22:24.408411 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="8435ac41-be8e-4ab7-8f3a-8f50f3b02416" containerName="console" Jul 05 13:22:24.412889 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:22:24.412871 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675406w4sf" Jul 05 13:22:24.415865 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:22:24.415845 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 05 13:22:24.416003 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:22:24.415848 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-m9dbt\"" Jul 05 13:22:24.417167 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:22:24.417138 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 05 13:22:24.425143 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:22:24.425113 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675406w4sf"] Jul 05 13:22:24.497677 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:22:24.497632 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fm6mf\" (UniqueName: \"kubernetes.io/projected/2665e689-013f-476f-8bce-99fcb939c837-kube-api-access-fm6mf\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675406w4sf\" (UID: \"2665e689-013f-476f-8bce-99fcb939c837\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675406w4sf" Jul 05 13:22:24.497878 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:22:24.497694 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2665e689-013f-476f-8bce-99fcb939c837-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675406w4sf\" (UID: \"2665e689-013f-476f-8bce-99fcb939c837\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675406w4sf" Jul 05 13:22:24.497878 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:22:24.497766 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2665e689-013f-476f-8bce-99fcb939c837-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675406w4sf\" (UID: \"2665e689-013f-476f-8bce-99fcb939c837\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675406w4sf" Jul 05 13:22:24.598305 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:22:24.598264 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fm6mf\" (UniqueName: \"kubernetes.io/projected/2665e689-013f-476f-8bce-99fcb939c837-kube-api-access-fm6mf\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675406w4sf\" (UID: \"2665e689-013f-476f-8bce-99fcb939c837\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675406w4sf" Jul 05 13:22:24.598305 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:22:24.598312 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2665e689-013f-476f-8bce-99fcb939c837-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675406w4sf\" (UID: \"2665e689-013f-476f-8bce-99fcb939c837\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675406w4sf" Jul 05 13:22:24.598552 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:22:24.598348 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2665e689-013f-476f-8bce-99fcb939c837-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675406w4sf\" (UID: \"2665e689-013f-476f-8bce-99fcb939c837\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675406w4sf" Jul 05 13:22:24.598739 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:22:24.598720 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2665e689-013f-476f-8bce-99fcb939c837-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675406w4sf\" (UID: \"2665e689-013f-476f-8bce-99fcb939c837\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675406w4sf" Jul 05 13:22:24.598814 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:22:24.598768 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2665e689-013f-476f-8bce-99fcb939c837-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675406w4sf\" (UID: \"2665e689-013f-476f-8bce-99fcb939c837\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675406w4sf" Jul 05 13:22:24.607036 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:22:24.607014 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fm6mf\" (UniqueName: \"kubernetes.io/projected/2665e689-013f-476f-8bce-99fcb939c837-kube-api-access-fm6mf\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675406w4sf\" (UID: \"2665e689-013f-476f-8bce-99fcb939c837\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675406w4sf" Jul 05 13:22:24.724344 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:22:24.724243 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675406w4sf" Jul 05 13:22:24.858061 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:22:24.858035 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675406w4sf"] Jul 05 13:22:24.860810 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:22:24.860782 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2665e689_013f_476f_8bce_99fcb939c837.slice/crio-ee3163ab7b7e670b49c57668040edd220412e0fcad7cf87538763b22b4f9bdf2 WatchSource:0}: Error finding container ee3163ab7b7e670b49c57668040edd220412e0fcad7cf87538763b22b4f9bdf2: Status 404 returned error can't find the container with id ee3163ab7b7e670b49c57668040edd220412e0fcad7cf87538763b22b4f9bdf2 Jul 05 13:22:25.589913 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:22:25.589873 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675406w4sf" event={"ID":"2665e689-013f-476f-8bce-99fcb939c837","Type":"ContainerStarted","Data":"ee3163ab7b7e670b49c57668040edd220412e0fcad7cf87538763b22b4f9bdf2"} Jul 05 13:22:31.617447 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:22:31.617412 2578 generic.go:356] "Generic (PLEG): container finished" podID="2665e689-013f-476f-8bce-99fcb939c837" containerID="4afbc4834b7e50622ed51038b7badf155aba6ea0e1ac7982ced51ccd4ca88538" exitCode=0 Jul 05 13:22:31.617866 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:22:31.617483 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675406w4sf" event={"ID":"2665e689-013f-476f-8bce-99fcb939c837","Type":"ContainerDied","Data":"4afbc4834b7e50622ed51038b7badf155aba6ea0e1ac7982ced51ccd4ca88538"} Jul 05 13:22:33.627736 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:22:33.627648 2578 generic.go:356] "Generic (PLEG): container finished" podID="2665e689-013f-476f-8bce-99fcb939c837" containerID="d1fb186cbb54b0d8a61c021f55bca6a9cacb23b7d57076c0b13dfb50c1d0418f" exitCode=0 Jul 05 13:22:33.627736 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:22:33.627693 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675406w4sf" event={"ID":"2665e689-013f-476f-8bce-99fcb939c837","Type":"ContainerDied","Data":"d1fb186cbb54b0d8a61c021f55bca6a9cacb23b7d57076c0b13dfb50c1d0418f"} Jul 05 13:22:39.656137 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:22:39.656097 2578 generic.go:356] "Generic (PLEG): container finished" podID="2665e689-013f-476f-8bce-99fcb939c837" containerID="11d93bc082182c83109946827c197ec1f1a0a387181d1772e1ffead3a59a25d6" exitCode=0 Jul 05 13:22:39.656505 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:22:39.656184 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675406w4sf" event={"ID":"2665e689-013f-476f-8bce-99fcb939c837","Type":"ContainerDied","Data":"11d93bc082182c83109946827c197ec1f1a0a387181d1772e1ffead3a59a25d6"} Jul 05 13:22:40.789843 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:22:40.789819 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675406w4sf" Jul 05 13:22:40.829858 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:22:40.829824 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2665e689-013f-476f-8bce-99fcb939c837-util\") pod \"2665e689-013f-476f-8bce-99fcb939c837\" (UID: \"2665e689-013f-476f-8bce-99fcb939c837\") " Jul 05 13:22:40.830022 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:22:40.829874 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fm6mf\" (UniqueName: \"kubernetes.io/projected/2665e689-013f-476f-8bce-99fcb939c837-kube-api-access-fm6mf\") pod \"2665e689-013f-476f-8bce-99fcb939c837\" (UID: \"2665e689-013f-476f-8bce-99fcb939c837\") " Jul 05 13:22:40.830022 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:22:40.829930 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2665e689-013f-476f-8bce-99fcb939c837-bundle\") pod \"2665e689-013f-476f-8bce-99fcb939c837\" (UID: \"2665e689-013f-476f-8bce-99fcb939c837\") " Jul 05 13:22:40.830699 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:22:40.830626 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2665e689-013f-476f-8bce-99fcb939c837-bundle" (OuterVolumeSpecName: "bundle") pod "2665e689-013f-476f-8bce-99fcb939c837" (UID: "2665e689-013f-476f-8bce-99fcb939c837"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 05 13:22:40.832229 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:22:40.832203 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2665e689-013f-476f-8bce-99fcb939c837-kube-api-access-fm6mf" (OuterVolumeSpecName: "kube-api-access-fm6mf") pod "2665e689-013f-476f-8bce-99fcb939c837" (UID: "2665e689-013f-476f-8bce-99fcb939c837"). InnerVolumeSpecName "kube-api-access-fm6mf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 05 13:22:40.835572 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:22:40.835550 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2665e689-013f-476f-8bce-99fcb939c837-util" (OuterVolumeSpecName: "util") pod "2665e689-013f-476f-8bce-99fcb939c837" (UID: "2665e689-013f-476f-8bce-99fcb939c837"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 05 13:22:40.930604 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:22:40.930490 2578 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2665e689-013f-476f-8bce-99fcb939c837-bundle\") on node \"ip-10-0-130-191.ec2.internal\" DevicePath \"\"" Jul 05 13:22:40.930604 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:22:40.930520 2578 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2665e689-013f-476f-8bce-99fcb939c837-util\") on node \"ip-10-0-130-191.ec2.internal\" DevicePath \"\"" Jul 05 13:22:40.930604 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:22:40.930530 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fm6mf\" (UniqueName: \"kubernetes.io/projected/2665e689-013f-476f-8bce-99fcb939c837-kube-api-access-fm6mf\") on node \"ip-10-0-130-191.ec2.internal\" DevicePath \"\"" Jul 05 13:22:41.668312 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:22:41.668286 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675406w4sf" Jul 05 13:22:41.668312 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:22:41.668306 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675406w4sf" event={"ID":"2665e689-013f-476f-8bce-99fcb939c837","Type":"ContainerDied","Data":"ee3163ab7b7e670b49c57668040edd220412e0fcad7cf87538763b22b4f9bdf2"} Jul 05 13:22:41.668520 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:22:41.668347 2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ee3163ab7b7e670b49c57668040edd220412e0fcad7cf87538763b22b4f9bdf2" Jul 05 13:22:47.428985 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:22:47.428948 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-c42b6"] Jul 05 13:22:47.429439 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:22:47.429266 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2665e689-013f-476f-8bce-99fcb939c837" containerName="pull" Jul 05 13:22:47.429439 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:22:47.429278 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="2665e689-013f-476f-8bce-99fcb939c837" containerName="pull" Jul 05 13:22:47.429439 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:22:47.429291 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2665e689-013f-476f-8bce-99fcb939c837" containerName="util" Jul 05 13:22:47.429439 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:22:47.429296 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="2665e689-013f-476f-8bce-99fcb939c837" containerName="util" Jul 05 13:22:47.429439 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:22:47.429307 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2665e689-013f-476f-8bce-99fcb939c837" containerName="extract" Jul 05 13:22:47.429439 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:22:47.429313 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="2665e689-013f-476f-8bce-99fcb939c837" containerName="extract" Jul 05 13:22:47.429439 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:22:47.429363 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="2665e689-013f-476f-8bce-99fcb939c837" containerName="extract" Jul 05 13:22:47.432895 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:22:47.432879 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-c42b6" Jul 05 13:22:47.439881 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:22:47.439857 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-d4sdg\"" Jul 05 13:22:47.440167 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:22:47.440152 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\"" Jul 05 13:22:47.440281 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:22:47.440263 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\"" Jul 05 13:22:47.449954 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:22:47.449927 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-c42b6"] Jul 05 13:22:47.483981 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:22:47.483946 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/85089688-fe2f-46bd-a1df-7f8a56fb8ab9-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-c42b6\" (UID: \"85089688-fe2f-46bd-a1df-7f8a56fb8ab9\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-c42b6" Jul 05 13:22:47.484163 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:22:47.484064 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kg4j6\" (UniqueName: \"kubernetes.io/projected/85089688-fe2f-46bd-a1df-7f8a56fb8ab9-kube-api-access-kg4j6\") pod \"cert-manager-operator-controller-manager-55c444bf86-c42b6\" (UID: \"85089688-fe2f-46bd-a1df-7f8a56fb8ab9\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-c42b6" Jul 05 13:22:47.584814 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:22:47.584772 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/85089688-fe2f-46bd-a1df-7f8a56fb8ab9-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-c42b6\" (UID: \"85089688-fe2f-46bd-a1df-7f8a56fb8ab9\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-c42b6" Jul 05 13:22:47.585012 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:22:47.584890 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kg4j6\" (UniqueName: \"kubernetes.io/projected/85089688-fe2f-46bd-a1df-7f8a56fb8ab9-kube-api-access-kg4j6\") pod \"cert-manager-operator-controller-manager-55c444bf86-c42b6\" (UID: \"85089688-fe2f-46bd-a1df-7f8a56fb8ab9\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-c42b6" Jul 05 13:22:47.585183 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:22:47.585160 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/85089688-fe2f-46bd-a1df-7f8a56fb8ab9-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-c42b6\" (UID: \"85089688-fe2f-46bd-a1df-7f8a56fb8ab9\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-c42b6" Jul 05 13:22:47.594107 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:22:47.594075 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kg4j6\" (UniqueName: \"kubernetes.io/projected/85089688-fe2f-46bd-a1df-7f8a56fb8ab9-kube-api-access-kg4j6\") pod \"cert-manager-operator-controller-manager-55c444bf86-c42b6\" (UID: \"85089688-fe2f-46bd-a1df-7f8a56fb8ab9\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-c42b6" Jul 05 13:22:47.741798 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:22:47.741706 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-c42b6" Jul 05 13:22:47.883112 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:22:47.883079 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-c42b6"] Jul 05 13:22:47.887686 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:22:47.887658 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod85089688_fe2f_46bd_a1df_7f8a56fb8ab9.slice/crio-c4c6f951213d89189400e585892b1bb4775d51b03761317aaffd78a04790608b WatchSource:0}: Error finding container c4c6f951213d89189400e585892b1bb4775d51b03761317aaffd78a04790608b: Status 404 returned error can't find the container with id c4c6f951213d89189400e585892b1bb4775d51b03761317aaffd78a04790608b Jul 05 13:22:48.698094 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:22:48.698050 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-c42b6" event={"ID":"85089688-fe2f-46bd-a1df-7f8a56fb8ab9","Type":"ContainerStarted","Data":"c4c6f951213d89189400e585892b1bb4775d51b03761317aaffd78a04790608b"} Jul 05 13:22:49.704485 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:22:49.704397 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-c42b6" event={"ID":"85089688-fe2f-46bd-a1df-7f8a56fb8ab9","Type":"ContainerStarted","Data":"80f30a7ab71b94b9a8441b16a8c6950928e9df1156d675105cf554f267221364"} Jul 05 13:22:49.739855 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:22:49.739797 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-c42b6" podStartSLOduration=1.253772175 podStartE2EDuration="2.739781052s" totalImagesPullingTime="1.486008877s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 13:22:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 13:22:49.736747936 +0000 UTC m=+454.195311052" watchObservedRunningTime="2026-07-05 13:22:49.739781052 +0000 UTC m=+454.198344157" Jul 05 13:22:51.057187 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:22:51.057150 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f775lp"] Jul 05 13:22:51.059474 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:22:51.059446 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f775lp" Jul 05 13:22:51.062339 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:22:51.062316 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-m9dbt\"" Jul 05 13:22:51.062457 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:22:51.062353 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 05 13:22:51.063546 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:22:51.063529 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 05 13:22:51.072733 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:22:51.072702 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f775lp"] Jul 05 13:22:51.112186 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:22:51.112143 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4473a76f-90c3-4421-889c-bb84fb3f986f-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f775lp\" (UID: \"4473a76f-90c3-4421-889c-bb84fb3f986f\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f775lp" Jul 05 13:22:51.112186 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:22:51.112192 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4473a76f-90c3-4421-889c-bb84fb3f986f-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f775lp\" (UID: \"4473a76f-90c3-4421-889c-bb84fb3f986f\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f775lp" Jul 05 13:22:51.112465 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:22:51.112223 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4r8wb\" (UniqueName: \"kubernetes.io/projected/4473a76f-90c3-4421-889c-bb84fb3f986f-kube-api-access-4r8wb\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f775lp\" (UID: \"4473a76f-90c3-4421-889c-bb84fb3f986f\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f775lp" Jul 05 13:22:51.213600 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:22:51.213543 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4473a76f-90c3-4421-889c-bb84fb3f986f-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f775lp\" (UID: \"4473a76f-90c3-4421-889c-bb84fb3f986f\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f775lp" Jul 05 13:22:51.213600 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:22:51.213596 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4473a76f-90c3-4421-889c-bb84fb3f986f-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f775lp\" (UID: \"4473a76f-90c3-4421-889c-bb84fb3f986f\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f775lp" Jul 05 13:22:51.213827 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:22:51.213631 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4r8wb\" (UniqueName: \"kubernetes.io/projected/4473a76f-90c3-4421-889c-bb84fb3f986f-kube-api-access-4r8wb\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f775lp\" (UID: \"4473a76f-90c3-4421-889c-bb84fb3f986f\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f775lp" Jul 05 13:22:51.213977 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:22:51.213958 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4473a76f-90c3-4421-889c-bb84fb3f986f-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f775lp\" (UID: \"4473a76f-90c3-4421-889c-bb84fb3f986f\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f775lp" Jul 05 13:22:51.214016 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:22:51.213988 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4473a76f-90c3-4421-889c-bb84fb3f986f-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f775lp\" (UID: \"4473a76f-90c3-4421-889c-bb84fb3f986f\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f775lp" Jul 05 13:22:51.225877 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:22:51.225846 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4r8wb\" (UniqueName: \"kubernetes.io/projected/4473a76f-90c3-4421-889c-bb84fb3f986f-kube-api-access-4r8wb\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f775lp\" (UID: \"4473a76f-90c3-4421-889c-bb84fb3f986f\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f775lp" Jul 05 13:22:51.369128 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:22:51.369035 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f775lp" Jul 05 13:22:51.511008 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:22:51.510982 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f775lp"] Jul 05 13:22:51.513434 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:22:51.513403 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4473a76f_90c3_4421_889c_bb84fb3f986f.slice/crio-dcacab6d985193aaad2d1e91832242cf6525b3d68fdbf5be601a610ec0222c31 WatchSource:0}: Error finding container dcacab6d985193aaad2d1e91832242cf6525b3d68fdbf5be601a610ec0222c31: Status 404 returned error can't find the container with id dcacab6d985193aaad2d1e91832242cf6525b3d68fdbf5be601a610ec0222c31 Jul 05 13:22:51.714694 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:22:51.714597 2578 generic.go:356] "Generic (PLEG): container finished" podID="4473a76f-90c3-4421-889c-bb84fb3f986f" containerID="35cc3a6767e315fe2e67b983b87fd6d74f9315052a9fce1c793bc10693371c66" exitCode=0 Jul 05 13:22:51.714694 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:22:51.714674 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f775lp" event={"ID":"4473a76f-90c3-4421-889c-bb84fb3f986f","Type":"ContainerDied","Data":"35cc3a6767e315fe2e67b983b87fd6d74f9315052a9fce1c793bc10693371c66"} Jul 05 13:22:51.714858 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:22:51.714773 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f775lp" event={"ID":"4473a76f-90c3-4421-889c-bb84fb3f986f","Type":"ContainerStarted","Data":"dcacab6d985193aaad2d1e91832242cf6525b3d68fdbf5be601a610ec0222c31"} Jul 05 13:22:53.296777 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:22:53.296743 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-ds2k5"] Jul 05 13:22:53.302441 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:22:53.302410 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-ds2k5" Jul 05 13:22:53.308777 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:22:53.308749 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Jul 05 13:22:53.308925 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:22:53.308781 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Jul 05 13:22:53.308925 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:22:53.308754 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-qcknn\"" Jul 05 13:22:53.312271 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:22:53.312218 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-ds2k5"] Jul 05 13:22:53.332308 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:22:53.332266 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0e6f9264-4ae2-4567-8212-44355eaa09ba-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-ds2k5\" (UID: \"0e6f9264-4ae2-4567-8212-44355eaa09ba\") " pod="cert-manager/cert-manager-webhook-6486ffd997-ds2k5" Jul 05 13:22:53.332489 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:22:53.332339 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wb5c6\" (UniqueName: \"kubernetes.io/projected/0e6f9264-4ae2-4567-8212-44355eaa09ba-kube-api-access-wb5c6\") pod \"cert-manager-webhook-6486ffd997-ds2k5\" (UID: \"0e6f9264-4ae2-4567-8212-44355eaa09ba\") " pod="cert-manager/cert-manager-webhook-6486ffd997-ds2k5" Jul 05 13:22:53.433534 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:22:53.433497 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0e6f9264-4ae2-4567-8212-44355eaa09ba-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-ds2k5\" (UID: \"0e6f9264-4ae2-4567-8212-44355eaa09ba\") " pod="cert-manager/cert-manager-webhook-6486ffd997-ds2k5" Jul 05 13:22:53.433743 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:22:53.433571 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wb5c6\" (UniqueName: \"kubernetes.io/projected/0e6f9264-4ae2-4567-8212-44355eaa09ba-kube-api-access-wb5c6\") pod \"cert-manager-webhook-6486ffd997-ds2k5\" (UID: \"0e6f9264-4ae2-4567-8212-44355eaa09ba\") " pod="cert-manager/cert-manager-webhook-6486ffd997-ds2k5" Jul 05 13:22:53.445858 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:22:53.445821 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wb5c6\" (UniqueName: \"kubernetes.io/projected/0e6f9264-4ae2-4567-8212-44355eaa09ba-kube-api-access-wb5c6\") pod \"cert-manager-webhook-6486ffd997-ds2k5\" (UID: \"0e6f9264-4ae2-4567-8212-44355eaa09ba\") " pod="cert-manager/cert-manager-webhook-6486ffd997-ds2k5" Jul 05 13:22:53.446160 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:22:53.446126 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0e6f9264-4ae2-4567-8212-44355eaa09ba-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-ds2k5\" (UID: \"0e6f9264-4ae2-4567-8212-44355eaa09ba\") " pod="cert-manager/cert-manager-webhook-6486ffd997-ds2k5" Jul 05 13:22:53.620457 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:22:53.620360 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-ds2k5" Jul 05 13:22:53.878035 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:22:53.878006 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-ds2k5"] Jul 05 13:22:53.880755 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:22:53.880719 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0e6f9264_4ae2_4567_8212_44355eaa09ba.slice/crio-753f63e3c84a35a42cf890c0b9fa69d28ef28ec1d1f4e339d84cd30c65734a0b WatchSource:0}: Error finding container 753f63e3c84a35a42cf890c0b9fa69d28ef28ec1d1f4e339d84cd30c65734a0b: Status 404 returned error can't find the container with id 753f63e3c84a35a42cf890c0b9fa69d28ef28ec1d1f4e339d84cd30c65734a0b Jul 05 13:22:54.733390 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:22:54.733284 2578 generic.go:356] "Generic (PLEG): container finished" podID="4473a76f-90c3-4421-889c-bb84fb3f986f" containerID="d03734172c407f4adec7c07da31c117dd72d5cfac6ca866dcc76d3658607a4f5" exitCode=0 Jul 05 13:22:54.733834 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:22:54.733390 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f775lp" event={"ID":"4473a76f-90c3-4421-889c-bb84fb3f986f","Type":"ContainerDied","Data":"d03734172c407f4adec7c07da31c117dd72d5cfac6ca866dcc76d3658607a4f5"} Jul 05 13:22:54.735099 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:22:54.735048 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-ds2k5" event={"ID":"0e6f9264-4ae2-4567-8212-44355eaa09ba","Type":"ContainerStarted","Data":"753f63e3c84a35a42cf890c0b9fa69d28ef28ec1d1f4e339d84cd30c65734a0b"} Jul 05 13:22:54.811864 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:22:54.811831 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-5s4hd"] Jul 05 13:22:54.814368 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:22:54.814338 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-5s4hd" Jul 05 13:22:54.822436 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:22:54.822408 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-lk8nq\"" Jul 05 13:22:54.844315 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:22:54.844268 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-5s4hd"] Jul 05 13:22:54.846452 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:22:54.846401 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/c9eda5b5-6f77-4047-aa11-34fe5e71ec5b-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-5s4hd\" (UID: \"c9eda5b5-6f77-4047-aa11-34fe5e71ec5b\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-5s4hd" Jul 05 13:22:54.846452 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:22:54.846450 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-flfm8\" (UniqueName: \"kubernetes.io/projected/c9eda5b5-6f77-4047-aa11-34fe5e71ec5b-kube-api-access-flfm8\") pod \"cert-manager-cainjector-559cdd8c87-5s4hd\" (UID: \"c9eda5b5-6f77-4047-aa11-34fe5e71ec5b\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-5s4hd" Jul 05 13:22:54.947609 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:22:54.947555 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/c9eda5b5-6f77-4047-aa11-34fe5e71ec5b-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-5s4hd\" (UID: \"c9eda5b5-6f77-4047-aa11-34fe5e71ec5b\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-5s4hd" Jul 05 13:22:54.947809 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:22:54.947632 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-flfm8\" (UniqueName: \"kubernetes.io/projected/c9eda5b5-6f77-4047-aa11-34fe5e71ec5b-kube-api-access-flfm8\") pod \"cert-manager-cainjector-559cdd8c87-5s4hd\" (UID: \"c9eda5b5-6f77-4047-aa11-34fe5e71ec5b\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-5s4hd" Jul 05 13:22:54.956856 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:22:54.956819 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/c9eda5b5-6f77-4047-aa11-34fe5e71ec5b-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-5s4hd\" (UID: \"c9eda5b5-6f77-4047-aa11-34fe5e71ec5b\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-5s4hd" Jul 05 13:22:54.957009 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:22:54.956829 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-flfm8\" (UniqueName: \"kubernetes.io/projected/c9eda5b5-6f77-4047-aa11-34fe5e71ec5b-kube-api-access-flfm8\") pod \"cert-manager-cainjector-559cdd8c87-5s4hd\" (UID: \"c9eda5b5-6f77-4047-aa11-34fe5e71ec5b\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-5s4hd" Jul 05 13:22:55.125295 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:22:55.125258 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-5s4hd" Jul 05 13:22:55.284521 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:22:55.284492 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-5s4hd"] Jul 05 13:22:55.287219 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:22:55.287186 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc9eda5b5_6f77_4047_aa11_34fe5e71ec5b.slice/crio-a64e0000fae28b11d49aee64223538d8378cb0685cb86eea875f84b0200dc057 WatchSource:0}: Error finding container a64e0000fae28b11d49aee64223538d8378cb0685cb86eea875f84b0200dc057: Status 404 returned error can't find the container with id a64e0000fae28b11d49aee64223538d8378cb0685cb86eea875f84b0200dc057 Jul 05 13:22:55.742573 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:22:55.742536 2578 generic.go:356] "Generic (PLEG): container finished" podID="4473a76f-90c3-4421-889c-bb84fb3f986f" containerID="b7a599df5e836c55013ad8bc99631625a2c3c7b8cc07b794a0d9e8f3ea575fa8" exitCode=0 Jul 05 13:22:55.743051 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:22:55.742623 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f775lp" event={"ID":"4473a76f-90c3-4421-889c-bb84fb3f986f","Type":"ContainerDied","Data":"b7a599df5e836c55013ad8bc99631625a2c3c7b8cc07b794a0d9e8f3ea575fa8"} Jul 05 13:22:55.743999 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:22:55.743971 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-5s4hd" event={"ID":"c9eda5b5-6f77-4047-aa11-34fe5e71ec5b","Type":"ContainerStarted","Data":"a64e0000fae28b11d49aee64223538d8378cb0685cb86eea875f84b0200dc057"} Jul 05 13:22:56.750475 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:22:56.750441 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-ds2k5" event={"ID":"0e6f9264-4ae2-4567-8212-44355eaa09ba","Type":"ContainerStarted","Data":"9becbb2e114f30989fcbd5ce6b83e99f5ce66516b7afc44ad50b0fbb1479e128"} Jul 05 13:22:56.750923 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:22:56.750497 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-ds2k5" Jul 05 13:22:56.752338 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:22:56.752305 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-5s4hd" event={"ID":"c9eda5b5-6f77-4047-aa11-34fe5e71ec5b","Type":"ContainerStarted","Data":"7478b9b101689fc15379cff4835b18cf64c8c949e3fdf8a0b7360828fc0796c8"} Jul 05 13:22:56.783751 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:22:56.783151 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-ds2k5" podStartSLOduration=1.257784474 podStartE2EDuration="3.783124988s" totalImagesPullingTime="2.525340514s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 13:22:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 13:22:56.779317181 +0000 UTC m=+461.237880285" watchObservedRunningTime="2026-07-05 13:22:56.783124988 +0000 UTC m=+461.241688093" Jul 05 13:22:56.811745 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:22:56.811695 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-5s4hd" podStartSLOduration=1.6914725640000001 podStartE2EDuration="2.811679269s" totalImagesPullingTime="1.120206705s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 13:22:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 13:22:56.811047901 +0000 UTC m=+461.269611008" watchObservedRunningTime="2026-07-05 13:22:56.811679269 +0000 UTC m=+461.270242374" Jul 05 13:22:56.919177 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:22:56.919152 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f775lp" Jul 05 13:22:56.967786 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:22:56.967757 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4473a76f-90c3-4421-889c-bb84fb3f986f-util\") pod \"4473a76f-90c3-4421-889c-bb84fb3f986f\" (UID: \"4473a76f-90c3-4421-889c-bb84fb3f986f\") " Jul 05 13:22:56.967971 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:22:56.967840 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4r8wb\" (UniqueName: \"kubernetes.io/projected/4473a76f-90c3-4421-889c-bb84fb3f986f-kube-api-access-4r8wb\") pod \"4473a76f-90c3-4421-889c-bb84fb3f986f\" (UID: \"4473a76f-90c3-4421-889c-bb84fb3f986f\") " Jul 05 13:22:56.967971 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:22:56.967883 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4473a76f-90c3-4421-889c-bb84fb3f986f-bundle\") pod \"4473a76f-90c3-4421-889c-bb84fb3f986f\" (UID: \"4473a76f-90c3-4421-889c-bb84fb3f986f\") " Jul 05 13:22:56.968796 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:22:56.968380 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4473a76f-90c3-4421-889c-bb84fb3f986f-bundle" (OuterVolumeSpecName: "bundle") pod "4473a76f-90c3-4421-889c-bb84fb3f986f" (UID: "4473a76f-90c3-4421-889c-bb84fb3f986f"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 05 13:22:56.970717 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:22:56.970677 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4473a76f-90c3-4421-889c-bb84fb3f986f-kube-api-access-4r8wb" (OuterVolumeSpecName: "kube-api-access-4r8wb") pod "4473a76f-90c3-4421-889c-bb84fb3f986f" (UID: "4473a76f-90c3-4421-889c-bb84fb3f986f"). InnerVolumeSpecName "kube-api-access-4r8wb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 05 13:22:56.975527 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:22:56.975481 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4473a76f-90c3-4421-889c-bb84fb3f986f-util" (OuterVolumeSpecName: "util") pod "4473a76f-90c3-4421-889c-bb84fb3f986f" (UID: "4473a76f-90c3-4421-889c-bb84fb3f986f"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 05 13:22:57.068928 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:22:57.068843 2578 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4473a76f-90c3-4421-889c-bb84fb3f986f-bundle\") on node \"ip-10-0-130-191.ec2.internal\" DevicePath \"\"" Jul 05 13:22:57.068928 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:22:57.068873 2578 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4473a76f-90c3-4421-889c-bb84fb3f986f-util\") on node \"ip-10-0-130-191.ec2.internal\" DevicePath \"\"" Jul 05 13:22:57.068928 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:22:57.068882 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4r8wb\" (UniqueName: \"kubernetes.io/projected/4473a76f-90c3-4421-889c-bb84fb3f986f-kube-api-access-4r8wb\") on node \"ip-10-0-130-191.ec2.internal\" DevicePath \"\"" Jul 05 13:22:57.762023 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:22:57.761981 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f775lp" event={"ID":"4473a76f-90c3-4421-889c-bb84fb3f986f","Type":"ContainerDied","Data":"dcacab6d985193aaad2d1e91832242cf6525b3d68fdbf5be601a610ec0222c31"} Jul 05 13:22:57.762023 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:22:57.762021 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f775lp" Jul 05 13:22:57.762023 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:22:57.762033 2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="dcacab6d985193aaad2d1e91832242cf6525b3d68fdbf5be601a610ec0222c31" Jul 05 13:23:01.637433 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:23:01.637397 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-7k4xv"] Jul 05 13:23:01.637830 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:23:01.637724 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4473a76f-90c3-4421-889c-bb84fb3f986f" containerName="util" Jul 05 13:23:01.637830 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:23:01.637736 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="4473a76f-90c3-4421-889c-bb84fb3f986f" containerName="util" Jul 05 13:23:01.637830 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:23:01.637754 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4473a76f-90c3-4421-889c-bb84fb3f986f" containerName="pull" Jul 05 13:23:01.637830 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:23:01.637761 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="4473a76f-90c3-4421-889c-bb84fb3f986f" containerName="pull" Jul 05 13:23:01.637830 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:23:01.637771 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4473a76f-90c3-4421-889c-bb84fb3f986f" containerName="extract" Jul 05 13:23:01.637830 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:23:01.637776 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="4473a76f-90c3-4421-889c-bb84fb3f986f" containerName="extract" Jul 05 13:23:01.638035 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:23:01.637846 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="4473a76f-90c3-4421-889c-bb84fb3f986f" containerName="extract" Jul 05 13:23:01.641491 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:23:01.641476 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-7k4xv" Jul 05 13:23:01.644377 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:23:01.644359 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-mthn7\"" Jul 05 13:23:01.653028 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:23:01.653004 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-7k4xv"] Jul 05 13:23:01.707634 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:23:01.707568 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/3eb4dd94-1b13-4ca2-95e8-6d8358be8e4e-bound-sa-token\") pod \"cert-manager-55b4d954d5-7k4xv\" (UID: \"3eb4dd94-1b13-4ca2-95e8-6d8358be8e4e\") " pod="cert-manager/cert-manager-55b4d954d5-7k4xv" Jul 05 13:23:01.707796 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:23:01.707653 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v5ktd\" (UniqueName: \"kubernetes.io/projected/3eb4dd94-1b13-4ca2-95e8-6d8358be8e4e-kube-api-access-v5ktd\") pod \"cert-manager-55b4d954d5-7k4xv\" (UID: \"3eb4dd94-1b13-4ca2-95e8-6d8358be8e4e\") " pod="cert-manager/cert-manager-55b4d954d5-7k4xv" Jul 05 13:23:01.808421 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:23:01.808388 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/3eb4dd94-1b13-4ca2-95e8-6d8358be8e4e-bound-sa-token\") pod \"cert-manager-55b4d954d5-7k4xv\" (UID: \"3eb4dd94-1b13-4ca2-95e8-6d8358be8e4e\") " pod="cert-manager/cert-manager-55b4d954d5-7k4xv" Jul 05 13:23:01.808568 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:23:01.808439 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v5ktd\" (UniqueName: \"kubernetes.io/projected/3eb4dd94-1b13-4ca2-95e8-6d8358be8e4e-kube-api-access-v5ktd\") pod \"cert-manager-55b4d954d5-7k4xv\" (UID: \"3eb4dd94-1b13-4ca2-95e8-6d8358be8e4e\") " pod="cert-manager/cert-manager-55b4d954d5-7k4xv" Jul 05 13:23:01.817737 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:23:01.817704 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/3eb4dd94-1b13-4ca2-95e8-6d8358be8e4e-bound-sa-token\") pod \"cert-manager-55b4d954d5-7k4xv\" (UID: \"3eb4dd94-1b13-4ca2-95e8-6d8358be8e4e\") " pod="cert-manager/cert-manager-55b4d954d5-7k4xv" Jul 05 13:23:01.817892 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:23:01.817856 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v5ktd\" (UniqueName: \"kubernetes.io/projected/3eb4dd94-1b13-4ca2-95e8-6d8358be8e4e-kube-api-access-v5ktd\") pod \"cert-manager-55b4d954d5-7k4xv\" (UID: \"3eb4dd94-1b13-4ca2-95e8-6d8358be8e4e\") " pod="cert-manager/cert-manager-55b4d954d5-7k4xv" Jul 05 13:23:01.950226 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:23:01.950133 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-7k4xv" Jul 05 13:23:02.085908 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:23:02.085873 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-7k4xv"] Jul 05 13:23:02.088751 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:23:02.088720 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3eb4dd94_1b13_4ca2_95e8_6d8358be8e4e.slice/crio-eb090591c4edc60547c1f9dca4b938c1964d278a177d75985d4330ba6de617ea WatchSource:0}: Error finding container eb090591c4edc60547c1f9dca4b938c1964d278a177d75985d4330ba6de617ea: Status 404 returned error can't find the container with id eb090591c4edc60547c1f9dca4b938c1964d278a177d75985d4330ba6de617ea Jul 05 13:23:02.764519 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:23:02.764477 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-ds2k5" Jul 05 13:23:02.787850 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:23:02.787820 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-7k4xv" event={"ID":"3eb4dd94-1b13-4ca2-95e8-6d8358be8e4e","Type":"ContainerStarted","Data":"9ce58cb78da5db1bbcd2e4229828aaa426027b1c135c40688d0846eac6bfa34b"} Jul 05 13:23:02.788022 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:23:02.787856 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-7k4xv" event={"ID":"3eb4dd94-1b13-4ca2-95e8-6d8358be8e4e","Type":"ContainerStarted","Data":"eb090591c4edc60547c1f9dca4b938c1964d278a177d75985d4330ba6de617ea"} Jul 05 13:23:02.803175 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:23:02.803114 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-7k4xv" podStartSLOduration=1.8030912749999999 podStartE2EDuration="1.803091275s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 13:23:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 13:23:02.802608142 +0000 UTC m=+467.261171242" watchObservedRunningTime="2026-07-05 13:23:02.803091275 +0000 UTC m=+467.261654381" Jul 05 13:23:10.620129 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:23:10.620091 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgnf9z"] Jul 05 13:23:10.623365 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:23:10.623347 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgnf9z" Jul 05 13:23:10.625947 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:23:10.625914 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 05 13:23:10.626069 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:23:10.626013 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 05 13:23:10.626140 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:23:10.626126 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-m9dbt\"" Jul 05 13:23:10.632065 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:23:10.632042 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgnf9z"] Jul 05 13:23:10.683409 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:23:10.683375 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/48762660-66de-47bd-a76a-e01638e7ee74-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgnf9z\" (UID: \"48762660-66de-47bd-a76a-e01638e7ee74\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgnf9z" Jul 05 13:23:10.683409 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:23:10.683423 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6fdzx\" (UniqueName: \"kubernetes.io/projected/48762660-66de-47bd-a76a-e01638e7ee74-kube-api-access-6fdzx\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgnf9z\" (UID: \"48762660-66de-47bd-a76a-e01638e7ee74\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgnf9z" Jul 05 13:23:10.683687 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:23:10.683453 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/48762660-66de-47bd-a76a-e01638e7ee74-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgnf9z\" (UID: \"48762660-66de-47bd-a76a-e01638e7ee74\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgnf9z" Jul 05 13:23:10.784925 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:23:10.784878 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/48762660-66de-47bd-a76a-e01638e7ee74-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgnf9z\" (UID: \"48762660-66de-47bd-a76a-e01638e7ee74\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgnf9z" Jul 05 13:23:10.785125 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:23:10.784945 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6fdzx\" (UniqueName: \"kubernetes.io/projected/48762660-66de-47bd-a76a-e01638e7ee74-kube-api-access-6fdzx\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgnf9z\" (UID: \"48762660-66de-47bd-a76a-e01638e7ee74\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgnf9z" Jul 05 13:23:10.785125 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:23:10.784982 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/48762660-66de-47bd-a76a-e01638e7ee74-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgnf9z\" (UID: \"48762660-66de-47bd-a76a-e01638e7ee74\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgnf9z" Jul 05 13:23:10.785317 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:23:10.785295 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/48762660-66de-47bd-a76a-e01638e7ee74-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgnf9z\" (UID: \"48762660-66de-47bd-a76a-e01638e7ee74\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgnf9z" Jul 05 13:23:10.785384 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:23:10.785357 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/48762660-66de-47bd-a76a-e01638e7ee74-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgnf9z\" (UID: \"48762660-66de-47bd-a76a-e01638e7ee74\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgnf9z" Jul 05 13:23:10.793508 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:23:10.793481 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6fdzx\" (UniqueName: \"kubernetes.io/projected/48762660-66de-47bd-a76a-e01638e7ee74-kube-api-access-6fdzx\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgnf9z\" (UID: \"48762660-66de-47bd-a76a-e01638e7ee74\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgnf9z" Jul 05 13:23:10.934224 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:23:10.934139 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgnf9z" Jul 05 13:23:11.070899 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:23:11.070873 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgnf9z"] Jul 05 13:23:11.073196 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:23:11.073169 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod48762660_66de_47bd_a76a_e01638e7ee74.slice/crio-cdff831909755b3d8979797e74cb7b8b2ab687306945ccb3b0de6f1be15b6697 WatchSource:0}: Error finding container cdff831909755b3d8979797e74cb7b8b2ab687306945ccb3b0de6f1be15b6697: Status 404 returned error can't find the container with id cdff831909755b3d8979797e74cb7b8b2ab687306945ccb3b0de6f1be15b6697 Jul 05 13:23:11.830910 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:23:11.830872 2578 generic.go:356] "Generic (PLEG): container finished" podID="48762660-66de-47bd-a76a-e01638e7ee74" containerID="5a5d70186ab3cf652cda6c577c3db534f833354028f12e2b65757ed81704fa3e" exitCode=0 Jul 05 13:23:11.831340 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:23:11.830956 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgnf9z" event={"ID":"48762660-66de-47bd-a76a-e01638e7ee74","Type":"ContainerDied","Data":"5a5d70186ab3cf652cda6c577c3db534f833354028f12e2b65757ed81704fa3e"} Jul 05 13:23:11.831340 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:23:11.831031 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgnf9z" event={"ID":"48762660-66de-47bd-a76a-e01638e7ee74","Type":"ContainerStarted","Data":"cdff831909755b3d8979797e74cb7b8b2ab687306945ccb3b0de6f1be15b6697"} Jul 05 13:23:12.839032 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:23:12.838984 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgnf9z" event={"ID":"48762660-66de-47bd-a76a-e01638e7ee74","Type":"ContainerStarted","Data":"b9b67531ef960530f46e040a0a16bc366321deb0290911f6977a8c9aadd7efc1"} Jul 05 13:23:13.845259 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:23:13.845219 2578 generic.go:356] "Generic (PLEG): container finished" podID="48762660-66de-47bd-a76a-e01638e7ee74" containerID="b9b67531ef960530f46e040a0a16bc366321deb0290911f6977a8c9aadd7efc1" exitCode=0 Jul 05 13:23:13.845744 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:23:13.845302 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgnf9z" event={"ID":"48762660-66de-47bd-a76a-e01638e7ee74","Type":"ContainerDied","Data":"b9b67531ef960530f46e040a0a16bc366321deb0290911f6977a8c9aadd7efc1"} Jul 05 13:23:14.852806 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:23:14.852765 2578 generic.go:356] "Generic (PLEG): container finished" podID="48762660-66de-47bd-a76a-e01638e7ee74" containerID="0be2401dce17efb24684dfca4c65ec07f3e10c38ed318361cc48616d57da8b71" exitCode=0 Jul 05 13:23:14.853228 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:23:14.852856 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgnf9z" event={"ID":"48762660-66de-47bd-a76a-e01638e7ee74","Type":"ContainerDied","Data":"0be2401dce17efb24684dfca4c65ec07f3e10c38ed318361cc48616d57da8b71"} Jul 05 13:23:15.987603 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:23:15.987559 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgnf9z" Jul 05 13:23:16.138395 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:23:16.138300 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/48762660-66de-47bd-a76a-e01638e7ee74-util\") pod \"48762660-66de-47bd-a76a-e01638e7ee74\" (UID: \"48762660-66de-47bd-a76a-e01638e7ee74\") " Jul 05 13:23:16.138572 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:23:16.138417 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/48762660-66de-47bd-a76a-e01638e7ee74-bundle\") pod \"48762660-66de-47bd-a76a-e01638e7ee74\" (UID: \"48762660-66de-47bd-a76a-e01638e7ee74\") " Jul 05 13:23:16.138572 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:23:16.138473 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6fdzx\" (UniqueName: \"kubernetes.io/projected/48762660-66de-47bd-a76a-e01638e7ee74-kube-api-access-6fdzx\") pod \"48762660-66de-47bd-a76a-e01638e7ee74\" (UID: \"48762660-66de-47bd-a76a-e01638e7ee74\") " Jul 05 13:23:16.139171 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:23:16.139141 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/48762660-66de-47bd-a76a-e01638e7ee74-bundle" (OuterVolumeSpecName: "bundle") pod "48762660-66de-47bd-a76a-e01638e7ee74" (UID: "48762660-66de-47bd-a76a-e01638e7ee74"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 05 13:23:16.140699 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:23:16.140675 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/48762660-66de-47bd-a76a-e01638e7ee74-kube-api-access-6fdzx" (OuterVolumeSpecName: "kube-api-access-6fdzx") pod "48762660-66de-47bd-a76a-e01638e7ee74" (UID: "48762660-66de-47bd-a76a-e01638e7ee74"). InnerVolumeSpecName "kube-api-access-6fdzx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 05 13:23:16.144559 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:23:16.144539 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/48762660-66de-47bd-a76a-e01638e7ee74-util" (OuterVolumeSpecName: "util") pod "48762660-66de-47bd-a76a-e01638e7ee74" (UID: "48762660-66de-47bd-a76a-e01638e7ee74"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 05 13:23:16.240337 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:23:16.240284 2578 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/48762660-66de-47bd-a76a-e01638e7ee74-bundle\") on node \"ip-10-0-130-191.ec2.internal\" DevicePath \"\"" Jul 05 13:23:16.240337 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:23:16.240335 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6fdzx\" (UniqueName: \"kubernetes.io/projected/48762660-66de-47bd-a76a-e01638e7ee74-kube-api-access-6fdzx\") on node \"ip-10-0-130-191.ec2.internal\" DevicePath \"\"" Jul 05 13:23:16.240528 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:23:16.240352 2578 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/48762660-66de-47bd-a76a-e01638e7ee74-util\") on node \"ip-10-0-130-191.ec2.internal\" DevicePath \"\"" Jul 05 13:23:16.864643 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:23:16.864612 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgnf9z" event={"ID":"48762660-66de-47bd-a76a-e01638e7ee74","Type":"ContainerDied","Data":"cdff831909755b3d8979797e74cb7b8b2ab687306945ccb3b0de6f1be15b6697"} Jul 05 13:23:16.864643 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:23:16.864653 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgnf9z" Jul 05 13:23:16.864852 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:23:16.864659 2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="cdff831909755b3d8979797e74cb7b8b2ab687306945ccb3b0de6f1be15b6697" Jul 05 13:23:27.907262 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:23:27.907226 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6674ff8b4f-rxh62"] Jul 05 13:23:27.907814 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:23:27.907755 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="48762660-66de-47bd-a76a-e01638e7ee74" containerName="extract" Jul 05 13:23:27.907814 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:23:27.907774 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="48762660-66de-47bd-a76a-e01638e7ee74" containerName="extract" Jul 05 13:23:27.907814 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:23:27.907788 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="48762660-66de-47bd-a76a-e01638e7ee74" containerName="pull" Jul 05 13:23:27.907814 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:23:27.907796 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="48762660-66de-47bd-a76a-e01638e7ee74" containerName="pull" Jul 05 13:23:27.907814 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:23:27.907813 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="48762660-66de-47bd-a76a-e01638e7ee74" containerName="util" Jul 05 13:23:27.908070 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:23:27.907822 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="48762660-66de-47bd-a76a-e01638e7ee74" containerName="util" Jul 05 13:23:27.908070 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:23:27.907916 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="48762660-66de-47bd-a76a-e01638e7ee74" containerName="extract" Jul 05 13:23:27.912519 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:23:27.912496 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-6674ff8b4f-rxh62" Jul 05 13:23:27.920076 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:23:27.920050 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 05 13:23:27.921342 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:23:27.921321 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 05 13:23:27.921472 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:23:27.921445 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-dk7w4\"" Jul 05 13:23:27.922896 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:23:27.922854 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 05 13:23:27.923005 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:23:27.922910 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 05 13:23:27.936367 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:23:27.936339 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/77458566-bec5-4550-8030-bf1f43f00555-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6674ff8b4f-rxh62\" (UID: \"77458566-bec5-4550-8030-bf1f43f00555\") " pod="opendatahub/opendatahub-operator-controller-manager-6674ff8b4f-rxh62" Jul 05 13:23:27.936367 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:23:27.936373 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/77458566-bec5-4550-8030-bf1f43f00555-tmp\") pod \"opendatahub-operator-controller-manager-6674ff8b4f-rxh62\" (UID: \"77458566-bec5-4550-8030-bf1f43f00555\") " pod="opendatahub/opendatahub-operator-controller-manager-6674ff8b4f-rxh62" Jul 05 13:23:27.936551 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:23:27.936391 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/77458566-bec5-4550-8030-bf1f43f00555-webhook-cert\") pod \"opendatahub-operator-controller-manager-6674ff8b4f-rxh62\" (UID: \"77458566-bec5-4550-8030-bf1f43f00555\") " pod="opendatahub/opendatahub-operator-controller-manager-6674ff8b4f-rxh62" Jul 05 13:23:27.936551 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:23:27.936493 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/77458566-bec5-4550-8030-bf1f43f00555-manifests\") pod \"opendatahub-operator-controller-manager-6674ff8b4f-rxh62\" (UID: \"77458566-bec5-4550-8030-bf1f43f00555\") " pod="opendatahub/opendatahub-operator-controller-manager-6674ff8b4f-rxh62" Jul 05 13:23:27.936551 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:23:27.936515 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2f8dn\" (UniqueName: \"kubernetes.io/projected/77458566-bec5-4550-8030-bf1f43f00555-kube-api-access-2f8dn\") pod \"opendatahub-operator-controller-manager-6674ff8b4f-rxh62\" (UID: \"77458566-bec5-4550-8030-bf1f43f00555\") " pod="opendatahub/opendatahub-operator-controller-manager-6674ff8b4f-rxh62" Jul 05 13:23:27.955539 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:23:27.955507 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6674ff8b4f-rxh62"] Jul 05 13:23:28.037821 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:23:28.037788 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2f8dn\" (UniqueName: \"kubernetes.io/projected/77458566-bec5-4550-8030-bf1f43f00555-kube-api-access-2f8dn\") pod \"opendatahub-operator-controller-manager-6674ff8b4f-rxh62\" (UID: \"77458566-bec5-4550-8030-bf1f43f00555\") " pod="opendatahub/opendatahub-operator-controller-manager-6674ff8b4f-rxh62" Jul 05 13:23:28.038015 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:23:28.037832 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/77458566-bec5-4550-8030-bf1f43f00555-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6674ff8b4f-rxh62\" (UID: \"77458566-bec5-4550-8030-bf1f43f00555\") " pod="opendatahub/opendatahub-operator-controller-manager-6674ff8b4f-rxh62" Jul 05 13:23:28.038015 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:23:28.037854 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/77458566-bec5-4550-8030-bf1f43f00555-tmp\") pod \"opendatahub-operator-controller-manager-6674ff8b4f-rxh62\" (UID: \"77458566-bec5-4550-8030-bf1f43f00555\") " pod="opendatahub/opendatahub-operator-controller-manager-6674ff8b4f-rxh62" Jul 05 13:23:28.038015 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:23:28.037882 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/77458566-bec5-4550-8030-bf1f43f00555-webhook-cert\") pod \"opendatahub-operator-controller-manager-6674ff8b4f-rxh62\" (UID: \"77458566-bec5-4550-8030-bf1f43f00555\") " pod="opendatahub/opendatahub-operator-controller-manager-6674ff8b4f-rxh62" Jul 05 13:23:28.038015 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:23:28.037957 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/77458566-bec5-4550-8030-bf1f43f00555-manifests\") pod \"opendatahub-operator-controller-manager-6674ff8b4f-rxh62\" (UID: \"77458566-bec5-4550-8030-bf1f43f00555\") " pod="opendatahub/opendatahub-operator-controller-manager-6674ff8b4f-rxh62" Jul 05 13:23:28.038435 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:23:28.038401 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/77458566-bec5-4550-8030-bf1f43f00555-manifests\") pod \"opendatahub-operator-controller-manager-6674ff8b4f-rxh62\" (UID: \"77458566-bec5-4550-8030-bf1f43f00555\") " pod="opendatahub/opendatahub-operator-controller-manager-6674ff8b4f-rxh62" Jul 05 13:23:28.040473 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:23:28.040447 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/77458566-bec5-4550-8030-bf1f43f00555-tmp\") pod \"opendatahub-operator-controller-manager-6674ff8b4f-rxh62\" (UID: \"77458566-bec5-4550-8030-bf1f43f00555\") " pod="opendatahub/opendatahub-operator-controller-manager-6674ff8b4f-rxh62" Jul 05 13:23:28.040608 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:23:28.040461 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/77458566-bec5-4550-8030-bf1f43f00555-webhook-cert\") pod \"opendatahub-operator-controller-manager-6674ff8b4f-rxh62\" (UID: \"77458566-bec5-4550-8030-bf1f43f00555\") " pod="opendatahub/opendatahub-operator-controller-manager-6674ff8b4f-rxh62" Jul 05 13:23:28.040608 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:23:28.040486 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/77458566-bec5-4550-8030-bf1f43f00555-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6674ff8b4f-rxh62\" (UID: \"77458566-bec5-4550-8030-bf1f43f00555\") " pod="opendatahub/opendatahub-operator-controller-manager-6674ff8b4f-rxh62" Jul 05 13:23:28.071927 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:23:28.071894 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2f8dn\" (UniqueName: \"kubernetes.io/projected/77458566-bec5-4550-8030-bf1f43f00555-kube-api-access-2f8dn\") pod \"opendatahub-operator-controller-manager-6674ff8b4f-rxh62\" (UID: \"77458566-bec5-4550-8030-bf1f43f00555\") " pod="opendatahub/opendatahub-operator-controller-manager-6674ff8b4f-rxh62" Jul 05 13:23:28.224265 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:23:28.224180 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-6674ff8b4f-rxh62" Jul 05 13:23:28.374153 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:23:28.374125 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6674ff8b4f-rxh62"] Jul 05 13:23:28.377632 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:23:28.377601 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod77458566_bec5_4550_8030_bf1f43f00555.slice/crio-757b911da23d2255ea64a9c6995ff84cb92f3d63032ae3d5118a1ba3806593ff WatchSource:0}: Error finding container 757b911da23d2255ea64a9c6995ff84cb92f3d63032ae3d5118a1ba3806593ff: Status 404 returned error can't find the container with id 757b911da23d2255ea64a9c6995ff84cb92f3d63032ae3d5118a1ba3806593ff Jul 05 13:23:28.925078 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:23:28.925031 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6674ff8b4f-rxh62" event={"ID":"77458566-bec5-4550-8030-bf1f43f00555","Type":"ContainerStarted","Data":"757b911da23d2255ea64a9c6995ff84cb92f3d63032ae3d5118a1ba3806593ff"} Jul 05 13:23:31.943154 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:23:31.943117 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6674ff8b4f-rxh62" event={"ID":"77458566-bec5-4550-8030-bf1f43f00555","Type":"ContainerStarted","Data":"5c1628b457f50a5105ad782d09a21582b3728e52b4a79e3e52dc6cf57edda5a9"} Jul 05 13:23:32.949437 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:23:32.949403 2578 generic.go:356] "Generic (PLEG): container finished" podID="77458566-bec5-4550-8030-bf1f43f00555" containerID="5c1628b457f50a5105ad782d09a21582b3728e52b4a79e3e52dc6cf57edda5a9" exitCode=0 Jul 05 13:23:32.949847 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:23:32.949477 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6674ff8b4f-rxh62" event={"ID":"77458566-bec5-4550-8030-bf1f43f00555","Type":"ContainerDied","Data":"5c1628b457f50a5105ad782d09a21582b3728e52b4a79e3e52dc6cf57edda5a9"} Jul 05 13:23:33.957187 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:23:33.957145 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6674ff8b4f-rxh62" event={"ID":"77458566-bec5-4550-8030-bf1f43f00555","Type":"ContainerStarted","Data":"19ac32f3278215b3242bc9172773aaefaa16ff7d009e1bc792ae5fc3f1204b04"} Jul 05 13:23:33.957685 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:23:33.957250 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-6674ff8b4f-rxh62" Jul 05 13:23:33.994929 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:23:33.994869 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-6674ff8b4f-rxh62" podStartSLOduration=2.862331456 podStartE2EDuration="6.994851131s" totalImagesPullingTime="3.058538476s" totalInitContainerRuntime="1.073981199s" isStatefulPod=false podCreationTimestamp="2026-07-05 13:23:27 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 13:23:33.991769971 +0000 UTC m=+498.450333089" watchObservedRunningTime="2026-07-05 13:23:33.994851131 +0000 UTC m=+498.453414236" Jul 05 13:23:44.965223 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:23:44.965192 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-6674ff8b4f-rxh62" Jul 05 13:23:48.463958 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:23:48.463921 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-7bcdf4856b-r2bgp"] Jul 05 13:23:48.467672 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:23:48.467643 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-7bcdf4856b-r2bgp" Jul 05 13:23:48.473283 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:23:48.473255 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 05 13:23:48.473425 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:23:48.473255 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 05 13:23:48.474642 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:23:48.474476 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-6mmzt\"" Jul 05 13:23:48.474717 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:23:48.474475 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Jul 05 13:23:48.474991 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:23:48.474969 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Jul 05 13:23:48.482808 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:23:48.482788 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-7bcdf4856b-r2bgp"] Jul 05 13:23:48.522301 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:23:48.522270 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ceedc936-a8ca-4855-9484-675b40b24330-tls-certs\") pod \"kube-auth-proxy-7bcdf4856b-r2bgp\" (UID: \"ceedc936-a8ca-4855-9484-675b40b24330\") " pod="openshift-ingress/kube-auth-proxy-7bcdf4856b-r2bgp" Jul 05 13:23:48.522477 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:23:48.522337 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ceedc936-a8ca-4855-9484-675b40b24330-tmp\") pod \"kube-auth-proxy-7bcdf4856b-r2bgp\" (UID: \"ceedc936-a8ca-4855-9484-675b40b24330\") " pod="openshift-ingress/kube-auth-proxy-7bcdf4856b-r2bgp" Jul 05 13:23:48.522477 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:23:48.522405 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8vtjf\" (UniqueName: \"kubernetes.io/projected/ceedc936-a8ca-4855-9484-675b40b24330-kube-api-access-8vtjf\") pod \"kube-auth-proxy-7bcdf4856b-r2bgp\" (UID: \"ceedc936-a8ca-4855-9484-675b40b24330\") " pod="openshift-ingress/kube-auth-proxy-7bcdf4856b-r2bgp" Jul 05 13:23:48.623442 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:23:48.623401 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8vtjf\" (UniqueName: \"kubernetes.io/projected/ceedc936-a8ca-4855-9484-675b40b24330-kube-api-access-8vtjf\") pod \"kube-auth-proxy-7bcdf4856b-r2bgp\" (UID: \"ceedc936-a8ca-4855-9484-675b40b24330\") " pod="openshift-ingress/kube-auth-proxy-7bcdf4856b-r2bgp" Jul 05 13:23:48.623679 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:23:48.623460 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ceedc936-a8ca-4855-9484-675b40b24330-tls-certs\") pod \"kube-auth-proxy-7bcdf4856b-r2bgp\" (UID: \"ceedc936-a8ca-4855-9484-675b40b24330\") " pod="openshift-ingress/kube-auth-proxy-7bcdf4856b-r2bgp" Jul 05 13:23:48.623679 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:23:48.623513 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ceedc936-a8ca-4855-9484-675b40b24330-tmp\") pod \"kube-auth-proxy-7bcdf4856b-r2bgp\" (UID: \"ceedc936-a8ca-4855-9484-675b40b24330\") " pod="openshift-ingress/kube-auth-proxy-7bcdf4856b-r2bgp" Jul 05 13:23:48.623808 ip-10-0-130-191 kubenswrapper[2578]: E0705 13:23:48.623675 2578 secret.go:189] Couldn't get secret openshift-ingress/kube-auth-proxy-tls: secret "kube-auth-proxy-tls" not found Jul 05 13:23:48.623808 ip-10-0-130-191 kubenswrapper[2578]: E0705 13:23:48.623749 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ceedc936-a8ca-4855-9484-675b40b24330-tls-certs podName:ceedc936-a8ca-4855-9484-675b40b24330 nodeName:}" failed. No retries permitted until 2026-07-05 13:23:49.12373253 +0000 UTC m=+513.582295616 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/ceedc936-a8ca-4855-9484-675b40b24330-tls-certs") pod "kube-auth-proxy-7bcdf4856b-r2bgp" (UID: "ceedc936-a8ca-4855-9484-675b40b24330") : secret "kube-auth-proxy-tls" not found Jul 05 13:23:48.626049 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:23:48.626027 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ceedc936-a8ca-4855-9484-675b40b24330-tmp\") pod \"kube-auth-proxy-7bcdf4856b-r2bgp\" (UID: \"ceedc936-a8ca-4855-9484-675b40b24330\") " pod="openshift-ingress/kube-auth-proxy-7bcdf4856b-r2bgp" Jul 05 13:23:48.647269 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:23:48.647233 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8vtjf\" (UniqueName: \"kubernetes.io/projected/ceedc936-a8ca-4855-9484-675b40b24330-kube-api-access-8vtjf\") pod \"kube-auth-proxy-7bcdf4856b-r2bgp\" (UID: \"ceedc936-a8ca-4855-9484-675b40b24330\") " pod="openshift-ingress/kube-auth-proxy-7bcdf4856b-r2bgp" Jul 05 13:23:49.128327 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:23:49.128290 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ceedc936-a8ca-4855-9484-675b40b24330-tls-certs\") pod \"kube-auth-proxy-7bcdf4856b-r2bgp\" (UID: \"ceedc936-a8ca-4855-9484-675b40b24330\") " pod="openshift-ingress/kube-auth-proxy-7bcdf4856b-r2bgp" Jul 05 13:23:49.130824 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:23:49.130791 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ceedc936-a8ca-4855-9484-675b40b24330-tls-certs\") pod \"kube-auth-proxy-7bcdf4856b-r2bgp\" (UID: \"ceedc936-a8ca-4855-9484-675b40b24330\") " pod="openshift-ingress/kube-auth-proxy-7bcdf4856b-r2bgp" Jul 05 13:23:49.378016 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:23:49.377971 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-7bcdf4856b-r2bgp" Jul 05 13:23:49.545756 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:23:49.545733 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-7bcdf4856b-r2bgp"] Jul 05 13:23:49.547960 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:23:49.547933 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podceedc936_a8ca_4855_9484_675b40b24330.slice/crio-4907660cba71c821d57de6a7be45fc28e2c25b174800ef23c1e7fb744b6c059a WatchSource:0}: Error finding container 4907660cba71c821d57de6a7be45fc28e2c25b174800ef23c1e7fb744b6c059a: Status 404 returned error can't find the container with id 4907660cba71c821d57de6a7be45fc28e2c25b174800ef23c1e7fb744b6c059a Jul 05 13:23:50.031120 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:23:50.031077 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-7bcdf4856b-r2bgp" event={"ID":"ceedc936-a8ca-4855-9484-675b40b24330","Type":"ContainerStarted","Data":"4907660cba71c821d57de6a7be45fc28e2c25b174800ef23c1e7fb744b6c059a"} Jul 05 13:23:52.043990 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:23:52.043884 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-7bcdf4856b-r2bgp" event={"ID":"ceedc936-a8ca-4855-9484-675b40b24330","Type":"ContainerStarted","Data":"647febef6f67f8a908ac03f983549e7dcf497b64155f496794794b069888f07f"} Jul 05 13:23:52.086862 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:23:52.086808 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-7bcdf4856b-r2bgp" podStartSLOduration=1.849180423 podStartE2EDuration="4.086793946s" totalImagesPullingTime="2.237613523s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 13:23:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 13:23:52.085248522 +0000 UTC m=+516.543811626" watchObservedRunningTime="2026-07-05 13:23:52.086793946 +0000 UTC m=+516.545357050" Jul 05 13:23:57.126526 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:23:57.126493 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-687b6f4bf4-g56mp"] Jul 05 13:23:57.133965 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:23:57.133933 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-687b6f4bf4-g56mp" Jul 05 13:23:57.142819 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:23:57.142799 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Jul 05 13:23:57.143783 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:23:57.143768 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Jul 05 13:23:57.144467 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:23:57.144454 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-sjwvg\"" Jul 05 13:23:57.144509 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:23:57.144457 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 05 13:23:57.148070 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:23:57.148056 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 05 13:23:57.154386 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:23:57.154369 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Jul 05 13:23:57.167008 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:23:57.166985 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-687b6f4bf4-g56mp"] Jul 05 13:23:57.305637 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:23:57.305572 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cz7sn\" (UniqueName: \"kubernetes.io/projected/e35137df-801a-4327-8fb7-54eb69d4a72d-kube-api-access-cz7sn\") pod \"lws-controller-manager-687b6f4bf4-g56mp\" (UID: \"e35137df-801a-4327-8fb7-54eb69d4a72d\") " pod="openshift-lws-operator/lws-controller-manager-687b6f4bf4-g56mp" Jul 05 13:23:57.305807 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:23:57.305671 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/e35137df-801a-4327-8fb7-54eb69d4a72d-manager-config\") pod \"lws-controller-manager-687b6f4bf4-g56mp\" (UID: \"e35137df-801a-4327-8fb7-54eb69d4a72d\") " pod="openshift-lws-operator/lws-controller-manager-687b6f4bf4-g56mp" Jul 05 13:23:57.305807 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:23:57.305720 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e35137df-801a-4327-8fb7-54eb69d4a72d-cert\") pod \"lws-controller-manager-687b6f4bf4-g56mp\" (UID: \"e35137df-801a-4327-8fb7-54eb69d4a72d\") " pod="openshift-lws-operator/lws-controller-manager-687b6f4bf4-g56mp" Jul 05 13:23:57.305807 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:23:57.305744 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/e35137df-801a-4327-8fb7-54eb69d4a72d-metrics-cert\") pod \"lws-controller-manager-687b6f4bf4-g56mp\" (UID: \"e35137df-801a-4327-8fb7-54eb69d4a72d\") " pod="openshift-lws-operator/lws-controller-manager-687b6f4bf4-g56mp" Jul 05 13:23:57.406850 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:23:57.406745 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e35137df-801a-4327-8fb7-54eb69d4a72d-cert\") pod \"lws-controller-manager-687b6f4bf4-g56mp\" (UID: \"e35137df-801a-4327-8fb7-54eb69d4a72d\") " pod="openshift-lws-operator/lws-controller-manager-687b6f4bf4-g56mp" Jul 05 13:23:57.406850 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:23:57.406800 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/e35137df-801a-4327-8fb7-54eb69d4a72d-metrics-cert\") pod \"lws-controller-manager-687b6f4bf4-g56mp\" (UID: \"e35137df-801a-4327-8fb7-54eb69d4a72d\") " pod="openshift-lws-operator/lws-controller-manager-687b6f4bf4-g56mp" Jul 05 13:23:57.407099 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:23:57.406863 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cz7sn\" (UniqueName: \"kubernetes.io/projected/e35137df-801a-4327-8fb7-54eb69d4a72d-kube-api-access-cz7sn\") pod \"lws-controller-manager-687b6f4bf4-g56mp\" (UID: \"e35137df-801a-4327-8fb7-54eb69d4a72d\") " pod="openshift-lws-operator/lws-controller-manager-687b6f4bf4-g56mp" Jul 05 13:23:57.407099 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:23:57.406899 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/e35137df-801a-4327-8fb7-54eb69d4a72d-manager-config\") pod \"lws-controller-manager-687b6f4bf4-g56mp\" (UID: \"e35137df-801a-4327-8fb7-54eb69d4a72d\") " pod="openshift-lws-operator/lws-controller-manager-687b6f4bf4-g56mp" Jul 05 13:23:57.407612 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:23:57.407559 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/e35137df-801a-4327-8fb7-54eb69d4a72d-manager-config\") pod \"lws-controller-manager-687b6f4bf4-g56mp\" (UID: \"e35137df-801a-4327-8fb7-54eb69d4a72d\") " pod="openshift-lws-operator/lws-controller-manager-687b6f4bf4-g56mp" Jul 05 13:23:57.409439 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:23:57.409422 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e35137df-801a-4327-8fb7-54eb69d4a72d-cert\") pod \"lws-controller-manager-687b6f4bf4-g56mp\" (UID: \"e35137df-801a-4327-8fb7-54eb69d4a72d\") " pod="openshift-lws-operator/lws-controller-manager-687b6f4bf4-g56mp" Jul 05 13:23:57.409496 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:23:57.409467 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/e35137df-801a-4327-8fb7-54eb69d4a72d-metrics-cert\") pod \"lws-controller-manager-687b6f4bf4-g56mp\" (UID: \"e35137df-801a-4327-8fb7-54eb69d4a72d\") " pod="openshift-lws-operator/lws-controller-manager-687b6f4bf4-g56mp" Jul 05 13:23:57.420465 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:23:57.420444 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cz7sn\" (UniqueName: \"kubernetes.io/projected/e35137df-801a-4327-8fb7-54eb69d4a72d-kube-api-access-cz7sn\") pod \"lws-controller-manager-687b6f4bf4-g56mp\" (UID: \"e35137df-801a-4327-8fb7-54eb69d4a72d\") " pod="openshift-lws-operator/lws-controller-manager-687b6f4bf4-g56mp" Jul 05 13:23:57.443186 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:23:57.443158 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-687b6f4bf4-g56mp" Jul 05 13:23:57.640111 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:23:57.640088 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-687b6f4bf4-g56mp"] Jul 05 13:23:57.642991 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:23:57.642964 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode35137df_801a_4327_8fb7_54eb69d4a72d.slice/crio-f7a43841554ab806c12c3532985c65c3b2fc6cfdca9ed8eb478bbf423bda8f01 WatchSource:0}: Error finding container f7a43841554ab806c12c3532985c65c3b2fc6cfdca9ed8eb478bbf423bda8f01: Status 404 returned error can't find the container with id f7a43841554ab806c12c3532985c65c3b2fc6cfdca9ed8eb478bbf423bda8f01 Jul 05 13:23:58.071425 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:23:58.071395 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-687b6f4bf4-g56mp" event={"ID":"e35137df-801a-4327-8fb7-54eb69d4a72d","Type":"ContainerStarted","Data":"f7a43841554ab806c12c3532985c65c3b2fc6cfdca9ed8eb478bbf423bda8f01"} Jul 05 13:24:12.140072 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:24:12.139984 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-687b6f4bf4-g56mp" event={"ID":"e35137df-801a-4327-8fb7-54eb69d4a72d","Type":"ContainerStarted","Data":"0a4e0ce63a0fa193ecaadbb93d0a3accf1c245c65f6dd177dd66fc0863e29487"} Jul 05 13:24:12.140484 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:24:12.140170 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-687b6f4bf4-g56mp" Jul 05 13:24:12.253810 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:24:12.253756 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-687b6f4bf4-g56mp" podStartSLOduration=1.129709925 podStartE2EDuration="15.253741353s" totalImagesPullingTime="14.124031428s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 13:23:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 13:24:12.252282972 +0000 UTC m=+536.710846078" watchObservedRunningTime="2026-07-05 13:24:12.253741353 +0000 UTC m=+536.712304457" Jul 05 13:24:23.146658 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:24:23.146626 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-687b6f4bf4-g56mp" Jul 05 13:25:15.941217 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:25:15.941186 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-tg2xx_1c214417-9bef-4007-ab01-e61b84d1b35e/console-operator/1.log" Jul 05 13:25:15.941718 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:25:15.941648 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-tg2xx_1c214417-9bef-4007-ab01-e61b84d1b35e/console-operator/1.log" Jul 05 13:25:15.948438 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:25:15.948417 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-656m6_7bdbd204-0fb7-4ebf-af05-432c370a814b/ovn-acl-logging/0.log" Jul 05 13:25:15.949200 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:25:15.949178 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-656m6_7bdbd204-0fb7-4ebf-af05-432c370a814b/ovn-acl-logging/0.log" Jul 05 13:25:25.859540 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:25:25.859502 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73pfsr7"] Jul 05 13:25:25.862802 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:25:25.862786 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73pfsr7" Jul 05 13:25:25.868111 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:25:25.868085 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 05 13:25:25.868429 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:25:25.868408 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-vkw6q\"" Jul 05 13:25:25.873360 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:25:25.873340 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 05 13:25:25.948662 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:25:25.948620 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73pfsr7"] Jul 05 13:25:25.964129 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:25:25.964099 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/617eb3c5-e598-495a-bcca-73f0ac7bf7cd-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73pfsr7\" (UID: \"617eb3c5-e598-495a-bcca-73f0ac7bf7cd\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73pfsr7" Jul 05 13:25:25.964129 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:25:25.964132 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-598dc\" (UniqueName: \"kubernetes.io/projected/617eb3c5-e598-495a-bcca-73f0ac7bf7cd-kube-api-access-598dc\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73pfsr7\" (UID: \"617eb3c5-e598-495a-bcca-73f0ac7bf7cd\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73pfsr7" Jul 05 13:25:25.964370 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:25:25.964182 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/617eb3c5-e598-495a-bcca-73f0ac7bf7cd-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73pfsr7\" (UID: \"617eb3c5-e598-495a-bcca-73f0ac7bf7cd\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73pfsr7" Jul 05 13:25:26.065205 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:25:26.065170 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/617eb3c5-e598-495a-bcca-73f0ac7bf7cd-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73pfsr7\" (UID: \"617eb3c5-e598-495a-bcca-73f0ac7bf7cd\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73pfsr7" Jul 05 13:25:26.065205 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:25:26.065203 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-598dc\" (UniqueName: \"kubernetes.io/projected/617eb3c5-e598-495a-bcca-73f0ac7bf7cd-kube-api-access-598dc\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73pfsr7\" (UID: \"617eb3c5-e598-495a-bcca-73f0ac7bf7cd\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73pfsr7" Jul 05 13:25:26.065450 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:25:26.065240 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/617eb3c5-e598-495a-bcca-73f0ac7bf7cd-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73pfsr7\" (UID: \"617eb3c5-e598-495a-bcca-73f0ac7bf7cd\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73pfsr7" Jul 05 13:25:26.065623 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:25:26.065564 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/617eb3c5-e598-495a-bcca-73f0ac7bf7cd-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73pfsr7\" (UID: \"617eb3c5-e598-495a-bcca-73f0ac7bf7cd\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73pfsr7" Jul 05 13:25:26.065700 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:25:26.065632 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/617eb3c5-e598-495a-bcca-73f0ac7bf7cd-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73pfsr7\" (UID: \"617eb3c5-e598-495a-bcca-73f0ac7bf7cd\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73pfsr7" Jul 05 13:25:26.077688 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:25:26.077667 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-598dc\" (UniqueName: \"kubernetes.io/projected/617eb3c5-e598-495a-bcca-73f0ac7bf7cd-kube-api-access-598dc\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73pfsr7\" (UID: \"617eb3c5-e598-495a-bcca-73f0ac7bf7cd\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73pfsr7" Jul 05 13:25:26.171509 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:25:26.171428 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73pfsr7" Jul 05 13:25:26.317150 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:25:26.317112 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qqr9g"] Jul 05 13:25:26.321689 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:25:26.321662 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qqr9g" Jul 05 13:25:26.342526 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:25:26.342488 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qqr9g"] Jul 05 13:25:26.345144 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:25:26.345093 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73pfsr7"] Jul 05 13:25:26.347136 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:25:26.347108 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod617eb3c5_e598_495a_bcca_73f0ac7bf7cd.slice/crio-290c6fe4753a92d113ebc98093e8f402821bcd9f66c790190ba149e79b863387 WatchSource:0}: Error finding container 290c6fe4753a92d113ebc98093e8f402821bcd9f66c790190ba149e79b863387: Status 404 returned error can't find the container with id 290c6fe4753a92d113ebc98093e8f402821bcd9f66c790190ba149e79b863387 Jul 05 13:25:26.368983 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:25:26.368959 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/44b648ae-87f1-479b-baa3-4124452bf1ab-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qqr9g\" (UID: \"44b648ae-87f1-479b-baa3-4124452bf1ab\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qqr9g" Jul 05 13:25:26.369091 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:25:26.368992 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c7cgs\" (UniqueName: \"kubernetes.io/projected/44b648ae-87f1-479b-baa3-4124452bf1ab-kube-api-access-c7cgs\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qqr9g\" (UID: \"44b648ae-87f1-479b-baa3-4124452bf1ab\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qqr9g" Jul 05 13:25:26.369152 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:25:26.369121 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/44b648ae-87f1-479b-baa3-4124452bf1ab-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qqr9g\" (UID: \"44b648ae-87f1-479b-baa3-4124452bf1ab\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qqr9g" Jul 05 13:25:26.470407 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:25:26.470375 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/44b648ae-87f1-479b-baa3-4124452bf1ab-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qqr9g\" (UID: \"44b648ae-87f1-479b-baa3-4124452bf1ab\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qqr9g" Jul 05 13:25:26.470545 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:25:26.470413 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/44b648ae-87f1-479b-baa3-4124452bf1ab-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qqr9g\" (UID: \"44b648ae-87f1-479b-baa3-4124452bf1ab\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qqr9g" Jul 05 13:25:26.470545 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:25:26.470433 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c7cgs\" (UniqueName: \"kubernetes.io/projected/44b648ae-87f1-479b-baa3-4124452bf1ab-kube-api-access-c7cgs\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qqr9g\" (UID: \"44b648ae-87f1-479b-baa3-4124452bf1ab\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qqr9g" Jul 05 13:25:26.470782 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:25:26.470764 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/44b648ae-87f1-479b-baa3-4124452bf1ab-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qqr9g\" (UID: \"44b648ae-87f1-479b-baa3-4124452bf1ab\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qqr9g" Jul 05 13:25:26.470821 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:25:26.470794 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/44b648ae-87f1-479b-baa3-4124452bf1ab-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qqr9g\" (UID: \"44b648ae-87f1-479b-baa3-4124452bf1ab\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qqr9g" Jul 05 13:25:26.487277 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:25:26.487253 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c7cgs\" (UniqueName: \"kubernetes.io/projected/44b648ae-87f1-479b-baa3-4124452bf1ab-kube-api-access-c7cgs\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qqr9g\" (UID: \"44b648ae-87f1-479b-baa3-4124452bf1ab\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qqr9g" Jul 05 13:25:26.496929 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:25:26.496904 2578 generic.go:356] "Generic (PLEG): container finished" podID="617eb3c5-e598-495a-bcca-73f0ac7bf7cd" containerID="b5cc6a00f16085d0a5d957e88bec48f8bbed82d3507efced1fc2f57700eac806" exitCode=0 Jul 05 13:25:26.497166 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:25:26.496943 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73pfsr7" event={"ID":"617eb3c5-e598-495a-bcca-73f0ac7bf7cd","Type":"ContainerDied","Data":"b5cc6a00f16085d0a5d957e88bec48f8bbed82d3507efced1fc2f57700eac806"} Jul 05 13:25:26.497166 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:25:26.496981 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73pfsr7" event={"ID":"617eb3c5-e598-495a-bcca-73f0ac7bf7cd","Type":"ContainerStarted","Data":"290c6fe4753a92d113ebc98093e8f402821bcd9f66c790190ba149e79b863387"} Jul 05 13:25:26.497422 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:25:26.497407 2578 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 05 13:25:26.631072 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:25:26.631033 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qqr9g" Jul 05 13:25:26.802144 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:25:26.802117 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qqr9g"] Jul 05 13:25:26.803512 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:25:26.803486 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod44b648ae_87f1_479b_baa3_4124452bf1ab.slice/crio-174e36160bcd11881039a7c11c7e673c4bc9315a246382f089687f2760e12c9e WatchSource:0}: Error finding container 174e36160bcd11881039a7c11c7e673c4bc9315a246382f089687f2760e12c9e: Status 404 returned error can't find the container with id 174e36160bcd11881039a7c11c7e673c4bc9315a246382f089687f2760e12c9e Jul 05 13:25:26.907102 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:25:26.907070 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0nrr9v"] Jul 05 13:25:26.910368 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:25:26.910349 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0nrr9v" Jul 05 13:25:26.972012 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:25:26.971947 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0nrr9v"] Jul 05 13:25:26.974277 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:25:26.974250 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hw68c\" (UniqueName: \"kubernetes.io/projected/5e2a0d79-fc0b-415e-aea9-c5ff86918630-kube-api-access-hw68c\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0nrr9v\" (UID: \"5e2a0d79-fc0b-415e-aea9-c5ff86918630\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0nrr9v" Jul 05 13:25:26.974405 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:25:26.974296 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5e2a0d79-fc0b-415e-aea9-c5ff86918630-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0nrr9v\" (UID: \"5e2a0d79-fc0b-415e-aea9-c5ff86918630\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0nrr9v" Jul 05 13:25:26.974405 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:25:26.974323 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5e2a0d79-fc0b-415e-aea9-c5ff86918630-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0nrr9v\" (UID: \"5e2a0d79-fc0b-415e-aea9-c5ff86918630\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0nrr9v" Jul 05 13:25:27.075281 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:25:27.075252 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hw68c\" (UniqueName: \"kubernetes.io/projected/5e2a0d79-fc0b-415e-aea9-c5ff86918630-kube-api-access-hw68c\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0nrr9v\" (UID: \"5e2a0d79-fc0b-415e-aea9-c5ff86918630\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0nrr9v" Jul 05 13:25:27.075413 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:25:27.075299 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5e2a0d79-fc0b-415e-aea9-c5ff86918630-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0nrr9v\" (UID: \"5e2a0d79-fc0b-415e-aea9-c5ff86918630\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0nrr9v" Jul 05 13:25:27.075413 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:25:27.075326 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5e2a0d79-fc0b-415e-aea9-c5ff86918630-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0nrr9v\" (UID: \"5e2a0d79-fc0b-415e-aea9-c5ff86918630\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0nrr9v" Jul 05 13:25:27.075740 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:25:27.075720 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5e2a0d79-fc0b-415e-aea9-c5ff86918630-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0nrr9v\" (UID: \"5e2a0d79-fc0b-415e-aea9-c5ff86918630\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0nrr9v" Jul 05 13:25:27.075783 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:25:27.075735 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5e2a0d79-fc0b-415e-aea9-c5ff86918630-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0nrr9v\" (UID: \"5e2a0d79-fc0b-415e-aea9-c5ff86918630\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0nrr9v" Jul 05 13:25:27.092945 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:25:27.092918 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hw68c\" (UniqueName: \"kubernetes.io/projected/5e2a0d79-fc0b-415e-aea9-c5ff86918630-kube-api-access-hw68c\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0nrr9v\" (UID: \"5e2a0d79-fc0b-415e-aea9-c5ff86918630\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0nrr9v" Jul 05 13:25:27.258208 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:25:27.258176 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0nrr9v" Jul 05 13:25:27.398988 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:25:27.398962 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0nrr9v"] Jul 05 13:25:27.400973 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:25:27.400948 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5e2a0d79_fc0b_415e_aea9_c5ff86918630.slice/crio-c4d3113ccf11655a5c8c8d7b713542b8e2e38c7db738b46c07bd4ad13989ddfc WatchSource:0}: Error finding container c4d3113ccf11655a5c8c8d7b713542b8e2e38c7db738b46c07bd4ad13989ddfc: Status 404 returned error can't find the container with id c4d3113ccf11655a5c8c8d7b713542b8e2e38c7db738b46c07bd4ad13989ddfc Jul 05 13:25:27.454050 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:25:27.454025 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759k96jz"] Jul 05 13:25:27.457426 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:25:27.457406 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759k96jz" Jul 05 13:25:27.472534 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:25:27.472503 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759k96jz"] Jul 05 13:25:27.504948 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:25:27.504917 2578 generic.go:356] "Generic (PLEG): container finished" podID="617eb3c5-e598-495a-bcca-73f0ac7bf7cd" containerID="d4d759f1548544285c4427f790e8022ff5d2a7f23140b8a05ccc7362c70121c5" exitCode=0 Jul 05 13:25:27.505178 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:25:27.504996 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73pfsr7" event={"ID":"617eb3c5-e598-495a-bcca-73f0ac7bf7cd","Type":"ContainerDied","Data":"d4d759f1548544285c4427f790e8022ff5d2a7f23140b8a05ccc7362c70121c5"} Jul 05 13:25:27.506949 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:25:27.506924 2578 generic.go:356] "Generic (PLEG): container finished" podID="44b648ae-87f1-479b-baa3-4124452bf1ab" containerID="dc8f25ad51185ae9866c0a8651caa3e6626f7af6680c2769409e3882d74eafc2" exitCode=0 Jul 05 13:25:27.507133 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:25:27.506960 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qqr9g" event={"ID":"44b648ae-87f1-479b-baa3-4124452bf1ab","Type":"ContainerDied","Data":"dc8f25ad51185ae9866c0a8651caa3e6626f7af6680c2769409e3882d74eafc2"} Jul 05 13:25:27.507133 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:25:27.507030 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qqr9g" event={"ID":"44b648ae-87f1-479b-baa3-4124452bf1ab","Type":"ContainerStarted","Data":"174e36160bcd11881039a7c11c7e673c4bc9315a246382f089687f2760e12c9e"} Jul 05 13:25:27.509550 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:25:27.509492 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0nrr9v" event={"ID":"5e2a0d79-fc0b-415e-aea9-c5ff86918630","Type":"ContainerStarted","Data":"4c9b37829ff637612f56443da4af7980ca5952d13de6e270e24f6ff80e236c50"} Jul 05 13:25:27.509550 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:25:27.509541 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0nrr9v" event={"ID":"5e2a0d79-fc0b-415e-aea9-c5ff86918630","Type":"ContainerStarted","Data":"c4d3113ccf11655a5c8c8d7b713542b8e2e38c7db738b46c07bd4ad13989ddfc"} Jul 05 13:25:27.578212 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:25:27.578185 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n99j4\" (UniqueName: \"kubernetes.io/projected/bc8169a7-6503-4d40-b6fa-4b225badcda0-kube-api-access-n99j4\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759k96jz\" (UID: \"bc8169a7-6503-4d40-b6fa-4b225badcda0\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759k96jz" Jul 05 13:25:27.578380 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:25:27.578222 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/bc8169a7-6503-4d40-b6fa-4b225badcda0-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759k96jz\" (UID: \"bc8169a7-6503-4d40-b6fa-4b225badcda0\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759k96jz" Jul 05 13:25:27.578380 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:25:27.578304 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/bc8169a7-6503-4d40-b6fa-4b225badcda0-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759k96jz\" (UID: \"bc8169a7-6503-4d40-b6fa-4b225badcda0\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759k96jz" Jul 05 13:25:27.679747 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:25:27.679705 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/bc8169a7-6503-4d40-b6fa-4b225badcda0-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759k96jz\" (UID: \"bc8169a7-6503-4d40-b6fa-4b225badcda0\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759k96jz" Jul 05 13:25:27.679909 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:25:27.679803 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n99j4\" (UniqueName: \"kubernetes.io/projected/bc8169a7-6503-4d40-b6fa-4b225badcda0-kube-api-access-n99j4\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759k96jz\" (UID: \"bc8169a7-6503-4d40-b6fa-4b225badcda0\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759k96jz" Jul 05 13:25:27.679909 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:25:27.679827 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/bc8169a7-6503-4d40-b6fa-4b225badcda0-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759k96jz\" (UID: \"bc8169a7-6503-4d40-b6fa-4b225badcda0\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759k96jz" Jul 05 13:25:27.680090 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:25:27.680070 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/bc8169a7-6503-4d40-b6fa-4b225badcda0-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759k96jz\" (UID: \"bc8169a7-6503-4d40-b6fa-4b225badcda0\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759k96jz" Jul 05 13:25:27.680186 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:25:27.680167 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/bc8169a7-6503-4d40-b6fa-4b225badcda0-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759k96jz\" (UID: \"bc8169a7-6503-4d40-b6fa-4b225badcda0\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759k96jz" Jul 05 13:25:27.698742 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:25:27.698716 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n99j4\" (UniqueName: \"kubernetes.io/projected/bc8169a7-6503-4d40-b6fa-4b225badcda0-kube-api-access-n99j4\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759k96jz\" (UID: \"bc8169a7-6503-4d40-b6fa-4b225badcda0\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759k96jz" Jul 05 13:25:27.767016 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:25:27.766939 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759k96jz" Jul 05 13:25:27.918468 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:25:27.918437 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759k96jz"] Jul 05 13:25:27.921997 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:25:27.921956 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbc8169a7_6503_4d40_b6fa_4b225badcda0.slice/crio-54e23daafce52a9e6ccd2e3eedff1a2745003314238774de5e0752dc24a6fee7 WatchSource:0}: Error finding container 54e23daafce52a9e6ccd2e3eedff1a2745003314238774de5e0752dc24a6fee7: Status 404 returned error can't find the container with id 54e23daafce52a9e6ccd2e3eedff1a2745003314238774de5e0752dc24a6fee7 Jul 05 13:25:28.517730 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:25:28.517699 2578 generic.go:356] "Generic (PLEG): container finished" podID="44b648ae-87f1-479b-baa3-4124452bf1ab" containerID="ac98b32837ed3e7719d669697c08a56f0b2cfb8413f08a4edede293122436881" exitCode=0 Jul 05 13:25:28.517933 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:25:28.517792 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qqr9g" event={"ID":"44b648ae-87f1-479b-baa3-4124452bf1ab","Type":"ContainerDied","Data":"ac98b32837ed3e7719d669697c08a56f0b2cfb8413f08a4edede293122436881"} Jul 05 13:25:28.519476 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:25:28.519458 2578 generic.go:356] "Generic (PLEG): container finished" podID="5e2a0d79-fc0b-415e-aea9-c5ff86918630" containerID="4c9b37829ff637612f56443da4af7980ca5952d13de6e270e24f6ff80e236c50" exitCode=0 Jul 05 13:25:28.519550 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:25:28.519521 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0nrr9v" event={"ID":"5e2a0d79-fc0b-415e-aea9-c5ff86918630","Type":"ContainerDied","Data":"4c9b37829ff637612f56443da4af7980ca5952d13de6e270e24f6ff80e236c50"} Jul 05 13:25:28.522151 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:25:28.522127 2578 generic.go:356] "Generic (PLEG): container finished" podID="617eb3c5-e598-495a-bcca-73f0ac7bf7cd" containerID="ed712f221a5ba0fe03e032869fe3057be8b07883cad90a1962aa2ae01d507349" exitCode=0 Jul 05 13:25:28.522239 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:25:28.522213 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73pfsr7" event={"ID":"617eb3c5-e598-495a-bcca-73f0ac7bf7cd","Type":"ContainerDied","Data":"ed712f221a5ba0fe03e032869fe3057be8b07883cad90a1962aa2ae01d507349"} Jul 05 13:25:28.523874 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:25:28.523846 2578 generic.go:356] "Generic (PLEG): container finished" podID="bc8169a7-6503-4d40-b6fa-4b225badcda0" containerID="079c83404c4abf06329b0290c20bbc7789c3578430813517688b3c75e6a83cfb" exitCode=0 Jul 05 13:25:28.523972 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:25:28.523869 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759k96jz" event={"ID":"bc8169a7-6503-4d40-b6fa-4b225badcda0","Type":"ContainerDied","Data":"079c83404c4abf06329b0290c20bbc7789c3578430813517688b3c75e6a83cfb"} Jul 05 13:25:28.523972 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:25:28.523942 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759k96jz" event={"ID":"bc8169a7-6503-4d40-b6fa-4b225badcda0","Type":"ContainerStarted","Data":"54e23daafce52a9e6ccd2e3eedff1a2745003314238774de5e0752dc24a6fee7"} Jul 05 13:25:29.532409 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:25:29.532376 2578 generic.go:356] "Generic (PLEG): container finished" podID="44b648ae-87f1-479b-baa3-4124452bf1ab" containerID="daa70a97fcce4adc75bfab3115f0a60d6a61bd04fae3c0617d9bbf6988d6f9b0" exitCode=0 Jul 05 13:25:29.532876 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:25:29.532453 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qqr9g" event={"ID":"44b648ae-87f1-479b-baa3-4124452bf1ab","Type":"ContainerDied","Data":"daa70a97fcce4adc75bfab3115f0a60d6a61bd04fae3c0617d9bbf6988d6f9b0"} Jul 05 13:25:29.534249 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:25:29.534229 2578 generic.go:356] "Generic (PLEG): container finished" podID="5e2a0d79-fc0b-415e-aea9-c5ff86918630" containerID="1f5c59416711ee6f3ad2547e261a5c5546ea1b50e165702ff2cd77d5f5492323" exitCode=0 Jul 05 13:25:29.534370 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:25:29.534304 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0nrr9v" event={"ID":"5e2a0d79-fc0b-415e-aea9-c5ff86918630","Type":"ContainerDied","Data":"1f5c59416711ee6f3ad2547e261a5c5546ea1b50e165702ff2cd77d5f5492323"} Jul 05 13:25:29.536229 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:25:29.536204 2578 generic.go:356] "Generic (PLEG): container finished" podID="bc8169a7-6503-4d40-b6fa-4b225badcda0" containerID="797e8967098c67e187613c82d7fe88ecb38946d8c7e16f0d20bc117dbd36942f" exitCode=0 Jul 05 13:25:29.536306 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:25:29.536268 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759k96jz" event={"ID":"bc8169a7-6503-4d40-b6fa-4b225badcda0","Type":"ContainerDied","Data":"797e8967098c67e187613c82d7fe88ecb38946d8c7e16f0d20bc117dbd36942f"} Jul 05 13:25:29.675852 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:25:29.675777 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73pfsr7" Jul 05 13:25:29.797643 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:25:29.797532 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/617eb3c5-e598-495a-bcca-73f0ac7bf7cd-bundle\") pod \"617eb3c5-e598-495a-bcca-73f0ac7bf7cd\" (UID: \"617eb3c5-e598-495a-bcca-73f0ac7bf7cd\") " Jul 05 13:25:29.797643 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:25:29.797623 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/617eb3c5-e598-495a-bcca-73f0ac7bf7cd-util\") pod \"617eb3c5-e598-495a-bcca-73f0ac7bf7cd\" (UID: \"617eb3c5-e598-495a-bcca-73f0ac7bf7cd\") " Jul 05 13:25:29.797643 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:25:29.797644 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-598dc\" (UniqueName: \"kubernetes.io/projected/617eb3c5-e598-495a-bcca-73f0ac7bf7cd-kube-api-access-598dc\") pod \"617eb3c5-e598-495a-bcca-73f0ac7bf7cd\" (UID: \"617eb3c5-e598-495a-bcca-73f0ac7bf7cd\") " Jul 05 13:25:29.798152 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:25:29.798125 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/617eb3c5-e598-495a-bcca-73f0ac7bf7cd-bundle" (OuterVolumeSpecName: "bundle") pod "617eb3c5-e598-495a-bcca-73f0ac7bf7cd" (UID: "617eb3c5-e598-495a-bcca-73f0ac7bf7cd"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 05 13:25:29.800002 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:25:29.799976 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/617eb3c5-e598-495a-bcca-73f0ac7bf7cd-kube-api-access-598dc" (OuterVolumeSpecName: "kube-api-access-598dc") pod "617eb3c5-e598-495a-bcca-73f0ac7bf7cd" (UID: "617eb3c5-e598-495a-bcca-73f0ac7bf7cd"). InnerVolumeSpecName "kube-api-access-598dc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 05 13:25:29.803257 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:25:29.803223 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/617eb3c5-e598-495a-bcca-73f0ac7bf7cd-util" (OuterVolumeSpecName: "util") pod "617eb3c5-e598-495a-bcca-73f0ac7bf7cd" (UID: "617eb3c5-e598-495a-bcca-73f0ac7bf7cd"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 05 13:25:29.899245 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:25:29.899204 2578 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/617eb3c5-e598-495a-bcca-73f0ac7bf7cd-bundle\") on node \"ip-10-0-130-191.ec2.internal\" DevicePath \"\"" Jul 05 13:25:29.899245 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:25:29.899236 2578 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/617eb3c5-e598-495a-bcca-73f0ac7bf7cd-util\") on node \"ip-10-0-130-191.ec2.internal\" DevicePath \"\"" Jul 05 13:25:29.899245 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:25:29.899246 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-598dc\" (UniqueName: \"kubernetes.io/projected/617eb3c5-e598-495a-bcca-73f0ac7bf7cd-kube-api-access-598dc\") on node \"ip-10-0-130-191.ec2.internal\" DevicePath \"\"" Jul 05 13:25:30.543147 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:25:30.543122 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73pfsr7" Jul 05 13:25:30.543573 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:25:30.543123 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73pfsr7" event={"ID":"617eb3c5-e598-495a-bcca-73f0ac7bf7cd","Type":"ContainerDied","Data":"290c6fe4753a92d113ebc98093e8f402821bcd9f66c790190ba149e79b863387"} Jul 05 13:25:30.543573 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:25:30.543250 2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="290c6fe4753a92d113ebc98093e8f402821bcd9f66c790190ba149e79b863387" Jul 05 13:25:30.545646 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:25:30.545623 2578 generic.go:356] "Generic (PLEG): container finished" podID="bc8169a7-6503-4d40-b6fa-4b225badcda0" containerID="f78c7368460fdf36a8fb71c3ba0ee706060cef634276a1a35ab3d7412fa62ef7" exitCode=0 Jul 05 13:25:30.545777 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:25:30.545742 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759k96jz" event={"ID":"bc8169a7-6503-4d40-b6fa-4b225badcda0","Type":"ContainerDied","Data":"f78c7368460fdf36a8fb71c3ba0ee706060cef634276a1a35ab3d7412fa62ef7"} Jul 05 13:25:30.547767 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:25:30.547745 2578 generic.go:356] "Generic (PLEG): container finished" podID="5e2a0d79-fc0b-415e-aea9-c5ff86918630" containerID="ea1f1f7c7a14fdf49ce418e95bfaff14f9b54768d05257a5afd9fc3e34e09ad9" exitCode=0 Jul 05 13:25:30.547875 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:25:30.547821 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0nrr9v" event={"ID":"5e2a0d79-fc0b-415e-aea9-c5ff86918630","Type":"ContainerDied","Data":"ea1f1f7c7a14fdf49ce418e95bfaff14f9b54768d05257a5afd9fc3e34e09ad9"} Jul 05 13:25:30.683403 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:25:30.683374 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qqr9g" Jul 05 13:25:30.806871 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:25:30.806788 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-c7cgs\" (UniqueName: \"kubernetes.io/projected/44b648ae-87f1-479b-baa3-4124452bf1ab-kube-api-access-c7cgs\") pod \"44b648ae-87f1-479b-baa3-4124452bf1ab\" (UID: \"44b648ae-87f1-479b-baa3-4124452bf1ab\") " Jul 05 13:25:30.807020 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:25:30.806894 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/44b648ae-87f1-479b-baa3-4124452bf1ab-util\") pod \"44b648ae-87f1-479b-baa3-4124452bf1ab\" (UID: \"44b648ae-87f1-479b-baa3-4124452bf1ab\") " Jul 05 13:25:30.807020 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:25:30.806923 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/44b648ae-87f1-479b-baa3-4124452bf1ab-bundle\") pod \"44b648ae-87f1-479b-baa3-4124452bf1ab\" (UID: \"44b648ae-87f1-479b-baa3-4124452bf1ab\") " Jul 05 13:25:30.807457 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:25:30.807436 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/44b648ae-87f1-479b-baa3-4124452bf1ab-bundle" (OuterVolumeSpecName: "bundle") pod "44b648ae-87f1-479b-baa3-4124452bf1ab" (UID: "44b648ae-87f1-479b-baa3-4124452bf1ab"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 05 13:25:30.809130 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:25:30.809109 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/44b648ae-87f1-479b-baa3-4124452bf1ab-kube-api-access-c7cgs" (OuterVolumeSpecName: "kube-api-access-c7cgs") pod "44b648ae-87f1-479b-baa3-4124452bf1ab" (UID: "44b648ae-87f1-479b-baa3-4124452bf1ab"). InnerVolumeSpecName "kube-api-access-c7cgs". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 05 13:25:30.812069 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:25:30.812032 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/44b648ae-87f1-479b-baa3-4124452bf1ab-util" (OuterVolumeSpecName: "util") pod "44b648ae-87f1-479b-baa3-4124452bf1ab" (UID: "44b648ae-87f1-479b-baa3-4124452bf1ab"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 05 13:25:30.908066 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:25:30.908017 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c7cgs\" (UniqueName: \"kubernetes.io/projected/44b648ae-87f1-479b-baa3-4124452bf1ab-kube-api-access-c7cgs\") on node \"ip-10-0-130-191.ec2.internal\" DevicePath \"\"" Jul 05 13:25:30.908066 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:25:30.908058 2578 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/44b648ae-87f1-479b-baa3-4124452bf1ab-util\") on node \"ip-10-0-130-191.ec2.internal\" DevicePath \"\"" Jul 05 13:25:30.908066 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:25:30.908067 2578 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/44b648ae-87f1-479b-baa3-4124452bf1ab-bundle\") on node \"ip-10-0-130-191.ec2.internal\" DevicePath \"\"" Jul 05 13:25:31.556283 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:25:31.556247 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qqr9g" Jul 05 13:25:31.556283 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:25:31.556258 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qqr9g" event={"ID":"44b648ae-87f1-479b-baa3-4124452bf1ab","Type":"ContainerDied","Data":"174e36160bcd11881039a7c11c7e673c4bc9315a246382f089687f2760e12c9e"} Jul 05 13:25:31.556701 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:25:31.556304 2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="174e36160bcd11881039a7c11c7e673c4bc9315a246382f089687f2760e12c9e" Jul 05 13:25:31.710438 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:25:31.710419 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759k96jz" Jul 05 13:25:31.713252 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:25:31.713235 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0nrr9v" Jul 05 13:25:31.814288 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:25:31.814206 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5e2a0d79-fc0b-415e-aea9-c5ff86918630-bundle\") pod \"5e2a0d79-fc0b-415e-aea9-c5ff86918630\" (UID: \"5e2a0d79-fc0b-415e-aea9-c5ff86918630\") " Jul 05 13:25:31.814288 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:25:31.814266 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hw68c\" (UniqueName: \"kubernetes.io/projected/5e2a0d79-fc0b-415e-aea9-c5ff86918630-kube-api-access-hw68c\") pod \"5e2a0d79-fc0b-415e-aea9-c5ff86918630\" (UID: \"5e2a0d79-fc0b-415e-aea9-c5ff86918630\") " Jul 05 13:25:31.814515 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:25:31.814306 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/bc8169a7-6503-4d40-b6fa-4b225badcda0-util\") pod \"bc8169a7-6503-4d40-b6fa-4b225badcda0\" (UID: \"bc8169a7-6503-4d40-b6fa-4b225badcda0\") " Jul 05 13:25:31.814515 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:25:31.814374 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n99j4\" (UniqueName: \"kubernetes.io/projected/bc8169a7-6503-4d40-b6fa-4b225badcda0-kube-api-access-n99j4\") pod \"bc8169a7-6503-4d40-b6fa-4b225badcda0\" (UID: \"bc8169a7-6503-4d40-b6fa-4b225badcda0\") " Jul 05 13:25:31.814515 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:25:31.814396 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/bc8169a7-6503-4d40-b6fa-4b225badcda0-bundle\") pod \"bc8169a7-6503-4d40-b6fa-4b225badcda0\" (UID: \"bc8169a7-6503-4d40-b6fa-4b225badcda0\") " Jul 05 13:25:31.814515 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:25:31.814425 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5e2a0d79-fc0b-415e-aea9-c5ff86918630-util\") pod \"5e2a0d79-fc0b-415e-aea9-c5ff86918630\" (UID: \"5e2a0d79-fc0b-415e-aea9-c5ff86918630\") " Jul 05 13:25:31.815189 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:25:31.814926 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bc8169a7-6503-4d40-b6fa-4b225badcda0-bundle" (OuterVolumeSpecName: "bundle") pod "bc8169a7-6503-4d40-b6fa-4b225badcda0" (UID: "bc8169a7-6503-4d40-b6fa-4b225badcda0"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 05 13:25:31.815189 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:25:31.814991 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5e2a0d79-fc0b-415e-aea9-c5ff86918630-bundle" (OuterVolumeSpecName: "bundle") pod "5e2a0d79-fc0b-415e-aea9-c5ff86918630" (UID: "5e2a0d79-fc0b-415e-aea9-c5ff86918630"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 05 13:25:31.816817 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:25:31.816792 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bc8169a7-6503-4d40-b6fa-4b225badcda0-kube-api-access-n99j4" (OuterVolumeSpecName: "kube-api-access-n99j4") pod "bc8169a7-6503-4d40-b6fa-4b225badcda0" (UID: "bc8169a7-6503-4d40-b6fa-4b225badcda0"). InnerVolumeSpecName "kube-api-access-n99j4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 05 13:25:31.816817 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:25:31.816798 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5e2a0d79-fc0b-415e-aea9-c5ff86918630-kube-api-access-hw68c" (OuterVolumeSpecName: "kube-api-access-hw68c") pod "5e2a0d79-fc0b-415e-aea9-c5ff86918630" (UID: "5e2a0d79-fc0b-415e-aea9-c5ff86918630"). InnerVolumeSpecName "kube-api-access-hw68c". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 05 13:25:31.823383 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:25:31.823356 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bc8169a7-6503-4d40-b6fa-4b225badcda0-util" (OuterVolumeSpecName: "util") pod "bc8169a7-6503-4d40-b6fa-4b225badcda0" (UID: "bc8169a7-6503-4d40-b6fa-4b225badcda0"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 05 13:25:31.823546 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:25:31.823526 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5e2a0d79-fc0b-415e-aea9-c5ff86918630-util" (OuterVolumeSpecName: "util") pod "5e2a0d79-fc0b-415e-aea9-c5ff86918630" (UID: "5e2a0d79-fc0b-415e-aea9-c5ff86918630"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 05 13:25:31.915692 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:25:31.915645 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hw68c\" (UniqueName: \"kubernetes.io/projected/5e2a0d79-fc0b-415e-aea9-c5ff86918630-kube-api-access-hw68c\") on node \"ip-10-0-130-191.ec2.internal\" DevicePath \"\"" Jul 05 13:25:31.915692 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:25:31.915687 2578 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/bc8169a7-6503-4d40-b6fa-4b225badcda0-util\") on node \"ip-10-0-130-191.ec2.internal\" DevicePath \"\"" Jul 05 13:25:31.915692 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:25:31.915701 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n99j4\" (UniqueName: \"kubernetes.io/projected/bc8169a7-6503-4d40-b6fa-4b225badcda0-kube-api-access-n99j4\") on node \"ip-10-0-130-191.ec2.internal\" DevicePath \"\"" Jul 05 13:25:31.916051 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:25:31.915714 2578 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/bc8169a7-6503-4d40-b6fa-4b225badcda0-bundle\") on node \"ip-10-0-130-191.ec2.internal\" DevicePath \"\"" Jul 05 13:25:31.916051 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:25:31.915729 2578 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5e2a0d79-fc0b-415e-aea9-c5ff86918630-util\") on node \"ip-10-0-130-191.ec2.internal\" DevicePath \"\"" Jul 05 13:25:31.916051 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:25:31.915737 2578 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5e2a0d79-fc0b-415e-aea9-c5ff86918630-bundle\") on node \"ip-10-0-130-191.ec2.internal\" DevicePath \"\"" Jul 05 13:25:32.562566 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:25:32.562536 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759k96jz" Jul 05 13:25:32.562961 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:25:32.562572 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759k96jz" event={"ID":"bc8169a7-6503-4d40-b6fa-4b225badcda0","Type":"ContainerDied","Data":"54e23daafce52a9e6ccd2e3eedff1a2745003314238774de5e0752dc24a6fee7"} Jul 05 13:25:32.562961 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:25:32.562669 2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="54e23daafce52a9e6ccd2e3eedff1a2745003314238774de5e0752dc24a6fee7" Jul 05 13:25:32.564813 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:25:32.564784 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0nrr9v" event={"ID":"5e2a0d79-fc0b-415e-aea9-c5ff86918630","Type":"ContainerDied","Data":"c4d3113ccf11655a5c8c8d7b713542b8e2e38c7db738b46c07bd4ad13989ddfc"} Jul 05 13:25:32.564813 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:25:32.564809 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0nrr9v" Jul 05 13:25:32.564960 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:25:32.564827 2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c4d3113ccf11655a5c8c8d7b713542b8e2e38c7db738b46c07bd4ad13989ddfc" Jul 05 13:25:33.152757 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:25:33.152720 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-d46699747-vtrh8"] Jul 05 13:25:33.153092 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:25:33.153076 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="44b648ae-87f1-479b-baa3-4124452bf1ab" containerName="pull" Jul 05 13:25:33.153092 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:25:33.153092 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="44b648ae-87f1-479b-baa3-4124452bf1ab" containerName="pull" Jul 05 13:25:33.153243 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:25:33.153105 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="617eb3c5-e598-495a-bcca-73f0ac7bf7cd" containerName="extract" Jul 05 13:25:33.153243 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:25:33.153111 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="617eb3c5-e598-495a-bcca-73f0ac7bf7cd" containerName="extract" Jul 05 13:25:33.153243 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:25:33.153126 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="617eb3c5-e598-495a-bcca-73f0ac7bf7cd" containerName="util" Jul 05 13:25:33.153243 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:25:33.153134 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="617eb3c5-e598-495a-bcca-73f0ac7bf7cd" containerName="util" Jul 05 13:25:33.153243 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:25:33.153140 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5e2a0d79-fc0b-415e-aea9-c5ff86918630" containerName="util" Jul 05 13:25:33.153243 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:25:33.153144 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="5e2a0d79-fc0b-415e-aea9-c5ff86918630" containerName="util" Jul 05 13:25:33.153243 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:25:33.153152 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5e2a0d79-fc0b-415e-aea9-c5ff86918630" containerName="extract" Jul 05 13:25:33.153243 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:25:33.153157 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="5e2a0d79-fc0b-415e-aea9-c5ff86918630" containerName="extract" Jul 05 13:25:33.153243 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:25:33.153165 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="617eb3c5-e598-495a-bcca-73f0ac7bf7cd" containerName="pull" Jul 05 13:25:33.153243 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:25:33.153169 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="617eb3c5-e598-495a-bcca-73f0ac7bf7cd" containerName="pull" Jul 05 13:25:33.153243 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:25:33.153175 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="44b648ae-87f1-479b-baa3-4124452bf1ab" containerName="extract" Jul 05 13:25:33.153243 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:25:33.153179 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="44b648ae-87f1-479b-baa3-4124452bf1ab" containerName="extract" Jul 05 13:25:33.153243 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:25:33.153187 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bc8169a7-6503-4d40-b6fa-4b225badcda0" containerName="pull" Jul 05 13:25:33.153243 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:25:33.153191 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="bc8169a7-6503-4d40-b6fa-4b225badcda0" containerName="pull" Jul 05 13:25:33.153243 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:25:33.153197 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bc8169a7-6503-4d40-b6fa-4b225badcda0" containerName="extract" Jul 05 13:25:33.153243 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:25:33.153205 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="bc8169a7-6503-4d40-b6fa-4b225badcda0" containerName="extract" Jul 05 13:25:33.153243 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:25:33.153213 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bc8169a7-6503-4d40-b6fa-4b225badcda0" containerName="util" Jul 05 13:25:33.153243 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:25:33.153218 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="bc8169a7-6503-4d40-b6fa-4b225badcda0" containerName="util" Jul 05 13:25:33.153243 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:25:33.153225 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5e2a0d79-fc0b-415e-aea9-c5ff86918630" containerName="pull" Jul 05 13:25:33.153243 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:25:33.153231 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="5e2a0d79-fc0b-415e-aea9-c5ff86918630" containerName="pull" Jul 05 13:25:33.153243 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:25:33.153240 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="44b648ae-87f1-479b-baa3-4124452bf1ab" containerName="util" Jul 05 13:25:33.153243 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:25:33.153246 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="44b648ae-87f1-479b-baa3-4124452bf1ab" containerName="util" Jul 05 13:25:33.153869 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:25:33.153305 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="44b648ae-87f1-479b-baa3-4124452bf1ab" containerName="extract" Jul 05 13:25:33.153869 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:25:33.153314 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="5e2a0d79-fc0b-415e-aea9-c5ff86918630" containerName="extract" Jul 05 13:25:33.153869 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:25:33.153319 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="617eb3c5-e598-495a-bcca-73f0ac7bf7cd" containerName="extract" Jul 05 13:25:33.153869 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:25:33.153325 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="bc8169a7-6503-4d40-b6fa-4b225badcda0" containerName="extract" Jul 05 13:25:33.157512 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:25:33.157491 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-d46699747-vtrh8" Jul 05 13:25:33.184494 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:25:33.184467 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-d46699747-vtrh8"] Jul 05 13:25:33.230487 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:25:33.230452 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ce871812-196b-4bc0-ade3-1bbe1e9eafcc-console-serving-cert\") pod \"console-d46699747-vtrh8\" (UID: \"ce871812-196b-4bc0-ade3-1bbe1e9eafcc\") " pod="openshift-console/console-d46699747-vtrh8" Jul 05 13:25:33.230487 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:25:33.230484 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ce871812-196b-4bc0-ade3-1bbe1e9eafcc-service-ca\") pod \"console-d46699747-vtrh8\" (UID: \"ce871812-196b-4bc0-ade3-1bbe1e9eafcc\") " pod="openshift-console/console-d46699747-vtrh8" Jul 05 13:25:33.230703 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:25:33.230505 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4xwmz\" (UniqueName: \"kubernetes.io/projected/ce871812-196b-4bc0-ade3-1bbe1e9eafcc-kube-api-access-4xwmz\") pod \"console-d46699747-vtrh8\" (UID: \"ce871812-196b-4bc0-ade3-1bbe1e9eafcc\") " pod="openshift-console/console-d46699747-vtrh8" Jul 05 13:25:33.230703 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:25:33.230554 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ce871812-196b-4bc0-ade3-1bbe1e9eafcc-oauth-serving-cert\") pod \"console-d46699747-vtrh8\" (UID: \"ce871812-196b-4bc0-ade3-1bbe1e9eafcc\") " pod="openshift-console/console-d46699747-vtrh8" Jul 05 13:25:33.230703 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:25:33.230632 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ce871812-196b-4bc0-ade3-1bbe1e9eafcc-console-oauth-config\") pod \"console-d46699747-vtrh8\" (UID: \"ce871812-196b-4bc0-ade3-1bbe1e9eafcc\") " pod="openshift-console/console-d46699747-vtrh8" Jul 05 13:25:33.230703 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:25:33.230677 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ce871812-196b-4bc0-ade3-1bbe1e9eafcc-console-config\") pod \"console-d46699747-vtrh8\" (UID: \"ce871812-196b-4bc0-ade3-1bbe1e9eafcc\") " pod="openshift-console/console-d46699747-vtrh8" Jul 05 13:25:33.230848 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:25:33.230740 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ce871812-196b-4bc0-ade3-1bbe1e9eafcc-trusted-ca-bundle\") pod \"console-d46699747-vtrh8\" (UID: \"ce871812-196b-4bc0-ade3-1bbe1e9eafcc\") " pod="openshift-console/console-d46699747-vtrh8" Jul 05 13:25:33.331136 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:25:33.331105 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ce871812-196b-4bc0-ade3-1bbe1e9eafcc-console-serving-cert\") pod \"console-d46699747-vtrh8\" (UID: \"ce871812-196b-4bc0-ade3-1bbe1e9eafcc\") " pod="openshift-console/console-d46699747-vtrh8" Jul 05 13:25:33.331136 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:25:33.331140 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ce871812-196b-4bc0-ade3-1bbe1e9eafcc-service-ca\") pod \"console-d46699747-vtrh8\" (UID: \"ce871812-196b-4bc0-ade3-1bbe1e9eafcc\") " pod="openshift-console/console-d46699747-vtrh8" Jul 05 13:25:33.331342 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:25:33.331162 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4xwmz\" (UniqueName: \"kubernetes.io/projected/ce871812-196b-4bc0-ade3-1bbe1e9eafcc-kube-api-access-4xwmz\") pod \"console-d46699747-vtrh8\" (UID: \"ce871812-196b-4bc0-ade3-1bbe1e9eafcc\") " pod="openshift-console/console-d46699747-vtrh8" Jul 05 13:25:33.331342 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:25:33.331183 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ce871812-196b-4bc0-ade3-1bbe1e9eafcc-oauth-serving-cert\") pod \"console-d46699747-vtrh8\" (UID: \"ce871812-196b-4bc0-ade3-1bbe1e9eafcc\") " pod="openshift-console/console-d46699747-vtrh8" Jul 05 13:25:33.331342 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:25:33.331206 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ce871812-196b-4bc0-ade3-1bbe1e9eafcc-console-oauth-config\") pod \"console-d46699747-vtrh8\" (UID: \"ce871812-196b-4bc0-ade3-1bbe1e9eafcc\") " pod="openshift-console/console-d46699747-vtrh8" Jul 05 13:25:33.331342 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:25:33.331247 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ce871812-196b-4bc0-ade3-1bbe1e9eafcc-console-config\") pod \"console-d46699747-vtrh8\" (UID: \"ce871812-196b-4bc0-ade3-1bbe1e9eafcc\") " pod="openshift-console/console-d46699747-vtrh8" Jul 05 13:25:33.331342 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:25:33.331323 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ce871812-196b-4bc0-ade3-1bbe1e9eafcc-trusted-ca-bundle\") pod \"console-d46699747-vtrh8\" (UID: \"ce871812-196b-4bc0-ade3-1bbe1e9eafcc\") " pod="openshift-console/console-d46699747-vtrh8" Jul 05 13:25:33.332058 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:25:33.332031 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ce871812-196b-4bc0-ade3-1bbe1e9eafcc-oauth-serving-cert\") pod \"console-d46699747-vtrh8\" (UID: \"ce871812-196b-4bc0-ade3-1bbe1e9eafcc\") " pod="openshift-console/console-d46699747-vtrh8" Jul 05 13:25:33.332205 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:25:33.332051 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ce871812-196b-4bc0-ade3-1bbe1e9eafcc-service-ca\") pod \"console-d46699747-vtrh8\" (UID: \"ce871812-196b-4bc0-ade3-1bbe1e9eafcc\") " pod="openshift-console/console-d46699747-vtrh8" Jul 05 13:25:33.332338 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:25:33.332218 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ce871812-196b-4bc0-ade3-1bbe1e9eafcc-trusted-ca-bundle\") pod \"console-d46699747-vtrh8\" (UID: \"ce871812-196b-4bc0-ade3-1bbe1e9eafcc\") " pod="openshift-console/console-d46699747-vtrh8" Jul 05 13:25:33.332338 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:25:33.332302 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ce871812-196b-4bc0-ade3-1bbe1e9eafcc-console-config\") pod \"console-d46699747-vtrh8\" (UID: \"ce871812-196b-4bc0-ade3-1bbe1e9eafcc\") " pod="openshift-console/console-d46699747-vtrh8" Jul 05 13:25:33.333810 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:25:33.333784 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ce871812-196b-4bc0-ade3-1bbe1e9eafcc-console-oauth-config\") pod \"console-d46699747-vtrh8\" (UID: \"ce871812-196b-4bc0-ade3-1bbe1e9eafcc\") " pod="openshift-console/console-d46699747-vtrh8" Jul 05 13:25:33.333904 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:25:33.333889 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ce871812-196b-4bc0-ade3-1bbe1e9eafcc-console-serving-cert\") pod \"console-d46699747-vtrh8\" (UID: \"ce871812-196b-4bc0-ade3-1bbe1e9eafcc\") " pod="openshift-console/console-d46699747-vtrh8" Jul 05 13:25:33.346684 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:25:33.346662 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4xwmz\" (UniqueName: \"kubernetes.io/projected/ce871812-196b-4bc0-ade3-1bbe1e9eafcc-kube-api-access-4xwmz\") pod \"console-d46699747-vtrh8\" (UID: \"ce871812-196b-4bc0-ade3-1bbe1e9eafcc\") " pod="openshift-console/console-d46699747-vtrh8" Jul 05 13:25:33.466849 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:25:33.466750 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-d46699747-vtrh8" Jul 05 13:25:33.629492 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:25:33.629468 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-d46699747-vtrh8"] Jul 05 13:25:33.631315 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:25:33.631283 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podce871812_196b_4bc0_ade3_1bbe1e9eafcc.slice/crio-c6a552fe9117583694839d7b401e272ac15ddaf3540e83ea90a7e0b022da685e WatchSource:0}: Error finding container c6a552fe9117583694839d7b401e272ac15ddaf3540e83ea90a7e0b022da685e: Status 404 returned error can't find the container with id c6a552fe9117583694839d7b401e272ac15ddaf3540e83ea90a7e0b022da685e Jul 05 13:25:34.577816 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:25:34.577779 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-d46699747-vtrh8" event={"ID":"ce871812-196b-4bc0-ade3-1bbe1e9eafcc","Type":"ContainerStarted","Data":"1bbe60f9553247b6991742ca9c2041231790bc9e8194d60883ac4b5b4eee6d20"} Jul 05 13:25:34.578017 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:25:34.577833 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-d46699747-vtrh8" event={"ID":"ce871812-196b-4bc0-ade3-1bbe1e9eafcc","Type":"ContainerStarted","Data":"c6a552fe9117583694839d7b401e272ac15ddaf3540e83ea90a7e0b022da685e"} Jul 05 13:25:34.624277 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:25:34.624224 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-d46699747-vtrh8" podStartSLOduration=1.6242088190000001 podStartE2EDuration="1.624208819s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 13:25:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 13:25:34.623607383 +0000 UTC m=+619.082170489" watchObservedRunningTime="2026-07-05 13:25:34.624208819 +0000 UTC m=+619.082771923" Jul 05 13:25:43.467065 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:25:43.466978 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-d46699747-vtrh8" Jul 05 13:25:43.467065 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:25:43.467041 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-d46699747-vtrh8" Jul 05 13:25:43.472963 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:25:43.472938 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-d46699747-vtrh8" Jul 05 13:25:43.627510 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:25:43.627486 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-d46699747-vtrh8" Jul 05 13:25:43.989985 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:25:43.989949 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-75cb8f6bcc-dbrfg"] Jul 05 13:26:00.074710 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:26:00.074677 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-xf4jh"] Jul 05 13:26:00.077727 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:26:00.077709 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-xf4jh" Jul 05 13:26:00.081951 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:26:00.081931 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 05 13:26:00.082156 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:26:00.082138 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\"" Jul 05 13:26:00.083433 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:26:00.083412 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-vkw6q\"" Jul 05 13:26:00.083433 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:26:00.083428 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 05 13:26:00.090706 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:26:00.090688 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\"" Jul 05 13:26:00.098078 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:26:00.098058 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-xf4jh"] Jul 05 13:26:00.169801 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:26:00.169763 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/dc8dab78-d3b4-4e5a-96e7-cf80d35cdc84-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-xf4jh\" (UID: \"dc8dab78-d3b4-4e5a-96e7-cf80d35cdc84\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-xf4jh" Jul 05 13:26:00.170013 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:26:00.169816 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/dc8dab78-d3b4-4e5a-96e7-cf80d35cdc84-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-xf4jh\" (UID: \"dc8dab78-d3b4-4e5a-96e7-cf80d35cdc84\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-xf4jh" Jul 05 13:26:00.170013 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:26:00.169930 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vzl6c\" (UniqueName: \"kubernetes.io/projected/dc8dab78-d3b4-4e5a-96e7-cf80d35cdc84-kube-api-access-vzl6c\") pod \"kuadrant-console-plugin-66b5774985-xf4jh\" (UID: \"dc8dab78-d3b4-4e5a-96e7-cf80d35cdc84\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-xf4jh" Jul 05 13:26:00.270642 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:26:00.270570 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/dc8dab78-d3b4-4e5a-96e7-cf80d35cdc84-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-xf4jh\" (UID: \"dc8dab78-d3b4-4e5a-96e7-cf80d35cdc84\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-xf4jh" Jul 05 13:26:00.270642 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:26:00.270647 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/dc8dab78-d3b4-4e5a-96e7-cf80d35cdc84-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-xf4jh\" (UID: \"dc8dab78-d3b4-4e5a-96e7-cf80d35cdc84\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-xf4jh" Jul 05 13:26:00.270894 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:26:00.270739 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vzl6c\" (UniqueName: \"kubernetes.io/projected/dc8dab78-d3b4-4e5a-96e7-cf80d35cdc84-kube-api-access-vzl6c\") pod \"kuadrant-console-plugin-66b5774985-xf4jh\" (UID: \"dc8dab78-d3b4-4e5a-96e7-cf80d35cdc84\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-xf4jh" Jul 05 13:26:00.271319 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:26:00.271253 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/dc8dab78-d3b4-4e5a-96e7-cf80d35cdc84-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-xf4jh\" (UID: \"dc8dab78-d3b4-4e5a-96e7-cf80d35cdc84\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-xf4jh" Jul 05 13:26:00.273143 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:26:00.273121 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/dc8dab78-d3b4-4e5a-96e7-cf80d35cdc84-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-xf4jh\" (UID: \"dc8dab78-d3b4-4e5a-96e7-cf80d35cdc84\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-xf4jh" Jul 05 13:26:00.283795 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:26:00.283685 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vzl6c\" (UniqueName: \"kubernetes.io/projected/dc8dab78-d3b4-4e5a-96e7-cf80d35cdc84-kube-api-access-vzl6c\") pod \"kuadrant-console-plugin-66b5774985-xf4jh\" (UID: \"dc8dab78-d3b4-4e5a-96e7-cf80d35cdc84\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-xf4jh" Jul 05 13:26:00.388145 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:26:00.388041 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-xf4jh" Jul 05 13:26:00.574783 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:26:00.574761 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-xf4jh"] Jul 05 13:26:00.576924 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:26:00.576896 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poddc8dab78_d3b4_4e5a_96e7_cf80d35cdc84.slice/crio-fb01a0a799b0171dd533887765f11b8d7a77d5aad940b0e49354bca3f6a43eea WatchSource:0}: Error finding container fb01a0a799b0171dd533887765f11b8d7a77d5aad940b0e49354bca3f6a43eea: Status 404 returned error can't find the container with id fb01a0a799b0171dd533887765f11b8d7a77d5aad940b0e49354bca3f6a43eea Jul 05 13:26:00.717766 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:26:00.717683 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-xf4jh" event={"ID":"dc8dab78-d3b4-4e5a-96e7-cf80d35cdc84","Type":"ContainerStarted","Data":"fb01a0a799b0171dd533887765f11b8d7a77d5aad940b0e49354bca3f6a43eea"} Jul 05 13:26:09.012191 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:26:09.012116 2578 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-75cb8f6bcc-dbrfg" podUID="bd354538-c932-466f-ae54-9575b7c11f52" containerName="console" containerID="cri-o://6c867577e39196148bb42c817fc740f50ee9770a18274d0f8f7939c41067a7c9" gracePeriod=15 Jul 05 13:26:10.409515 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:26:10.409477 2578 patch_prober.go:28] interesting pod/console-75cb8f6bcc-dbrfg container/console namespace/openshift-console: Readiness probe status=failure output="Get \"https://10.132.0.25:8443/health\": dial tcp 10.132.0.25:8443: connect: connection refused" start-of-body= Jul 05 13:26:10.409945 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:26:10.409535 2578 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-console/console-75cb8f6bcc-dbrfg" podUID="bd354538-c932-466f-ae54-9575b7c11f52" containerName="console" probeResult="failure" output="Get \"https://10.132.0.25:8443/health\": dial tcp 10.132.0.25:8443: connect: connection refused" Jul 05 13:26:20.409137 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:26:20.409103 2578 patch_prober.go:28] interesting pod/console-75cb8f6bcc-dbrfg container/console namespace/openshift-console: Readiness probe status=failure output="Get \"https://10.132.0.25:8443/health\": dial tcp 10.132.0.25:8443: connect: connection refused" start-of-body= Jul 05 13:26:20.409631 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:26:20.409196 2578 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-console/console-75cb8f6bcc-dbrfg" podUID="bd354538-c932-466f-ae54-9575b7c11f52" containerName="console" probeResult="failure" output="Get \"https://10.132.0.25:8443/health\": dial tcp 10.132.0.25:8443: connect: connection refused" Jul 05 13:26:21.183121 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:26:21.183070 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-75cb8f6bcc-dbrfg" Jul 05 13:26:21.274871 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:26:21.274839 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/bd354538-c932-466f-ae54-9575b7c11f52-console-serving-cert\") pod \"bd354538-c932-466f-ae54-9575b7c11f52\" (UID: \"bd354538-c932-466f-ae54-9575b7c11f52\") " Jul 05 13:26:21.275051 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:26:21.274906 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qkfpl\" (UniqueName: \"kubernetes.io/projected/bd354538-c932-466f-ae54-9575b7c11f52-kube-api-access-qkfpl\") pod \"bd354538-c932-466f-ae54-9575b7c11f52\" (UID: \"bd354538-c932-466f-ae54-9575b7c11f52\") " Jul 05 13:26:21.275051 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:26:21.274934 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/bd354538-c932-466f-ae54-9575b7c11f52-console-config\") pod \"bd354538-c932-466f-ae54-9575b7c11f52\" (UID: \"bd354538-c932-466f-ae54-9575b7c11f52\") " Jul 05 13:26:21.275051 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:26:21.274971 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/bd354538-c932-466f-ae54-9575b7c11f52-service-ca\") pod \"bd354538-c932-466f-ae54-9575b7c11f52\" (UID: \"bd354538-c932-466f-ae54-9575b7c11f52\") " Jul 05 13:26:21.275213 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:26:21.275138 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/bd354538-c932-466f-ae54-9575b7c11f52-oauth-serving-cert\") pod \"bd354538-c932-466f-ae54-9575b7c11f52\" (UID: \"bd354538-c932-466f-ae54-9575b7c11f52\") " Jul 05 13:26:21.275213 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:26:21.275205 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/bd354538-c932-466f-ae54-9575b7c11f52-console-oauth-config\") pod \"bd354538-c932-466f-ae54-9575b7c11f52\" (UID: \"bd354538-c932-466f-ae54-9575b7c11f52\") " Jul 05 13:26:21.275322 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:26:21.275237 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bd354538-c932-466f-ae54-9575b7c11f52-trusted-ca-bundle\") pod \"bd354538-c932-466f-ae54-9575b7c11f52\" (UID: \"bd354538-c932-466f-ae54-9575b7c11f52\") " Jul 05 13:26:21.275428 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:26:21.275406 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bd354538-c932-466f-ae54-9575b7c11f52-service-ca" (OuterVolumeSpecName: "service-ca") pod "bd354538-c932-466f-ae54-9575b7c11f52" (UID: "bd354538-c932-466f-ae54-9575b7c11f52"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 05 13:26:21.275495 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:26:21.275435 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bd354538-c932-466f-ae54-9575b7c11f52-console-config" (OuterVolumeSpecName: "console-config") pod "bd354538-c932-466f-ae54-9575b7c11f52" (UID: "bd354538-c932-466f-ae54-9575b7c11f52"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 05 13:26:21.275495 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:26:21.275441 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bd354538-c932-466f-ae54-9575b7c11f52-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "bd354538-c932-466f-ae54-9575b7c11f52" (UID: "bd354538-c932-466f-ae54-9575b7c11f52"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 05 13:26:21.275752 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:26:21.275731 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bd354538-c932-466f-ae54-9575b7c11f52-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "bd354538-c932-466f-ae54-9575b7c11f52" (UID: "bd354538-c932-466f-ae54-9575b7c11f52"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 05 13:26:21.276007 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:26:21.275951 2578 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bd354538-c932-466f-ae54-9575b7c11f52-trusted-ca-bundle\") on node \"ip-10-0-130-191.ec2.internal\" DevicePath \"\"" Jul 05 13:26:21.276007 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:26:21.275981 2578 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/bd354538-c932-466f-ae54-9575b7c11f52-console-config\") on node \"ip-10-0-130-191.ec2.internal\" DevicePath \"\"" Jul 05 13:26:21.276007 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:26:21.275998 2578 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/bd354538-c932-466f-ae54-9575b7c11f52-service-ca\") on node \"ip-10-0-130-191.ec2.internal\" DevicePath \"\"" Jul 05 13:26:21.276007 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:26:21.276012 2578 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/bd354538-c932-466f-ae54-9575b7c11f52-oauth-serving-cert\") on node \"ip-10-0-130-191.ec2.internal\" DevicePath \"\"" Jul 05 13:26:21.277111 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:26:21.277087 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bd354538-c932-466f-ae54-9575b7c11f52-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "bd354538-c932-466f-ae54-9575b7c11f52" (UID: "bd354538-c932-466f-ae54-9575b7c11f52"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 05 13:26:21.277232 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:26:21.277210 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bd354538-c932-466f-ae54-9575b7c11f52-kube-api-access-qkfpl" (OuterVolumeSpecName: "kube-api-access-qkfpl") pod "bd354538-c932-466f-ae54-9575b7c11f52" (UID: "bd354538-c932-466f-ae54-9575b7c11f52"). InnerVolumeSpecName "kube-api-access-qkfpl". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 05 13:26:21.277456 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:26:21.277434 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bd354538-c932-466f-ae54-9575b7c11f52-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "bd354538-c932-466f-ae54-9575b7c11f52" (UID: "bd354538-c932-466f-ae54-9575b7c11f52"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 05 13:26:21.376902 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:26:21.376818 2578 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/bd354538-c932-466f-ae54-9575b7c11f52-console-oauth-config\") on node \"ip-10-0-130-191.ec2.internal\" DevicePath \"\"" Jul 05 13:26:21.376902 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:26:21.376852 2578 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/bd354538-c932-466f-ae54-9575b7c11f52-console-serving-cert\") on node \"ip-10-0-130-191.ec2.internal\" DevicePath \"\"" Jul 05 13:26:21.376902 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:26:21.376861 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qkfpl\" (UniqueName: \"kubernetes.io/projected/bd354538-c932-466f-ae54-9575b7c11f52-kube-api-access-qkfpl\") on node \"ip-10-0-130-191.ec2.internal\" DevicePath \"\"" Jul 05 13:26:21.848667 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:26:21.848635 2578 generic.go:356] "Generic (PLEG): container finished" podID="bd354538-c932-466f-ae54-9575b7c11f52" containerID="6c867577e39196148bb42c817fc740f50ee9770a18274d0f8f7939c41067a7c9" exitCode=0 Jul 05 13:26:21.849116 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:26:21.848685 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-75cb8f6bcc-dbrfg" Jul 05 13:26:21.849116 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:26:21.848699 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-75cb8f6bcc-dbrfg" event={"ID":"bd354538-c932-466f-ae54-9575b7c11f52","Type":"ContainerDied","Data":"6c867577e39196148bb42c817fc740f50ee9770a18274d0f8f7939c41067a7c9"} Jul 05 13:26:21.849116 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:26:21.848774 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-75cb8f6bcc-dbrfg" event={"ID":"bd354538-c932-466f-ae54-9575b7c11f52","Type":"ContainerDied","Data":"6322269673a65c4a7fd63f427953014d3de334c2078a6191afecdc96596cae82"} Jul 05 13:26:21.849116 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:26:21.848795 2578 scope.go:117] "RemoveContainer" containerID="6c867577e39196148bb42c817fc740f50ee9770a18274d0f8f7939c41067a7c9" Jul 05 13:26:21.850375 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:26:21.850348 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-xf4jh" event={"ID":"dc8dab78-d3b4-4e5a-96e7-cf80d35cdc84","Type":"ContainerStarted","Data":"38b0bd100e98563066b2112ba1d3bb270af26033a08f09dcef36f1297c9a0bac"} Jul 05 13:26:21.858670 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:26:21.858653 2578 scope.go:117] "RemoveContainer" containerID="6c867577e39196148bb42c817fc740f50ee9770a18274d0f8f7939c41067a7c9" Jul 05 13:26:21.858920 ip-10-0-130-191 kubenswrapper[2578]: E0705 13:26:21.858899 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6c867577e39196148bb42c817fc740f50ee9770a18274d0f8f7939c41067a7c9\": container with ID starting with 6c867577e39196148bb42c817fc740f50ee9770a18274d0f8f7939c41067a7c9 not found: ID does not exist" containerID="6c867577e39196148bb42c817fc740f50ee9770a18274d0f8f7939c41067a7c9" Jul 05 13:26:21.858978 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:26:21.858929 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6c867577e39196148bb42c817fc740f50ee9770a18274d0f8f7939c41067a7c9"} err="failed to get container status \"6c867577e39196148bb42c817fc740f50ee9770a18274d0f8f7939c41067a7c9\": rpc error: code = NotFound desc = could not find container \"6c867577e39196148bb42c817fc740f50ee9770a18274d0f8f7939c41067a7c9\": container with ID starting with 6c867577e39196148bb42c817fc740f50ee9770a18274d0f8f7939c41067a7c9 not found: ID does not exist" Jul 05 13:26:21.909268 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:26:21.909218 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-xf4jh" podStartSLOduration=1.397364715 podStartE2EDuration="21.909200886s" totalImagesPullingTime="20.511836171s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 13:26:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 13:26:21.90715506 +0000 UTC m=+666.365718165" watchObservedRunningTime="2026-07-05 13:26:21.909200886 +0000 UTC m=+666.367763991" Jul 05 13:26:21.963914 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:26:21.963881 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-75cb8f6bcc-dbrfg"] Jul 05 13:26:21.975403 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:26:21.975378 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-75cb8f6bcc-dbrfg"] Jul 05 13:26:22.009491 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:26:22.009462 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bd354538-c932-466f-ae54-9575b7c11f52" path="/var/lib/kubelet/pods/bd354538-c932-466f-ae54-9575b7c11f52/volumes" Jul 05 13:26:46.188195 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:26:46.188153 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-58cd955675-s6pwh"] Jul 05 13:26:46.188808 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:26:46.188681 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bd354538-c932-466f-ae54-9575b7c11f52" containerName="console" Jul 05 13:26:46.188808 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:26:46.188701 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="bd354538-c932-466f-ae54-9575b7c11f52" containerName="console" Jul 05 13:26:46.188808 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:26:46.188800 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="bd354538-c932-466f-ae54-9575b7c11f52" containerName="console" Jul 05 13:26:46.190760 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:26:46.190742 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-s6pwh" Jul 05 13:26:46.194369 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:26:46.194343 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-pcns5\"" Jul 05 13:26:46.209440 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:26:46.209410 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-s6pwh"] Jul 05 13:26:46.285546 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:26:46.285500 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xbcd7\" (UniqueName: \"kubernetes.io/projected/b8b8c4c4-bb44-43a3-ba15-8d65909defb4-kube-api-access-xbcd7\") pod \"authorino-58cd955675-s6pwh\" (UID: \"b8b8c4c4-bb44-43a3-ba15-8d65909defb4\") " pod="kuadrant-system/authorino-58cd955675-s6pwh" Jul 05 13:26:46.386440 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:26:46.386379 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xbcd7\" (UniqueName: \"kubernetes.io/projected/b8b8c4c4-bb44-43a3-ba15-8d65909defb4-kube-api-access-xbcd7\") pod \"authorino-58cd955675-s6pwh\" (UID: \"b8b8c4c4-bb44-43a3-ba15-8d65909defb4\") " pod="kuadrant-system/authorino-58cd955675-s6pwh" Jul 05 13:26:46.409596 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:26:46.409557 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xbcd7\" (UniqueName: \"kubernetes.io/projected/b8b8c4c4-bb44-43a3-ba15-8d65909defb4-kube-api-access-xbcd7\") pod \"authorino-58cd955675-s6pwh\" (UID: \"b8b8c4c4-bb44-43a3-ba15-8d65909defb4\") " pod="kuadrant-system/authorino-58cd955675-s6pwh" Jul 05 13:26:46.499456 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:26:46.499425 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-s6pwh" Jul 05 13:26:46.577010 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:26:46.576977 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-75d556bd77-bgcmg"] Jul 05 13:26:46.579617 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:26:46.579596 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-bgcmg" Jul 05 13:26:46.587921 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:26:46.587892 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4ww5f\" (UniqueName: \"kubernetes.io/projected/87359623-0083-4c03-993a-cfc3b0715b9a-kube-api-access-4ww5f\") pod \"authorino-75d556bd77-bgcmg\" (UID: \"87359623-0083-4c03-993a-cfc3b0715b9a\") " pod="kuadrant-system/authorino-75d556bd77-bgcmg" Jul 05 13:26:46.598125 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:26:46.598043 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-bgcmg"] Jul 05 13:26:46.650893 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:26:46.650862 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-s6pwh"] Jul 05 13:26:46.654060 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:26:46.654034 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb8b8c4c4_bb44_43a3_ba15_8d65909defb4.slice/crio-2d404f2a58aa88efcdbb00f8bc85d734a45b35daceb024abfcb14c852ab45663 WatchSource:0}: Error finding container 2d404f2a58aa88efcdbb00f8bc85d734a45b35daceb024abfcb14c852ab45663: Status 404 returned error can't find the container with id 2d404f2a58aa88efcdbb00f8bc85d734a45b35daceb024abfcb14c852ab45663 Jul 05 13:26:46.689038 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:26:46.688994 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4ww5f\" (UniqueName: \"kubernetes.io/projected/87359623-0083-4c03-993a-cfc3b0715b9a-kube-api-access-4ww5f\") pod \"authorino-75d556bd77-bgcmg\" (UID: \"87359623-0083-4c03-993a-cfc3b0715b9a\") " pod="kuadrant-system/authorino-75d556bd77-bgcmg" Jul 05 13:26:46.698996 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:26:46.698961 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4ww5f\" (UniqueName: \"kubernetes.io/projected/87359623-0083-4c03-993a-cfc3b0715b9a-kube-api-access-4ww5f\") pod \"authorino-75d556bd77-bgcmg\" (UID: \"87359623-0083-4c03-993a-cfc3b0715b9a\") " pod="kuadrant-system/authorino-75d556bd77-bgcmg" Jul 05 13:26:46.889802 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:26:46.889699 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-bgcmg" Jul 05 13:26:46.983805 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:26:46.983771 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-s6pwh" event={"ID":"b8b8c4c4-bb44-43a3-ba15-8d65909defb4","Type":"ContainerStarted","Data":"2d404f2a58aa88efcdbb00f8bc85d734a45b35daceb024abfcb14c852ab45663"} Jul 05 13:26:47.042870 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:26:47.042844 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-bgcmg"] Jul 05 13:26:47.044482 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:26:47.044453 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod87359623_0083_4c03_993a_cfc3b0715b9a.slice/crio-389d2f5581a2315211e7a90b3fb97126c503ea65cd9daf9bd937231824ef0d2a WatchSource:0}: Error finding container 389d2f5581a2315211e7a90b3fb97126c503ea65cd9daf9bd937231824ef0d2a: Status 404 returned error can't find the container with id 389d2f5581a2315211e7a90b3fb97126c503ea65cd9daf9bd937231824ef0d2a Jul 05 13:26:47.997610 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:26:47.997542 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-bgcmg" event={"ID":"87359623-0083-4c03-993a-cfc3b0715b9a","Type":"ContainerStarted","Data":"389d2f5581a2315211e7a90b3fb97126c503ea65cd9daf9bd937231824ef0d2a"} Jul 05 13:26:51.018264 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:26:51.018207 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-bgcmg" event={"ID":"87359623-0083-4c03-993a-cfc3b0715b9a","Type":"ContainerStarted","Data":"44bdf99b48e8746476279941ce69ea1b65f8f3591e9dbff44da608cad599a11c"} Jul 05 13:26:51.019749 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:26:51.019725 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-s6pwh" event={"ID":"b8b8c4c4-bb44-43a3-ba15-8d65909defb4","Type":"ContainerStarted","Data":"feb5a0b01c888cb79478fdb31a8d740223f73c72d5b77ef3d6fcd2a3c18ad554"} Jul 05 13:26:51.044598 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:26:51.044537 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-75d556bd77-bgcmg" podStartSLOduration=1.796287211 podStartE2EDuration="5.044522925s" totalImagesPullingTime="3.248235714s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 13:26:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 13:26:51.041923838 +0000 UTC m=+695.500486943" watchObservedRunningTime="2026-07-05 13:26:51.044522925 +0000 UTC m=+695.503086008" Jul 05 13:26:51.086022 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:26:51.085970 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-58cd955675-s6pwh" podStartSLOduration=1.463969181 podStartE2EDuration="5.085954025s" totalImagesPullingTime="3.621984844s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 13:26:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 13:26:51.082773598 +0000 UTC m=+695.541336705" watchObservedRunningTime="2026-07-05 13:26:51.085954025 +0000 UTC m=+695.544517130" Jul 05 13:26:51.104379 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:26:51.104343 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-s6pwh"] Jul 05 13:26:53.029324 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:26:53.029283 2578 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-58cd955675-s6pwh" podUID="b8b8c4c4-bb44-43a3-ba15-8d65909defb4" containerName="authorino" containerID="cri-o://feb5a0b01c888cb79478fdb31a8d740223f73c72d5b77ef3d6fcd2a3c18ad554" gracePeriod=30 Jul 05 13:26:53.285065 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:26:53.284992 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-s6pwh" Jul 05 13:26:53.351207 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:26:53.351177 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xbcd7\" (UniqueName: \"kubernetes.io/projected/b8b8c4c4-bb44-43a3-ba15-8d65909defb4-kube-api-access-xbcd7\") pod \"b8b8c4c4-bb44-43a3-ba15-8d65909defb4\" (UID: \"b8b8c4c4-bb44-43a3-ba15-8d65909defb4\") " Jul 05 13:26:53.353391 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:26:53.353360 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b8b8c4c4-bb44-43a3-ba15-8d65909defb4-kube-api-access-xbcd7" (OuterVolumeSpecName: "kube-api-access-xbcd7") pod "b8b8c4c4-bb44-43a3-ba15-8d65909defb4" (UID: "b8b8c4c4-bb44-43a3-ba15-8d65909defb4"). InnerVolumeSpecName "kube-api-access-xbcd7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 05 13:26:53.452135 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:26:53.452089 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xbcd7\" (UniqueName: \"kubernetes.io/projected/b8b8c4c4-bb44-43a3-ba15-8d65909defb4-kube-api-access-xbcd7\") on node \"ip-10-0-130-191.ec2.internal\" DevicePath \"\"" Jul 05 13:26:54.035573 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:26:54.035538 2578 generic.go:356] "Generic (PLEG): container finished" podID="b8b8c4c4-bb44-43a3-ba15-8d65909defb4" containerID="feb5a0b01c888cb79478fdb31a8d740223f73c72d5b77ef3d6fcd2a3c18ad554" exitCode=0 Jul 05 13:26:54.036080 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:26:54.035603 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-s6pwh" Jul 05 13:26:54.036080 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:26:54.035607 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-s6pwh" event={"ID":"b8b8c4c4-bb44-43a3-ba15-8d65909defb4","Type":"ContainerDied","Data":"feb5a0b01c888cb79478fdb31a8d740223f73c72d5b77ef3d6fcd2a3c18ad554"} Jul 05 13:26:54.036080 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:26:54.035650 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-s6pwh" event={"ID":"b8b8c4c4-bb44-43a3-ba15-8d65909defb4","Type":"ContainerDied","Data":"2d404f2a58aa88efcdbb00f8bc85d734a45b35daceb024abfcb14c852ab45663"} Jul 05 13:26:54.036080 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:26:54.035673 2578 scope.go:117] "RemoveContainer" containerID="feb5a0b01c888cb79478fdb31a8d740223f73c72d5b77ef3d6fcd2a3c18ad554" Jul 05 13:26:54.044737 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:26:54.044720 2578 scope.go:117] "RemoveContainer" containerID="feb5a0b01c888cb79478fdb31a8d740223f73c72d5b77ef3d6fcd2a3c18ad554" Jul 05 13:26:54.045000 ip-10-0-130-191 kubenswrapper[2578]: E0705 13:26:54.044980 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"feb5a0b01c888cb79478fdb31a8d740223f73c72d5b77ef3d6fcd2a3c18ad554\": container with ID starting with feb5a0b01c888cb79478fdb31a8d740223f73c72d5b77ef3d6fcd2a3c18ad554 not found: ID does not exist" containerID="feb5a0b01c888cb79478fdb31a8d740223f73c72d5b77ef3d6fcd2a3c18ad554" Jul 05 13:26:54.045065 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:26:54.045011 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"feb5a0b01c888cb79478fdb31a8d740223f73c72d5b77ef3d6fcd2a3c18ad554"} err="failed to get container status \"feb5a0b01c888cb79478fdb31a8d740223f73c72d5b77ef3d6fcd2a3c18ad554\": rpc error: code = NotFound desc = could not find container \"feb5a0b01c888cb79478fdb31a8d740223f73c72d5b77ef3d6fcd2a3c18ad554\": container with ID starting with feb5a0b01c888cb79478fdb31a8d740223f73c72d5b77ef3d6fcd2a3c18ad554 not found: ID does not exist" Jul 05 13:26:54.061631 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:26:54.061595 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-s6pwh"] Jul 05 13:26:54.076188 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:26:54.076147 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-58cd955675-s6pwh"] Jul 05 13:26:56.008208 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:26:56.008168 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b8b8c4c4-bb44-43a3-ba15-8d65909defb4" path="/var/lib/kubelet/pods/b8b8c4c4-bb44-43a3-ba15-8d65909defb4/volumes" Jul 05 13:27:19.811900 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:27:19.811863 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5d4b67c446-k8v2t"] Jul 05 13:27:19.812346 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:27:19.812198 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b8b8c4c4-bb44-43a3-ba15-8d65909defb4" containerName="authorino" Jul 05 13:27:19.812346 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:27:19.812209 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="b8b8c4c4-bb44-43a3-ba15-8d65909defb4" containerName="authorino" Jul 05 13:27:19.812346 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:27:19.812269 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="b8b8c4c4-bb44-43a3-ba15-8d65909defb4" containerName="authorino" Jul 05 13:27:19.816548 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:27:19.816527 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-k8v2t" Jul 05 13:27:19.826972 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:27:19.826947 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-k8v2t"] Jul 05 13:27:19.993875 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:27:19.993837 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-62hdj\" (UniqueName: \"kubernetes.io/projected/7c8bba8e-afa1-4b4a-94b2-e612a92db3c4-kube-api-access-62hdj\") pod \"authorino-5d4b67c446-k8v2t\" (UID: \"7c8bba8e-afa1-4b4a-94b2-e612a92db3c4\") " pod="kuadrant-system/authorino-5d4b67c446-k8v2t" Jul 05 13:27:20.080889 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:27:20.080787 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-k8v2t"] Jul 05 13:27:20.081129 ip-10-0-130-191 kubenswrapper[2578]: E0705 13:27:20.081104 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-62hdj], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-5d4b67c446-k8v2t" podUID="7c8bba8e-afa1-4b4a-94b2-e612a92db3c4" Jul 05 13:27:20.094891 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:27:20.094863 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-62hdj\" (UniqueName: \"kubernetes.io/projected/7c8bba8e-afa1-4b4a-94b2-e612a92db3c4-kube-api-access-62hdj\") pod \"authorino-5d4b67c446-k8v2t\" (UID: \"7c8bba8e-afa1-4b4a-94b2-e612a92db3c4\") " pod="kuadrant-system/authorino-5d4b67c446-k8v2t" Jul 05 13:27:20.110503 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:27:20.110470 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-62hdj\" (UniqueName: \"kubernetes.io/projected/7c8bba8e-afa1-4b4a-94b2-e612a92db3c4-kube-api-access-62hdj\") pod \"authorino-5d4b67c446-k8v2t\" (UID: \"7c8bba8e-afa1-4b4a-94b2-e612a92db3c4\") " pod="kuadrant-system/authorino-5d4b67c446-k8v2t" Jul 05 13:27:20.117896 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:27:20.117871 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-84dc7c466-d8cxd"] Jul 05 13:27:20.121261 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:27:20.121245 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-84dc7c466-d8cxd" Jul 05 13:27:20.134772 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:27:20.134742 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-84dc7c466-d8cxd"] Jul 05 13:27:20.168533 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:27:20.168501 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-k8v2t" Jul 05 13:27:20.175173 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:27:20.175151 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-k8v2t" Jul 05 13:27:20.296567 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:27:20.296531 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-62hdj\" (UniqueName: \"kubernetes.io/projected/7c8bba8e-afa1-4b4a-94b2-e612a92db3c4-kube-api-access-62hdj\") pod \"7c8bba8e-afa1-4b4a-94b2-e612a92db3c4\" (UID: \"7c8bba8e-afa1-4b4a-94b2-e612a92db3c4\") " Jul 05 13:27:20.296761 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:27:20.296733 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xvst4\" (UniqueName: \"kubernetes.io/projected/3523ea73-bcd3-4c31-a9bc-fb886293bce1-kube-api-access-xvst4\") pod \"authorino-84dc7c466-d8cxd\" (UID: \"3523ea73-bcd3-4c31-a9bc-fb886293bce1\") " pod="kuadrant-system/authorino-84dc7c466-d8cxd" Jul 05 13:27:20.298798 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:27:20.298770 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7c8bba8e-afa1-4b4a-94b2-e612a92db3c4-kube-api-access-62hdj" (OuterVolumeSpecName: "kube-api-access-62hdj") pod "7c8bba8e-afa1-4b4a-94b2-e612a92db3c4" (UID: "7c8bba8e-afa1-4b4a-94b2-e612a92db3c4"). InnerVolumeSpecName "kube-api-access-62hdj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 05 13:27:20.358630 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:27:20.358526 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-84dc7c466-d8cxd"] Jul 05 13:27:20.358807 ip-10-0-130-191 kubenswrapper[2578]: E0705 13:27:20.358786 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-xvst4], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-84dc7c466-d8cxd" podUID="3523ea73-bcd3-4c31-a9bc-fb886293bce1" Jul 05 13:27:20.397236 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:27:20.397194 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xvst4\" (UniqueName: \"kubernetes.io/projected/3523ea73-bcd3-4c31-a9bc-fb886293bce1-kube-api-access-xvst4\") pod \"authorino-84dc7c466-d8cxd\" (UID: \"3523ea73-bcd3-4c31-a9bc-fb886293bce1\") " pod="kuadrant-system/authorino-84dc7c466-d8cxd" Jul 05 13:27:20.397411 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:27:20.397261 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-62hdj\" (UniqueName: \"kubernetes.io/projected/7c8bba8e-afa1-4b4a-94b2-e612a92db3c4-kube-api-access-62hdj\") on node \"ip-10-0-130-191.ec2.internal\" DevicePath \"\"" Jul 05 13:27:20.428378 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:27:20.428348 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xvst4\" (UniqueName: \"kubernetes.io/projected/3523ea73-bcd3-4c31-a9bc-fb886293bce1-kube-api-access-xvst4\") pod \"authorino-84dc7c466-d8cxd\" (UID: \"3523ea73-bcd3-4c31-a9bc-fb886293bce1\") " pod="kuadrant-system/authorino-84dc7c466-d8cxd" Jul 05 13:27:20.429731 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:27:20.429706 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-59c5956854-rg6gr"] Jul 05 13:27:20.433642 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:27:20.433621 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-59c5956854-rg6gr" Jul 05 13:27:20.444511 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:27:20.444486 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\"" Jul 05 13:27:20.475341 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:27:20.475309 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-59c5956854-rg6gr"] Jul 05 13:27:20.598592 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:27:20.598541 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/cd256815-f3cf-4ffe-a132-9a1d473db8c9-tls-cert\") pod \"authorino-59c5956854-rg6gr\" (UID: \"cd256815-f3cf-4ffe-a132-9a1d473db8c9\") " pod="kuadrant-system/authorino-59c5956854-rg6gr" Jul 05 13:27:20.598793 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:27:20.598634 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cpnh9\" (UniqueName: \"kubernetes.io/projected/cd256815-f3cf-4ffe-a132-9a1d473db8c9-kube-api-access-cpnh9\") pod \"authorino-59c5956854-rg6gr\" (UID: \"cd256815-f3cf-4ffe-a132-9a1d473db8c9\") " pod="kuadrant-system/authorino-59c5956854-rg6gr" Jul 05 13:27:20.700141 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:27:20.700025 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/cd256815-f3cf-4ffe-a132-9a1d473db8c9-tls-cert\") pod \"authorino-59c5956854-rg6gr\" (UID: \"cd256815-f3cf-4ffe-a132-9a1d473db8c9\") " pod="kuadrant-system/authorino-59c5956854-rg6gr" Jul 05 13:27:20.700141 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:27:20.700093 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cpnh9\" (UniqueName: \"kubernetes.io/projected/cd256815-f3cf-4ffe-a132-9a1d473db8c9-kube-api-access-cpnh9\") pod \"authorino-59c5956854-rg6gr\" (UID: \"cd256815-f3cf-4ffe-a132-9a1d473db8c9\") " pod="kuadrant-system/authorino-59c5956854-rg6gr" Jul 05 13:27:20.702625 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:27:20.702596 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/cd256815-f3cf-4ffe-a132-9a1d473db8c9-tls-cert\") pod \"authorino-59c5956854-rg6gr\" (UID: \"cd256815-f3cf-4ffe-a132-9a1d473db8c9\") " pod="kuadrant-system/authorino-59c5956854-rg6gr" Jul 05 13:27:20.721872 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:27:20.721838 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cpnh9\" (UniqueName: \"kubernetes.io/projected/cd256815-f3cf-4ffe-a132-9a1d473db8c9-kube-api-access-cpnh9\") pod \"authorino-59c5956854-rg6gr\" (UID: \"cd256815-f3cf-4ffe-a132-9a1d473db8c9\") " pod="kuadrant-system/authorino-59c5956854-rg6gr" Jul 05 13:27:20.745487 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:27:20.745456 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-59c5956854-rg6gr" Jul 05 13:27:20.904860 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:27:20.904828 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-59c5956854-rg6gr"] Jul 05 13:27:20.906845 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:27:20.906807 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podcd256815_f3cf_4ffe_a132_9a1d473db8c9.slice/crio-192ceb5de2a78c9d9ef9b261d307f24648deb58df618a4bf4a9b26fb399fb752 WatchSource:0}: Error finding container 192ceb5de2a78c9d9ef9b261d307f24648deb58df618a4bf4a9b26fb399fb752: Status 404 returned error can't find the container with id 192ceb5de2a78c9d9ef9b261d307f24648deb58df618a4bf4a9b26fb399fb752 Jul 05 13:27:21.175543 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:27:21.175512 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-59c5956854-rg6gr" event={"ID":"cd256815-f3cf-4ffe-a132-9a1d473db8c9","Type":"ContainerStarted","Data":"192ceb5de2a78c9d9ef9b261d307f24648deb58df618a4bf4a9b26fb399fb752"} Jul 05 13:27:21.175543 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:27:21.175518 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-84dc7c466-d8cxd" Jul 05 13:27:21.175804 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:27:21.175559 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-k8v2t" Jul 05 13:27:21.184994 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:27:21.184974 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-84dc7c466-d8cxd" Jul 05 13:27:21.228608 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:27:21.228547 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-k8v2t"] Jul 05 13:27:21.231241 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:27:21.231209 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-k8v2t"] Jul 05 13:27:21.305620 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:27:21.305572 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xvst4\" (UniqueName: \"kubernetes.io/projected/3523ea73-bcd3-4c31-a9bc-fb886293bce1-kube-api-access-xvst4\") pod \"3523ea73-bcd3-4c31-a9bc-fb886293bce1\" (UID: \"3523ea73-bcd3-4c31-a9bc-fb886293bce1\") " Jul 05 13:27:21.307975 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:27:21.307939 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3523ea73-bcd3-4c31-a9bc-fb886293bce1-kube-api-access-xvst4" (OuterVolumeSpecName: "kube-api-access-xvst4") pod "3523ea73-bcd3-4c31-a9bc-fb886293bce1" (UID: "3523ea73-bcd3-4c31-a9bc-fb886293bce1"). InnerVolumeSpecName "kube-api-access-xvst4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 05 13:27:21.407290 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:27:21.407255 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xvst4\" (UniqueName: \"kubernetes.io/projected/3523ea73-bcd3-4c31-a9bc-fb886293bce1-kube-api-access-xvst4\") on node \"ip-10-0-130-191.ec2.internal\" DevicePath \"\"" Jul 05 13:27:22.008746 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:27:22.008710 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7c8bba8e-afa1-4b4a-94b2-e612a92db3c4" path="/var/lib/kubelet/pods/7c8bba8e-afa1-4b4a-94b2-e612a92db3c4/volumes" Jul 05 13:27:22.182663 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:27:22.182614 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-59c5956854-rg6gr" event={"ID":"cd256815-f3cf-4ffe-a132-9a1d473db8c9","Type":"ContainerStarted","Data":"e3799010864463b12984840f3c76e113460e129c8d15a64b510ca4c7b4755f86"} Jul 05 13:27:22.182865 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:27:22.182679 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-84dc7c466-d8cxd" Jul 05 13:27:22.206519 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:27:22.206468 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-59c5956854-rg6gr" podStartSLOduration=1.856908744 podStartE2EDuration="2.206451893s" totalImagesPullingTime="349.543149ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 13:27:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 13:27:22.205447618 +0000 UTC m=+726.664010727" watchObservedRunningTime="2026-07-05 13:27:22.206451893 +0000 UTC m=+726.665014999" Jul 05 13:27:22.240215 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:27:22.240182 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-bgcmg"] Jul 05 13:27:22.240442 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:27:22.240403 2578 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-75d556bd77-bgcmg" podUID="87359623-0083-4c03-993a-cfc3b0715b9a" containerName="authorino" containerID="cri-o://44bdf99b48e8746476279941ce69ea1b65f8f3591e9dbff44da608cad599a11c" gracePeriod=30 Jul 05 13:27:22.248656 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:27:22.248628 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-84dc7c466-d8cxd"] Jul 05 13:27:22.254451 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:27:22.254425 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-84dc7c466-d8cxd"] Jul 05 13:27:22.527000 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:27:22.526967 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-bgcmg" Jul 05 13:27:22.718862 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:27:22.718771 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4ww5f\" (UniqueName: \"kubernetes.io/projected/87359623-0083-4c03-993a-cfc3b0715b9a-kube-api-access-4ww5f\") pod \"87359623-0083-4c03-993a-cfc3b0715b9a\" (UID: \"87359623-0083-4c03-993a-cfc3b0715b9a\") " Jul 05 13:27:22.721020 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:27:22.720994 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/87359623-0083-4c03-993a-cfc3b0715b9a-kube-api-access-4ww5f" (OuterVolumeSpecName: "kube-api-access-4ww5f") pod "87359623-0083-4c03-993a-cfc3b0715b9a" (UID: "87359623-0083-4c03-993a-cfc3b0715b9a"). InnerVolumeSpecName "kube-api-access-4ww5f". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 05 13:27:22.819938 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:27:22.819897 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4ww5f\" (UniqueName: \"kubernetes.io/projected/87359623-0083-4c03-993a-cfc3b0715b9a-kube-api-access-4ww5f\") on node \"ip-10-0-130-191.ec2.internal\" DevicePath \"\"" Jul 05 13:27:23.188799 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:27:23.188765 2578 generic.go:356] "Generic (PLEG): container finished" podID="87359623-0083-4c03-993a-cfc3b0715b9a" containerID="44bdf99b48e8746476279941ce69ea1b65f8f3591e9dbff44da608cad599a11c" exitCode=0 Jul 05 13:27:23.188799 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:27:23.188800 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-bgcmg" Jul 05 13:27:23.189277 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:27:23.188847 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-bgcmg" event={"ID":"87359623-0083-4c03-993a-cfc3b0715b9a","Type":"ContainerDied","Data":"44bdf99b48e8746476279941ce69ea1b65f8f3591e9dbff44da608cad599a11c"} Jul 05 13:27:23.189277 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:27:23.188894 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-bgcmg" event={"ID":"87359623-0083-4c03-993a-cfc3b0715b9a","Type":"ContainerDied","Data":"389d2f5581a2315211e7a90b3fb97126c503ea65cd9daf9bd937231824ef0d2a"} Jul 05 13:27:23.189277 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:27:23.188911 2578 scope.go:117] "RemoveContainer" containerID="44bdf99b48e8746476279941ce69ea1b65f8f3591e9dbff44da608cad599a11c" Jul 05 13:27:23.198003 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:27:23.197983 2578 scope.go:117] "RemoveContainer" containerID="44bdf99b48e8746476279941ce69ea1b65f8f3591e9dbff44da608cad599a11c" Jul 05 13:27:23.198275 ip-10-0-130-191 kubenswrapper[2578]: E0705 13:27:23.198256 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"44bdf99b48e8746476279941ce69ea1b65f8f3591e9dbff44da608cad599a11c\": container with ID starting with 44bdf99b48e8746476279941ce69ea1b65f8f3591e9dbff44da608cad599a11c not found: ID does not exist" containerID="44bdf99b48e8746476279941ce69ea1b65f8f3591e9dbff44da608cad599a11c" Jul 05 13:27:23.198323 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:27:23.198284 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"44bdf99b48e8746476279941ce69ea1b65f8f3591e9dbff44da608cad599a11c"} err="failed to get container status \"44bdf99b48e8746476279941ce69ea1b65f8f3591e9dbff44da608cad599a11c\": rpc error: code = NotFound desc = could not find container \"44bdf99b48e8746476279941ce69ea1b65f8f3591e9dbff44da608cad599a11c\": container with ID starting with 44bdf99b48e8746476279941ce69ea1b65f8f3591e9dbff44da608cad599a11c not found: ID does not exist" Jul 05 13:27:23.214266 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:27:23.214242 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-bgcmg"] Jul 05 13:27:23.233072 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:27:23.233043 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-75d556bd77-bgcmg"] Jul 05 13:27:24.009092 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:27:24.009060 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3523ea73-bcd3-4c31-a9bc-fb886293bce1" path="/var/lib/kubelet/pods/3523ea73-bcd3-4c31-a9bc-fb886293bce1/volumes" Jul 05 13:27:24.009283 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:27:24.009271 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="87359623-0083-4c03-993a-cfc3b0715b9a" path="/var/lib/kubelet/pods/87359623-0083-4c03-993a-cfc3b0715b9a/volumes" Jul 05 13:28:02.787919 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:28:02.787891 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-65c5985bdd-gfz4q"] Jul 05 13:28:02.788363 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:28:02.788220 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="87359623-0083-4c03-993a-cfc3b0715b9a" containerName="authorino" Jul 05 13:28:02.788363 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:28:02.788230 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="87359623-0083-4c03-993a-cfc3b0715b9a" containerName="authorino" Jul 05 13:28:02.788363 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:28:02.788294 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="87359623-0083-4c03-993a-cfc3b0715b9a" containerName="authorino" Jul 05 13:28:02.791406 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:28:02.791390 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-65c5985bdd-gfz4q" Jul 05 13:28:02.798326 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:28:02.798028 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-serving-cert\"" Jul 05 13:28:02.798326 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:28:02.798191 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-sdtdk\"" Jul 05 13:28:02.804122 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:28:02.804098 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-65c5985bdd-gfz4q"] Jul 05 13:28:02.867155 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:28:02.867109 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0fd9cbe7-4dfd-4e71-bc32-503f6360d12b-maas-api-tls\") pod \"maas-api-65c5985bdd-gfz4q\" (UID: \"0fd9cbe7-4dfd-4e71-bc32-503f6360d12b\") " pod="opendatahub/maas-api-65c5985bdd-gfz4q" Jul 05 13:28:02.867334 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:28:02.867163 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vcxbn\" (UniqueName: \"kubernetes.io/projected/0fd9cbe7-4dfd-4e71-bc32-503f6360d12b-kube-api-access-vcxbn\") pod \"maas-api-65c5985bdd-gfz4q\" (UID: \"0fd9cbe7-4dfd-4e71-bc32-503f6360d12b\") " pod="opendatahub/maas-api-65c5985bdd-gfz4q" Jul 05 13:28:02.867334 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:28:02.867259 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0fd9cbe7-4dfd-4e71-bc32-503f6360d12b-openshift-service-ca\") pod \"maas-api-65c5985bdd-gfz4q\" (UID: \"0fd9cbe7-4dfd-4e71-bc32-503f6360d12b\") " pod="opendatahub/maas-api-65c5985bdd-gfz4q" Jul 05 13:28:02.968555 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:28:02.968514 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0fd9cbe7-4dfd-4e71-bc32-503f6360d12b-maas-api-tls\") pod \"maas-api-65c5985bdd-gfz4q\" (UID: \"0fd9cbe7-4dfd-4e71-bc32-503f6360d12b\") " pod="opendatahub/maas-api-65c5985bdd-gfz4q" Jul 05 13:28:02.968757 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:28:02.968563 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vcxbn\" (UniqueName: \"kubernetes.io/projected/0fd9cbe7-4dfd-4e71-bc32-503f6360d12b-kube-api-access-vcxbn\") pod \"maas-api-65c5985bdd-gfz4q\" (UID: \"0fd9cbe7-4dfd-4e71-bc32-503f6360d12b\") " pod="opendatahub/maas-api-65c5985bdd-gfz4q" Jul 05 13:28:02.968757 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:28:02.968619 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0fd9cbe7-4dfd-4e71-bc32-503f6360d12b-openshift-service-ca\") pod \"maas-api-65c5985bdd-gfz4q\" (UID: \"0fd9cbe7-4dfd-4e71-bc32-503f6360d12b\") " pod="opendatahub/maas-api-65c5985bdd-gfz4q" Jul 05 13:28:02.969208 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:28:02.969189 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0fd9cbe7-4dfd-4e71-bc32-503f6360d12b-openshift-service-ca\") pod \"maas-api-65c5985bdd-gfz4q\" (UID: \"0fd9cbe7-4dfd-4e71-bc32-503f6360d12b\") " pod="opendatahub/maas-api-65c5985bdd-gfz4q" Jul 05 13:28:02.971134 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:28:02.971113 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0fd9cbe7-4dfd-4e71-bc32-503f6360d12b-maas-api-tls\") pod \"maas-api-65c5985bdd-gfz4q\" (UID: \"0fd9cbe7-4dfd-4e71-bc32-503f6360d12b\") " pod="opendatahub/maas-api-65c5985bdd-gfz4q" Jul 05 13:28:02.984495 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:28:02.984466 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vcxbn\" (UniqueName: \"kubernetes.io/projected/0fd9cbe7-4dfd-4e71-bc32-503f6360d12b-kube-api-access-vcxbn\") pod \"maas-api-65c5985bdd-gfz4q\" (UID: \"0fd9cbe7-4dfd-4e71-bc32-503f6360d12b\") " pod="opendatahub/maas-api-65c5985bdd-gfz4q" Jul 05 13:28:03.102207 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:28:03.102124 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-65c5985bdd-gfz4q" Jul 05 13:28:03.240083 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:28:03.240055 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-65c5985bdd-gfz4q"] Jul 05 13:28:03.241641 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:28:03.241617 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0fd9cbe7_4dfd_4e71_bc32_503f6360d12b.slice/crio-27c70ddc75f9dc437da8fa4dd2c10fe93667209f662aab769dd3739e5f9e9320 WatchSource:0}: Error finding container 27c70ddc75f9dc437da8fa4dd2c10fe93667209f662aab769dd3739e5f9e9320: Status 404 returned error can't find the container with id 27c70ddc75f9dc437da8fa4dd2c10fe93667209f662aab769dd3739e5f9e9320 Jul 05 13:28:03.399875 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:28:03.399798 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-65c5985bdd-gfz4q" event={"ID":"0fd9cbe7-4dfd-4e71-bc32-503f6360d12b","Type":"ContainerStarted","Data":"27c70ddc75f9dc437da8fa4dd2c10fe93667209f662aab769dd3739e5f9e9320"} Jul 05 13:28:03.444335 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:28:03.444297 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-7964457696-wn6j4"] Jul 05 13:28:03.448860 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:28:03.448836 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7964457696-wn6j4" Jul 05 13:28:03.452906 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:28:03.452883 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-nf6m4\"" Jul 05 13:28:03.453123 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:28:03.453104 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\"" Jul 05 13:28:03.467952 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:28:03.467925 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7964457696-wn6j4"] Jul 05 13:28:03.573703 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:28:03.573658 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fzfc7\" (UniqueName: \"kubernetes.io/projected/0e615cfd-cbda-41d6-af90-ad6958fa40c9-kube-api-access-fzfc7\") pod \"payload-pre-processing-7964457696-wn6j4\" (UID: \"0e615cfd-cbda-41d6-af90-ad6958fa40c9\") " pod="openshift-ingress/payload-pre-processing-7964457696-wn6j4" Jul 05 13:28:03.573893 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:28:03.573722 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0e615cfd-cbda-41d6-af90-ad6958fa40c9-plugins-config-volume\") pod \"payload-pre-processing-7964457696-wn6j4\" (UID: \"0e615cfd-cbda-41d6-af90-ad6958fa40c9\") " pod="openshift-ingress/payload-pre-processing-7964457696-wn6j4" Jul 05 13:28:03.675005 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:28:03.674915 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0e615cfd-cbda-41d6-af90-ad6958fa40c9-plugins-config-volume\") pod \"payload-pre-processing-7964457696-wn6j4\" (UID: \"0e615cfd-cbda-41d6-af90-ad6958fa40c9\") " pod="openshift-ingress/payload-pre-processing-7964457696-wn6j4" Jul 05 13:28:03.675180 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:28:03.675031 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fzfc7\" (UniqueName: \"kubernetes.io/projected/0e615cfd-cbda-41d6-af90-ad6958fa40c9-kube-api-access-fzfc7\") pod \"payload-pre-processing-7964457696-wn6j4\" (UID: \"0e615cfd-cbda-41d6-af90-ad6958fa40c9\") " pod="openshift-ingress/payload-pre-processing-7964457696-wn6j4" Jul 05 13:28:03.675634 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:28:03.675611 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0e615cfd-cbda-41d6-af90-ad6958fa40c9-plugins-config-volume\") pod \"payload-pre-processing-7964457696-wn6j4\" (UID: \"0e615cfd-cbda-41d6-af90-ad6958fa40c9\") " pod="openshift-ingress/payload-pre-processing-7964457696-wn6j4" Jul 05 13:28:03.683560 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:28:03.683534 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fzfc7\" (UniqueName: \"kubernetes.io/projected/0e615cfd-cbda-41d6-af90-ad6958fa40c9-kube-api-access-fzfc7\") pod \"payload-pre-processing-7964457696-wn6j4\" (UID: \"0e615cfd-cbda-41d6-af90-ad6958fa40c9\") " pod="openshift-ingress/payload-pre-processing-7964457696-wn6j4" Jul 05 13:28:03.759094 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:28:03.759064 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7964457696-wn6j4" Jul 05 13:28:03.922991 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:28:03.922902 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7964457696-wn6j4"] Jul 05 13:28:03.924754 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:28:03.924720 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0e615cfd_cbda_41d6_af90_ad6958fa40c9.slice/crio-a30d59a67058014476895f3a5dfe2bbe2ce30f2ea9773443e135164751d177e0 WatchSource:0}: Error finding container a30d59a67058014476895f3a5dfe2bbe2ce30f2ea9773443e135164751d177e0: Status 404 returned error can't find the container with id a30d59a67058014476895f3a5dfe2bbe2ce30f2ea9773443e135164751d177e0 Jul 05 13:28:04.406653 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:28:04.406613 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7964457696-wn6j4" event={"ID":"0e615cfd-cbda-41d6-af90-ad6958fa40c9","Type":"ContainerStarted","Data":"a30d59a67058014476895f3a5dfe2bbe2ce30f2ea9773443e135164751d177e0"} Jul 05 13:28:05.413808 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:28:05.413768 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-65c5985bdd-gfz4q" event={"ID":"0fd9cbe7-4dfd-4e71-bc32-503f6360d12b","Type":"ContainerStarted","Data":"9a3969ec7075df21e19763b952e6654605db7a2f0ccffd12665186d447b7a3d0"} Jul 05 13:28:05.414295 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:28:05.413962 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-65c5985bdd-gfz4q" Jul 05 13:28:05.415513 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:28:05.415486 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7964457696-wn6j4" event={"ID":"0e615cfd-cbda-41d6-af90-ad6958fa40c9","Type":"ContainerStarted","Data":"3851f8100a46cbdd73b6b505e294014f4baa63fbea0746e87a3fa07913791ced"} Jul 05 13:28:05.415653 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:28:05.415535 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-7964457696-wn6j4" Jul 05 13:28:05.434870 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:28:05.434818 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-65c5985bdd-gfz4q" podStartSLOduration=1.811104289 podStartE2EDuration="3.43480236s" totalImagesPullingTime="1.623698071s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 13:28:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 13:28:05.433182823 +0000 UTC m=+769.891745940" watchObservedRunningTime="2026-07-05 13:28:05.43480236 +0000 UTC m=+769.893365465" Jul 05 13:28:05.455024 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:28:05.451803 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-7964457696-wn6j4" podStartSLOduration=1.086211343 podStartE2EDuration="2.451779811s" totalImagesPullingTime="1.365568468s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 13:28:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 13:28:05.448763756 +0000 UTC m=+769.907326855" watchObservedRunningTime="2026-07-05 13:28:05.451779811 +0000 UTC m=+769.910342916" Jul 05 13:28:11.426938 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:28:11.426911 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-65c5985bdd-gfz4q" Jul 05 13:28:16.423503 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:28:16.423472 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-7964457696-wn6j4" Jul 05 13:28:26.247408 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:28:26.247373 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-x9kvt"] Jul 05 13:28:26.252431 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:28:26.252412 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-x9kvt" Jul 05 13:28:26.255324 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:28:26.255302 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 05 13:28:26.256557 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:28:26.256540 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 05 13:28:26.256728 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:28:26.256705 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-sxwzc\"" Jul 05 13:28:26.256799 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:28:26.256728 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\"" Jul 05 13:28:26.262217 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:28:26.262192 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-x9kvt"] Jul 05 13:28:26.273091 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:28:26.273063 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8fd8e4e7-246a-4591-94c9-d7e662169ca8-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-x9kvt\" (UID: \"8fd8e4e7-246a-4591-94c9-d7e662169ca8\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-x9kvt" Jul 05 13:28:26.273192 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:28:26.273103 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8fd8e4e7-246a-4591-94c9-d7e662169ca8-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-x9kvt\" (UID: \"8fd8e4e7-246a-4591-94c9-d7e662169ca8\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-x9kvt" Jul 05 13:28:26.273192 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:28:26.273183 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8fd8e4e7-246a-4591-94c9-d7e662169ca8-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-x9kvt\" (UID: \"8fd8e4e7-246a-4591-94c9-d7e662169ca8\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-x9kvt" Jul 05 13:28:26.273274 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:28:26.273247 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8fd8e4e7-246a-4591-94c9-d7e662169ca8-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-x9kvt\" (UID: \"8fd8e4e7-246a-4591-94c9-d7e662169ca8\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-x9kvt" Jul 05 13:28:26.273316 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:28:26.273302 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rqs9z\" (UniqueName: \"kubernetes.io/projected/8fd8e4e7-246a-4591-94c9-d7e662169ca8-kube-api-access-rqs9z\") pod \"e2e-distinct-simulated-kserve-59975d69f9-x9kvt\" (UID: \"8fd8e4e7-246a-4591-94c9-d7e662169ca8\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-x9kvt" Jul 05 13:28:26.273358 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:28:26.273350 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8fd8e4e7-246a-4591-94c9-d7e662169ca8-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-x9kvt\" (UID: \"8fd8e4e7-246a-4591-94c9-d7e662169ca8\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-x9kvt" Jul 05 13:28:26.374332 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:28:26.374298 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8fd8e4e7-246a-4591-94c9-d7e662169ca8-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-x9kvt\" (UID: \"8fd8e4e7-246a-4591-94c9-d7e662169ca8\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-x9kvt" Jul 05 13:28:26.374332 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:28:26.374332 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8fd8e4e7-246a-4591-94c9-d7e662169ca8-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-x9kvt\" (UID: \"8fd8e4e7-246a-4591-94c9-d7e662169ca8\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-x9kvt" Jul 05 13:28:26.374938 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:28:26.374365 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8fd8e4e7-246a-4591-94c9-d7e662169ca8-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-x9kvt\" (UID: \"8fd8e4e7-246a-4591-94c9-d7e662169ca8\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-x9kvt" Jul 05 13:28:26.374938 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:28:26.374400 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8fd8e4e7-246a-4591-94c9-d7e662169ca8-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-x9kvt\" (UID: \"8fd8e4e7-246a-4591-94c9-d7e662169ca8\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-x9kvt" Jul 05 13:28:26.374938 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:28:26.374434 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rqs9z\" (UniqueName: \"kubernetes.io/projected/8fd8e4e7-246a-4591-94c9-d7e662169ca8-kube-api-access-rqs9z\") pod \"e2e-distinct-simulated-kserve-59975d69f9-x9kvt\" (UID: \"8fd8e4e7-246a-4591-94c9-d7e662169ca8\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-x9kvt" Jul 05 13:28:26.374938 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:28:26.374480 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8fd8e4e7-246a-4591-94c9-d7e662169ca8-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-x9kvt\" (UID: \"8fd8e4e7-246a-4591-94c9-d7e662169ca8\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-x9kvt" Jul 05 13:28:26.374938 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:28:26.374758 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8fd8e4e7-246a-4591-94c9-d7e662169ca8-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-x9kvt\" (UID: \"8fd8e4e7-246a-4591-94c9-d7e662169ca8\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-x9kvt" Jul 05 13:28:26.374938 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:28:26.374826 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8fd8e4e7-246a-4591-94c9-d7e662169ca8-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-x9kvt\" (UID: \"8fd8e4e7-246a-4591-94c9-d7e662169ca8\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-x9kvt" Jul 05 13:28:26.374938 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:28:26.374857 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8fd8e4e7-246a-4591-94c9-d7e662169ca8-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-x9kvt\" (UID: \"8fd8e4e7-246a-4591-94c9-d7e662169ca8\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-x9kvt" Jul 05 13:28:26.376968 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:28:26.376946 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8fd8e4e7-246a-4591-94c9-d7e662169ca8-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-x9kvt\" (UID: \"8fd8e4e7-246a-4591-94c9-d7e662169ca8\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-x9kvt" Jul 05 13:28:26.377066 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:28:26.377027 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8fd8e4e7-246a-4591-94c9-d7e662169ca8-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-x9kvt\" (UID: \"8fd8e4e7-246a-4591-94c9-d7e662169ca8\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-x9kvt" Jul 05 13:28:26.382791 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:28:26.382770 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rqs9z\" (UniqueName: \"kubernetes.io/projected/8fd8e4e7-246a-4591-94c9-d7e662169ca8-kube-api-access-rqs9z\") pod \"e2e-distinct-simulated-kserve-59975d69f9-x9kvt\" (UID: \"8fd8e4e7-246a-4591-94c9-d7e662169ca8\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-x9kvt" Jul 05 13:28:26.564046 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:28:26.563948 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-x9kvt" Jul 05 13:28:26.709199 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:28:26.709168 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-x9kvt"] Jul 05 13:28:26.710545 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:28:26.710515 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8fd8e4e7_246a_4591_94c9_d7e662169ca8.slice/crio-371fc162b3793e89acf62cb6c8d5344f9c2f825022f4b89d917a07631b0ec7a5 WatchSource:0}: Error finding container 371fc162b3793e89acf62cb6c8d5344f9c2f825022f4b89d917a07631b0ec7a5: Status 404 returned error can't find the container with id 371fc162b3793e89acf62cb6c8d5344f9c2f825022f4b89d917a07631b0ec7a5 Jul 05 13:28:27.541799 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:28:27.541756 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-x9kvt" event={"ID":"8fd8e4e7-246a-4591-94c9-d7e662169ca8","Type":"ContainerStarted","Data":"371fc162b3793e89acf62cb6c8d5344f9c2f825022f4b89d917a07631b0ec7a5"} Jul 05 13:28:28.548989 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:28:28.548951 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-x9kvt" event={"ID":"8fd8e4e7-246a-4591-94c9-d7e662169ca8","Type":"ContainerStarted","Data":"b0238de44547974f922efef42145e6b4c0895a12abad6691536b328b750fddef"} Jul 05 13:28:28.570272 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:28:28.570220 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-x9kvt" podStartSLOduration=1.489933895 podStartE2EDuration="2.570206427s" totalImagesPullingTime="1.080272532s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 13:28:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 13:28:28.567234304 +0000 UTC m=+793.025797410" watchObservedRunningTime="2026-07-05 13:28:28.570206427 +0000 UTC m=+793.028769532" Jul 05 13:28:36.564692 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:28:36.564647 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-x9kvt" Jul 05 13:28:36.565192 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:28:36.564722 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-x9kvt" Jul 05 13:28:36.577801 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:28:36.577760 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-x9kvt" Jul 05 13:28:36.607864 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:28:36.607834 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-x9kvt" Jul 05 13:28:40.070341 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:28:40.070311 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbvcspf"] Jul 05 13:28:40.075914 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:28:40.075893 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbvcspf" Jul 05 13:28:40.081050 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:28:40.081027 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\"" Jul 05 13:28:40.087252 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:28:40.087226 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbvcspf"] Jul 05 13:28:40.191539 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:28:40.191497 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/155ef003-5f1d-49d9-be0d-48a9942cf9a7-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbvcspf\" (UID: \"155ef003-5f1d-49d9-be0d-48a9942cf9a7\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbvcspf" Jul 05 13:28:40.191539 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:28:40.191545 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/155ef003-5f1d-49d9-be0d-48a9942cf9a7-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbvcspf\" (UID: \"155ef003-5f1d-49d9-be0d-48a9942cf9a7\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbvcspf" Jul 05 13:28:40.191785 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:28:40.191637 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/155ef003-5f1d-49d9-be0d-48a9942cf9a7-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbvcspf\" (UID: \"155ef003-5f1d-49d9-be0d-48a9942cf9a7\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbvcspf" Jul 05 13:28:40.191785 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:28:40.191671 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/155ef003-5f1d-49d9-be0d-48a9942cf9a7-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbvcspf\" (UID: \"155ef003-5f1d-49d9-be0d-48a9942cf9a7\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbvcspf" Jul 05 13:28:40.191785 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:28:40.191696 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/155ef003-5f1d-49d9-be0d-48a9942cf9a7-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbvcspf\" (UID: \"155ef003-5f1d-49d9-be0d-48a9942cf9a7\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbvcspf" Jul 05 13:28:40.191785 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:28:40.191715 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q26jk\" (UniqueName: \"kubernetes.io/projected/155ef003-5f1d-49d9-be0d-48a9942cf9a7-kube-api-access-q26jk\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbvcspf\" (UID: \"155ef003-5f1d-49d9-be0d-48a9942cf9a7\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbvcspf" Jul 05 13:28:40.292292 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:28:40.292245 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/155ef003-5f1d-49d9-be0d-48a9942cf9a7-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbvcspf\" (UID: \"155ef003-5f1d-49d9-be0d-48a9942cf9a7\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbvcspf" Jul 05 13:28:40.292473 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:28:40.292311 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/155ef003-5f1d-49d9-be0d-48a9942cf9a7-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbvcspf\" (UID: \"155ef003-5f1d-49d9-be0d-48a9942cf9a7\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbvcspf" Jul 05 13:28:40.292473 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:28:40.292352 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/155ef003-5f1d-49d9-be0d-48a9942cf9a7-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbvcspf\" (UID: \"155ef003-5f1d-49d9-be0d-48a9942cf9a7\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbvcspf" Jul 05 13:28:40.292473 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:28:40.292381 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q26jk\" (UniqueName: \"kubernetes.io/projected/155ef003-5f1d-49d9-be0d-48a9942cf9a7-kube-api-access-q26jk\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbvcspf\" (UID: \"155ef003-5f1d-49d9-be0d-48a9942cf9a7\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbvcspf" Jul 05 13:28:40.292473 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:28:40.292464 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/155ef003-5f1d-49d9-be0d-48a9942cf9a7-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbvcspf\" (UID: \"155ef003-5f1d-49d9-be0d-48a9942cf9a7\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbvcspf" Jul 05 13:28:40.292728 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:28:40.292505 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/155ef003-5f1d-49d9-be0d-48a9942cf9a7-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbvcspf\" (UID: \"155ef003-5f1d-49d9-be0d-48a9942cf9a7\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbvcspf" Jul 05 13:28:40.292855 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:28:40.292823 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/155ef003-5f1d-49d9-be0d-48a9942cf9a7-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbvcspf\" (UID: \"155ef003-5f1d-49d9-be0d-48a9942cf9a7\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbvcspf" Jul 05 13:28:40.292921 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:28:40.292888 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/155ef003-5f1d-49d9-be0d-48a9942cf9a7-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbvcspf\" (UID: \"155ef003-5f1d-49d9-be0d-48a9942cf9a7\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbvcspf" Jul 05 13:28:40.292966 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:28:40.292922 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/155ef003-5f1d-49d9-be0d-48a9942cf9a7-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbvcspf\" (UID: \"155ef003-5f1d-49d9-be0d-48a9942cf9a7\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbvcspf" Jul 05 13:28:40.294809 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:28:40.294792 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/155ef003-5f1d-49d9-be0d-48a9942cf9a7-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbvcspf\" (UID: \"155ef003-5f1d-49d9-be0d-48a9942cf9a7\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbvcspf" Jul 05 13:28:40.295067 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:28:40.295051 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/155ef003-5f1d-49d9-be0d-48a9942cf9a7-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbvcspf\" (UID: \"155ef003-5f1d-49d9-be0d-48a9942cf9a7\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbvcspf" Jul 05 13:28:40.304274 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:28:40.304244 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q26jk\" (UniqueName: \"kubernetes.io/projected/155ef003-5f1d-49d9-be0d-48a9942cf9a7-kube-api-access-q26jk\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbvcspf\" (UID: \"155ef003-5f1d-49d9-be0d-48a9942cf9a7\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbvcspf" Jul 05 13:28:40.387397 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:28:40.387289 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbvcspf" Jul 05 13:28:40.525793 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:28:40.525766 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbvcspf"] Jul 05 13:28:40.527360 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:28:40.527330 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod155ef003_5f1d_49d9_be0d_48a9942cf9a7.slice/crio-5c2a4254f01dad0636cb7700e2090fea1858377534046f321ef6bdc1289e47e7 WatchSource:0}: Error finding container 5c2a4254f01dad0636cb7700e2090fea1858377534046f321ef6bdc1289e47e7: Status 404 returned error can't find the container with id 5c2a4254f01dad0636cb7700e2090fea1858377534046f321ef6bdc1289e47e7 Jul 05 13:28:40.614526 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:28:40.614489 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbvcspf" event={"ID":"155ef003-5f1d-49d9-be0d-48a9942cf9a7","Type":"ContainerStarted","Data":"5c2a4254f01dad0636cb7700e2090fea1858377534046f321ef6bdc1289e47e7"} Jul 05 13:28:41.621784 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:28:41.621749 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbvcspf" event={"ID":"155ef003-5f1d-49d9-be0d-48a9942cf9a7","Type":"ContainerStarted","Data":"e3d901032e0c148d56597fc1e332364b02c427f5f91edcef69b7143517a0ca2f"} Jul 05 13:28:41.643313 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:28:41.643260 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbvcspf" podStartSLOduration=1.464826406 podStartE2EDuration="1.643243475s" totalImagesPullingTime="178.417069ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 13:28:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 13:28:41.642068718 +0000 UTC m=+806.100631826" watchObservedRunningTime="2026-07-05 13:28:41.643243475 +0000 UTC m=+806.101806579" Jul 05 13:28:50.387657 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:28:50.387614 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbvcspf" Jul 05 13:28:50.388133 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:28:50.387691 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbvcspf" Jul 05 13:28:50.400541 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:28:50.400507 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbvcspf" Jul 05 13:28:50.682274 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:28:50.682190 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbvcspf" Jul 05 13:30:00.169972 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:30:00.169938 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-key-cleanup-29720970-bbnmt"] Jul 05 13:30:00.173251 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:30:00.173235 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29720970-bbnmt" Jul 05 13:30:00.196675 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:30:00.196646 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-29720970-bbnmt"] Jul 05 13:30:00.316534 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:30:00.316482 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sv2ql\" (UniqueName: \"kubernetes.io/projected/053537e5-cdbd-476d-a1d5-463b3c95548e-kube-api-access-sv2ql\") pod \"maas-api-key-cleanup-29720970-bbnmt\" (UID: \"053537e5-cdbd-476d-a1d5-463b3c95548e\") " pod="opendatahub/maas-api-key-cleanup-29720970-bbnmt" Jul 05 13:30:00.417274 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:30:00.417242 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sv2ql\" (UniqueName: \"kubernetes.io/projected/053537e5-cdbd-476d-a1d5-463b3c95548e-kube-api-access-sv2ql\") pod \"maas-api-key-cleanup-29720970-bbnmt\" (UID: \"053537e5-cdbd-476d-a1d5-463b3c95548e\") " pod="opendatahub/maas-api-key-cleanup-29720970-bbnmt" Jul 05 13:30:00.426357 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:30:00.426287 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sv2ql\" (UniqueName: \"kubernetes.io/projected/053537e5-cdbd-476d-a1d5-463b3c95548e-kube-api-access-sv2ql\") pod \"maas-api-key-cleanup-29720970-bbnmt\" (UID: \"053537e5-cdbd-476d-a1d5-463b3c95548e\") " pod="opendatahub/maas-api-key-cleanup-29720970-bbnmt" Jul 05 13:30:00.510277 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:30:00.510238 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29720970-bbnmt" Jul 05 13:30:00.644112 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:30:00.644088 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-29720970-bbnmt"] Jul 05 13:30:00.645927 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:30:00.645894 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod053537e5_cdbd_476d_a1d5_463b3c95548e.slice/crio-794b6938df6887ef391b966e760d4863ebdf67e21036202d0395e7a9e9beac23 WatchSource:0}: Error finding container 794b6938df6887ef391b966e760d4863ebdf67e21036202d0395e7a9e9beac23: Status 404 returned error can't find the container with id 794b6938df6887ef391b966e760d4863ebdf67e21036202d0395e7a9e9beac23 Jul 05 13:30:01.045387 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:30:01.045352 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29720970-bbnmt" event={"ID":"053537e5-cdbd-476d-a1d5-463b3c95548e","Type":"ContainerStarted","Data":"794b6938df6887ef391b966e760d4863ebdf67e21036202d0395e7a9e9beac23"} Jul 05 13:30:04.063694 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:30:04.063605 2578 generic.go:356] "Generic (PLEG): container finished" podID="053537e5-cdbd-476d-a1d5-463b3c95548e" containerID="6023fd5988468d78140249fe76d25ec03efe97fc16676a566b8c82c9244baa70" exitCode=0 Jul 05 13:30:04.063694 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:30:04.063649 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29720970-bbnmt" event={"ID":"053537e5-cdbd-476d-a1d5-463b3c95548e","Type":"ContainerDied","Data":"6023fd5988468d78140249fe76d25ec03efe97fc16676a566b8c82c9244baa70"} Jul 05 13:30:05.207017 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:30:05.206990 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29720970-bbnmt" Jul 05 13:30:05.259431 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:30:05.259403 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sv2ql\" (UniqueName: \"kubernetes.io/projected/053537e5-cdbd-476d-a1d5-463b3c95548e-kube-api-access-sv2ql\") pod \"053537e5-cdbd-476d-a1d5-463b3c95548e\" (UID: \"053537e5-cdbd-476d-a1d5-463b3c95548e\") " Jul 05 13:30:05.261603 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:30:05.261566 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/053537e5-cdbd-476d-a1d5-463b3c95548e-kube-api-access-sv2ql" (OuterVolumeSpecName: "kube-api-access-sv2ql") pod "053537e5-cdbd-476d-a1d5-463b3c95548e" (UID: "053537e5-cdbd-476d-a1d5-463b3c95548e"). InnerVolumeSpecName "kube-api-access-sv2ql". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 05 13:30:05.360666 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:30:05.360553 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sv2ql\" (UniqueName: \"kubernetes.io/projected/053537e5-cdbd-476d-a1d5-463b3c95548e-kube-api-access-sv2ql\") on node \"ip-10-0-130-191.ec2.internal\" DevicePath \"\"" Jul 05 13:30:06.075521 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:30:06.075491 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29720970-bbnmt" Jul 05 13:30:06.075521 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:30:06.075494 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29720970-bbnmt" event={"ID":"053537e5-cdbd-476d-a1d5-463b3c95548e","Type":"ContainerDied","Data":"794b6938df6887ef391b966e760d4863ebdf67e21036202d0395e7a9e9beac23"} Jul 05 13:30:06.075758 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:30:06.075548 2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="794b6938df6887ef391b966e760d4863ebdf67e21036202d0395e7a9e9beac23" Jul 05 13:30:15.980818 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:30:15.980741 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-tg2xx_1c214417-9bef-4007-ab01-e61b84d1b35e/console-operator/1.log" Jul 05 13:30:15.981589 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:30:15.981564 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-tg2xx_1c214417-9bef-4007-ab01-e61b84d1b35e/console-operator/1.log" Jul 05 13:30:15.988497 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:30:15.988461 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-656m6_7bdbd204-0fb7-4ebf-af05-432c370a814b/ovn-acl-logging/0.log" Jul 05 13:30:15.994921 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:30:15.989960 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-656m6_7bdbd204-0fb7-4ebf-af05-432c370a814b/ovn-acl-logging/0.log" Jul 05 13:30:26.909191 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:30:26.909159 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-7c96d999f9-hdwjz"] Jul 05 13:30:26.911802 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:30:26.909517 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="053537e5-cdbd-476d-a1d5-463b3c95548e" containerName="cleanup" Jul 05 13:30:26.911802 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:30:26.909528 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="053537e5-cdbd-476d-a1d5-463b3c95548e" containerName="cleanup" Jul 05 13:30:26.911802 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:30:26.909604 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="053537e5-cdbd-476d-a1d5-463b3c95548e" containerName="cleanup" Jul 05 13:30:26.914416 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:30:26.914386 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7c96d999f9-hdwjz" Jul 05 13:30:26.921673 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:30:26.921643 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7c96d999f9-hdwjz"] Jul 05 13:30:26.950838 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:30:26.950801 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cx97n\" (UniqueName: \"kubernetes.io/projected/51f8dd4a-c2ae-423b-93e3-a915c42ce31a-kube-api-access-cx97n\") pod \"authorino-7c96d999f9-hdwjz\" (UID: \"51f8dd4a-c2ae-423b-93e3-a915c42ce31a\") " pod="kuadrant-system/authorino-7c96d999f9-hdwjz" Jul 05 13:30:26.951001 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:30:26.950877 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/51f8dd4a-c2ae-423b-93e3-a915c42ce31a-tls-cert\") pod \"authorino-7c96d999f9-hdwjz\" (UID: \"51f8dd4a-c2ae-423b-93e3-a915c42ce31a\") " pod="kuadrant-system/authorino-7c96d999f9-hdwjz" Jul 05 13:30:27.052270 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:30:27.052236 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cx97n\" (UniqueName: \"kubernetes.io/projected/51f8dd4a-c2ae-423b-93e3-a915c42ce31a-kube-api-access-cx97n\") pod \"authorino-7c96d999f9-hdwjz\" (UID: \"51f8dd4a-c2ae-423b-93e3-a915c42ce31a\") " pod="kuadrant-system/authorino-7c96d999f9-hdwjz" Jul 05 13:30:27.052480 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:30:27.052323 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/51f8dd4a-c2ae-423b-93e3-a915c42ce31a-tls-cert\") pod \"authorino-7c96d999f9-hdwjz\" (UID: \"51f8dd4a-c2ae-423b-93e3-a915c42ce31a\") " pod="kuadrant-system/authorino-7c96d999f9-hdwjz" Jul 05 13:30:27.054951 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:30:27.054929 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/51f8dd4a-c2ae-423b-93e3-a915c42ce31a-tls-cert\") pod \"authorino-7c96d999f9-hdwjz\" (UID: \"51f8dd4a-c2ae-423b-93e3-a915c42ce31a\") " pod="kuadrant-system/authorino-7c96d999f9-hdwjz" Jul 05 13:30:27.061000 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:30:27.060977 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cx97n\" (UniqueName: \"kubernetes.io/projected/51f8dd4a-c2ae-423b-93e3-a915c42ce31a-kube-api-access-cx97n\") pod \"authorino-7c96d999f9-hdwjz\" (UID: \"51f8dd4a-c2ae-423b-93e3-a915c42ce31a\") " pod="kuadrant-system/authorino-7c96d999f9-hdwjz" Jul 05 13:30:27.226751 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:30:27.226667 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7c96d999f9-hdwjz" Jul 05 13:30:27.387643 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:30:27.387614 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7c96d999f9-hdwjz"] Jul 05 13:30:27.388966 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:30:27.388941 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod51f8dd4a_c2ae_423b_93e3_a915c42ce31a.slice/crio-7d57c030a1c428a921cd7665d14415cc26fad3e90706c607b8ba7da20bfd9849 WatchSource:0}: Error finding container 7d57c030a1c428a921cd7665d14415cc26fad3e90706c607b8ba7da20bfd9849: Status 404 returned error can't find the container with id 7d57c030a1c428a921cd7665d14415cc26fad3e90706c607b8ba7da20bfd9849 Jul 05 13:30:27.390408 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:30:27.390386 2578 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 05 13:30:28.207626 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:30:28.207499 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7c96d999f9-hdwjz" event={"ID":"51f8dd4a-c2ae-423b-93e3-a915c42ce31a","Type":"ContainerStarted","Data":"5b9caa9c5748a089a2d81ca7bdf5bf2af13a563691d1319ee08c68ca3f0fab10"} Jul 05 13:30:28.207626 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:30:28.207556 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7c96d999f9-hdwjz" event={"ID":"51f8dd4a-c2ae-423b-93e3-a915c42ce31a","Type":"ContainerStarted","Data":"7d57c030a1c428a921cd7665d14415cc26fad3e90706c607b8ba7da20bfd9849"} Jul 05 13:30:28.226458 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:30:28.226388 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-7c96d999f9-hdwjz" podStartSLOduration=1.838994124 podStartE2EDuration="2.226368481s" totalImagesPullingTime="387.374357ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 13:30:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 13:30:28.225250597 +0000 UTC m=+912.683813703" watchObservedRunningTime="2026-07-05 13:30:28.226368481 +0000 UTC m=+912.684931588" Jul 05 13:30:28.255882 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:30:28.255841 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-59c5956854-rg6gr"] Jul 05 13:30:28.256086 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:30:28.256059 2578 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-59c5956854-rg6gr" podUID="cd256815-f3cf-4ffe-a132-9a1d473db8c9" containerName="authorino" containerID="cri-o://e3799010864463b12984840f3c76e113460e129c8d15a64b510ca4c7b4755f86" gracePeriod=30 Jul 05 13:30:28.519139 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:30:28.519112 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-59c5956854-rg6gr" Jul 05 13:30:28.575970 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:30:28.567859 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/cd256815-f3cf-4ffe-a132-9a1d473db8c9-tls-cert\") pod \"cd256815-f3cf-4ffe-a132-9a1d473db8c9\" (UID: \"cd256815-f3cf-4ffe-a132-9a1d473db8c9\") " Jul 05 13:30:28.575970 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:30:28.568007 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cpnh9\" (UniqueName: \"kubernetes.io/projected/cd256815-f3cf-4ffe-a132-9a1d473db8c9-kube-api-access-cpnh9\") pod \"cd256815-f3cf-4ffe-a132-9a1d473db8c9\" (UID: \"cd256815-f3cf-4ffe-a132-9a1d473db8c9\") " Jul 05 13:30:28.576769 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:30:28.576731 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cd256815-f3cf-4ffe-a132-9a1d473db8c9-kube-api-access-cpnh9" (OuterVolumeSpecName: "kube-api-access-cpnh9") pod "cd256815-f3cf-4ffe-a132-9a1d473db8c9" (UID: "cd256815-f3cf-4ffe-a132-9a1d473db8c9"). InnerVolumeSpecName "kube-api-access-cpnh9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 05 13:30:28.590464 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:30:28.590433 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cd256815-f3cf-4ffe-a132-9a1d473db8c9-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "cd256815-f3cf-4ffe-a132-9a1d473db8c9" (UID: "cd256815-f3cf-4ffe-a132-9a1d473db8c9"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 05 13:30:28.669669 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:30:28.669631 2578 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/cd256815-f3cf-4ffe-a132-9a1d473db8c9-tls-cert\") on node \"ip-10-0-130-191.ec2.internal\" DevicePath \"\"" Jul 05 13:30:28.669669 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:30:28.669664 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cpnh9\" (UniqueName: \"kubernetes.io/projected/cd256815-f3cf-4ffe-a132-9a1d473db8c9-kube-api-access-cpnh9\") on node \"ip-10-0-130-191.ec2.internal\" DevicePath \"\"" Jul 05 13:30:29.214775 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:30:29.214739 2578 generic.go:356] "Generic (PLEG): container finished" podID="cd256815-f3cf-4ffe-a132-9a1d473db8c9" containerID="e3799010864463b12984840f3c76e113460e129c8d15a64b510ca4c7b4755f86" exitCode=0 Jul 05 13:30:29.215213 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:30:29.214795 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-59c5956854-rg6gr" Jul 05 13:30:29.215213 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:30:29.214826 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-59c5956854-rg6gr" event={"ID":"cd256815-f3cf-4ffe-a132-9a1d473db8c9","Type":"ContainerDied","Data":"e3799010864463b12984840f3c76e113460e129c8d15a64b510ca4c7b4755f86"} Jul 05 13:30:29.215213 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:30:29.214869 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-59c5956854-rg6gr" event={"ID":"cd256815-f3cf-4ffe-a132-9a1d473db8c9","Type":"ContainerDied","Data":"192ceb5de2a78c9d9ef9b261d307f24648deb58df618a4bf4a9b26fb399fb752"} Jul 05 13:30:29.215213 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:30:29.214889 2578 scope.go:117] "RemoveContainer" containerID="e3799010864463b12984840f3c76e113460e129c8d15a64b510ca4c7b4755f86" Jul 05 13:30:29.224071 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:30:29.224046 2578 scope.go:117] "RemoveContainer" containerID="e3799010864463b12984840f3c76e113460e129c8d15a64b510ca4c7b4755f86" Jul 05 13:30:29.224395 ip-10-0-130-191 kubenswrapper[2578]: E0705 13:30:29.224374 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e3799010864463b12984840f3c76e113460e129c8d15a64b510ca4c7b4755f86\": container with ID starting with e3799010864463b12984840f3c76e113460e129c8d15a64b510ca4c7b4755f86 not found: ID does not exist" containerID="e3799010864463b12984840f3c76e113460e129c8d15a64b510ca4c7b4755f86" Jul 05 13:30:29.224478 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:30:29.224406 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e3799010864463b12984840f3c76e113460e129c8d15a64b510ca4c7b4755f86"} err="failed to get container status \"e3799010864463b12984840f3c76e113460e129c8d15a64b510ca4c7b4755f86\": rpc error: code = NotFound desc = could not find container \"e3799010864463b12984840f3c76e113460e129c8d15a64b510ca4c7b4755f86\": container with ID starting with e3799010864463b12984840f3c76e113460e129c8d15a64b510ca4c7b4755f86 not found: ID does not exist" Jul 05 13:30:29.240635 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:30:29.240601 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-59c5956854-rg6gr"] Jul 05 13:30:29.247430 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:30:29.247404 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-59c5956854-rg6gr"] Jul 05 13:30:30.009905 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:30:30.009875 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cd256815-f3cf-4ffe-a132-9a1d473db8c9" path="/var/lib/kubelet/pods/cd256815-f3cf-4ffe-a132-9a1d473db8c9/volumes" Jul 05 13:35:16.028572 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:35:16.028545 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-tg2xx_1c214417-9bef-4007-ab01-e61b84d1b35e/console-operator/1.log" Jul 05 13:35:16.034256 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:35:16.034228 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-tg2xx_1c214417-9bef-4007-ab01-e61b84d1b35e/console-operator/1.log" Jul 05 13:35:16.039139 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:35:16.039119 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-656m6_7bdbd204-0fb7-4ebf-af05-432c370a814b/ovn-acl-logging/0.log" Jul 05 13:35:16.041488 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:35:16.041468 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-656m6_7bdbd204-0fb7-4ebf-af05-432c370a814b/ovn-acl-logging/0.log" Jul 05 13:40:16.070597 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:40:16.070551 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-tg2xx_1c214417-9bef-4007-ab01-e61b84d1b35e/console-operator/1.log" Jul 05 13:40:16.074384 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:40:16.074362 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-tg2xx_1c214417-9bef-4007-ab01-e61b84d1b35e/console-operator/1.log" Jul 05 13:40:16.079209 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:40:16.079187 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-656m6_7bdbd204-0fb7-4ebf-af05-432c370a814b/ovn-acl-logging/0.log" Jul 05 13:40:16.083141 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:40:16.083117 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-656m6_7bdbd204-0fb7-4ebf-af05-432c370a814b/ovn-acl-logging/0.log" Jul 05 13:43:47.728361 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:43:47.728327 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-ait-82ec44a3-cff9f5cdc-w6g8m"] Jul 05 13:43:47.728884 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:43:47.728733 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cd256815-f3cf-4ffe-a132-9a1d473db8c9" containerName="authorino" Jul 05 13:43:47.728884 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:43:47.728745 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="cd256815-f3cf-4ffe-a132-9a1d473db8c9" containerName="authorino" Jul 05 13:43:47.728884 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:43:47.728815 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="cd256815-f3cf-4ffe-a132-9a1d473db8c9" containerName="authorino" Jul 05 13:43:47.731690 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:43:47.731672 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-82ec44a3-cff9f5cdc-w6g8m" Jul 05 13:43:47.735294 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:43:47.735271 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-ait-82ec44a3-serving-cert\"" Jul 05 13:43:47.746934 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:43:47.746911 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-ait-82ec44a3-cff9f5cdc-w6g8m"] Jul 05 13:43:47.843841 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:43:47.843800 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d0ebad49-145c-4d6e-9613-216c8f3b3e59-openshift-service-ca\") pod \"maas-api-e2e-ait-82ec44a3-cff9f5cdc-w6g8m\" (UID: \"d0ebad49-145c-4d6e-9613-216c8f3b3e59\") " pod="opendatahub/maas-api-e2e-ait-82ec44a3-cff9f5cdc-w6g8m" Jul 05 13:43:47.844018 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:43:47.843860 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d0ebad49-145c-4d6e-9613-216c8f3b3e59-maas-api-tls\") pod \"maas-api-e2e-ait-82ec44a3-cff9f5cdc-w6g8m\" (UID: \"d0ebad49-145c-4d6e-9613-216c8f3b3e59\") " pod="opendatahub/maas-api-e2e-ait-82ec44a3-cff9f5cdc-w6g8m" Jul 05 13:43:47.844018 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:43:47.843896 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tsxvd\" (UniqueName: \"kubernetes.io/projected/d0ebad49-145c-4d6e-9613-216c8f3b3e59-kube-api-access-tsxvd\") pod \"maas-api-e2e-ait-82ec44a3-cff9f5cdc-w6g8m\" (UID: \"d0ebad49-145c-4d6e-9613-216c8f3b3e59\") " pod="opendatahub/maas-api-e2e-ait-82ec44a3-cff9f5cdc-w6g8m" Jul 05 13:43:47.945186 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:43:47.945154 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tsxvd\" (UniqueName: \"kubernetes.io/projected/d0ebad49-145c-4d6e-9613-216c8f3b3e59-kube-api-access-tsxvd\") pod \"maas-api-e2e-ait-82ec44a3-cff9f5cdc-w6g8m\" (UID: \"d0ebad49-145c-4d6e-9613-216c8f3b3e59\") " pod="opendatahub/maas-api-e2e-ait-82ec44a3-cff9f5cdc-w6g8m" Jul 05 13:43:47.945351 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:43:47.945231 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d0ebad49-145c-4d6e-9613-216c8f3b3e59-openshift-service-ca\") pod \"maas-api-e2e-ait-82ec44a3-cff9f5cdc-w6g8m\" (UID: \"d0ebad49-145c-4d6e-9613-216c8f3b3e59\") " pod="opendatahub/maas-api-e2e-ait-82ec44a3-cff9f5cdc-w6g8m" Jul 05 13:43:47.945351 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:43:47.945272 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d0ebad49-145c-4d6e-9613-216c8f3b3e59-maas-api-tls\") pod \"maas-api-e2e-ait-82ec44a3-cff9f5cdc-w6g8m\" (UID: \"d0ebad49-145c-4d6e-9613-216c8f3b3e59\") " pod="opendatahub/maas-api-e2e-ait-82ec44a3-cff9f5cdc-w6g8m" Jul 05 13:43:47.945429 ip-10-0-130-191 kubenswrapper[2578]: E0705 13:43:47.945387 2578 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-ait-82ec44a3-serving-cert: secret "maas-api-e2e-ait-82ec44a3-serving-cert" not found Jul 05 13:43:47.945462 ip-10-0-130-191 kubenswrapper[2578]: E0705 13:43:47.945456 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d0ebad49-145c-4d6e-9613-216c8f3b3e59-maas-api-tls podName:d0ebad49-145c-4d6e-9613-216c8f3b3e59 nodeName:}" failed. No retries permitted until 2026-07-05 13:43:48.445436228 +0000 UTC m=+1712.903999313 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/d0ebad49-145c-4d6e-9613-216c8f3b3e59-maas-api-tls") pod "maas-api-e2e-ait-82ec44a3-cff9f5cdc-w6g8m" (UID: "d0ebad49-145c-4d6e-9613-216c8f3b3e59") : secret "maas-api-e2e-ait-82ec44a3-serving-cert" not found Jul 05 13:43:47.945968 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:43:47.945950 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d0ebad49-145c-4d6e-9613-216c8f3b3e59-openshift-service-ca\") pod \"maas-api-e2e-ait-82ec44a3-cff9f5cdc-w6g8m\" (UID: \"d0ebad49-145c-4d6e-9613-216c8f3b3e59\") " pod="opendatahub/maas-api-e2e-ait-82ec44a3-cff9f5cdc-w6g8m" Jul 05 13:43:47.955502 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:43:47.955477 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tsxvd\" (UniqueName: \"kubernetes.io/projected/d0ebad49-145c-4d6e-9613-216c8f3b3e59-kube-api-access-tsxvd\") pod \"maas-api-e2e-ait-82ec44a3-cff9f5cdc-w6g8m\" (UID: \"d0ebad49-145c-4d6e-9613-216c8f3b3e59\") " pod="opendatahub/maas-api-e2e-ait-82ec44a3-cff9f5cdc-w6g8m" Jul 05 13:43:48.449598 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:43:48.449541 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d0ebad49-145c-4d6e-9613-216c8f3b3e59-maas-api-tls\") pod \"maas-api-e2e-ait-82ec44a3-cff9f5cdc-w6g8m\" (UID: \"d0ebad49-145c-4d6e-9613-216c8f3b3e59\") " pod="opendatahub/maas-api-e2e-ait-82ec44a3-cff9f5cdc-w6g8m" Jul 05 13:43:48.452227 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:43:48.452204 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d0ebad49-145c-4d6e-9613-216c8f3b3e59-maas-api-tls\") pod \"maas-api-e2e-ait-82ec44a3-cff9f5cdc-w6g8m\" (UID: \"d0ebad49-145c-4d6e-9613-216c8f3b3e59\") " pod="opendatahub/maas-api-e2e-ait-82ec44a3-cff9f5cdc-w6g8m" Jul 05 13:43:48.642418 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:43:48.642384 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-82ec44a3-cff9f5cdc-w6g8m" Jul 05 13:43:48.782109 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:43:48.782085 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-ait-82ec44a3-cff9f5cdc-w6g8m"] Jul 05 13:43:48.784794 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:43:48.784765 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd0ebad49_145c_4d6e_9613_216c8f3b3e59.slice/crio-08fc8622d6af564434ee6f7ab8c69f74836d5f0a07d3c8a9f9954fc94c5a94fa WatchSource:0}: Error finding container 08fc8622d6af564434ee6f7ab8c69f74836d5f0a07d3c8a9f9954fc94c5a94fa: Status 404 returned error can't find the container with id 08fc8622d6af564434ee6f7ab8c69f74836d5f0a07d3c8a9f9954fc94c5a94fa Jul 05 13:43:48.786445 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:43:48.786429 2578 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 05 13:43:49.631551 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:43:49.631463 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-82ec44a3-cff9f5cdc-w6g8m" event={"ID":"d0ebad49-145c-4d6e-9613-216c8f3b3e59","Type":"ContainerStarted","Data":"98084850d8f096f123d193dd5336fe6c9359f4633e86a59dd5ebc5e9a8a98ca3"} Jul 05 13:43:49.631551 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:43:49.631520 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-82ec44a3-cff9f5cdc-w6g8m" event={"ID":"d0ebad49-145c-4d6e-9613-216c8f3b3e59","Type":"ContainerStarted","Data":"08fc8622d6af564434ee6f7ab8c69f74836d5f0a07d3c8a9f9954fc94c5a94fa"} Jul 05 13:43:49.631551 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:43:49.631551 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-ait-82ec44a3-cff9f5cdc-w6g8m" Jul 05 13:43:49.653053 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:43:49.653004 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-ait-82ec44a3-cff9f5cdc-w6g8m" podStartSLOduration=2.121833616 podStartE2EDuration="2.652989107s" totalImagesPullingTime="531.155491ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 13:43:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 13:43:49.650768383 +0000 UTC m=+1714.109331478" watchObservedRunningTime="2026-07-05 13:43:49.652989107 +0000 UTC m=+1714.111552226" Jul 05 13:43:53.123572 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:43:53.123540 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-ait-82ec44a3-cff9f5cdc-w6g8m"] Jul 05 13:43:53.124056 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:43:53.123755 2578 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-ait-82ec44a3-cff9f5cdc-w6g8m" podUID="d0ebad49-145c-4d6e-9613-216c8f3b3e59" containerName="maas-api" containerID="cri-o://98084850d8f096f123d193dd5336fe6c9359f4633e86a59dd5ebc5e9a8a98ca3" gracePeriod=30 Jul 05 13:43:53.373135 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:43:53.373110 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-82ec44a3-cff9f5cdc-w6g8m" Jul 05 13:43:53.391237 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:43:53.391175 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d0ebad49-145c-4d6e-9613-216c8f3b3e59-maas-api-tls\") pod \"d0ebad49-145c-4d6e-9613-216c8f3b3e59\" (UID: \"d0ebad49-145c-4d6e-9613-216c8f3b3e59\") " Jul 05 13:43:53.391237 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:43:53.391229 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d0ebad49-145c-4d6e-9613-216c8f3b3e59-openshift-service-ca\") pod \"d0ebad49-145c-4d6e-9613-216c8f3b3e59\" (UID: \"d0ebad49-145c-4d6e-9613-216c8f3b3e59\") " Jul 05 13:43:53.391414 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:43:53.391342 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tsxvd\" (UniqueName: \"kubernetes.io/projected/d0ebad49-145c-4d6e-9613-216c8f3b3e59-kube-api-access-tsxvd\") pod \"d0ebad49-145c-4d6e-9613-216c8f3b3e59\" (UID: \"d0ebad49-145c-4d6e-9613-216c8f3b3e59\") " Jul 05 13:43:53.391531 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:43:53.391514 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d0ebad49-145c-4d6e-9613-216c8f3b3e59-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "d0ebad49-145c-4d6e-9613-216c8f3b3e59" (UID: "d0ebad49-145c-4d6e-9613-216c8f3b3e59"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 05 13:43:53.391683 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:43:53.391667 2578 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d0ebad49-145c-4d6e-9613-216c8f3b3e59-openshift-service-ca\") on node \"ip-10-0-130-191.ec2.internal\" DevicePath \"\"" Jul 05 13:43:53.393444 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:43:53.393413 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d0ebad49-145c-4d6e-9613-216c8f3b3e59-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "d0ebad49-145c-4d6e-9613-216c8f3b3e59" (UID: "d0ebad49-145c-4d6e-9613-216c8f3b3e59"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 05 13:43:53.393532 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:43:53.393456 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d0ebad49-145c-4d6e-9613-216c8f3b3e59-kube-api-access-tsxvd" (OuterVolumeSpecName: "kube-api-access-tsxvd") pod "d0ebad49-145c-4d6e-9613-216c8f3b3e59" (UID: "d0ebad49-145c-4d6e-9613-216c8f3b3e59"). InnerVolumeSpecName "kube-api-access-tsxvd". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 05 13:43:53.493006 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:43:53.492967 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tsxvd\" (UniqueName: \"kubernetes.io/projected/d0ebad49-145c-4d6e-9613-216c8f3b3e59-kube-api-access-tsxvd\") on node \"ip-10-0-130-191.ec2.internal\" DevicePath \"\"" Jul 05 13:43:53.493006 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:43:53.493001 2578 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d0ebad49-145c-4d6e-9613-216c8f3b3e59-maas-api-tls\") on node \"ip-10-0-130-191.ec2.internal\" DevicePath \"\"" Jul 05 13:43:53.654892 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:43:53.654800 2578 generic.go:356] "Generic (PLEG): container finished" podID="d0ebad49-145c-4d6e-9613-216c8f3b3e59" containerID="98084850d8f096f123d193dd5336fe6c9359f4633e86a59dd5ebc5e9a8a98ca3" exitCode=0 Jul 05 13:43:53.654892 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:43:53.654847 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-82ec44a3-cff9f5cdc-w6g8m" Jul 05 13:43:53.655096 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:43:53.654893 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-82ec44a3-cff9f5cdc-w6g8m" event={"ID":"d0ebad49-145c-4d6e-9613-216c8f3b3e59","Type":"ContainerDied","Data":"98084850d8f096f123d193dd5336fe6c9359f4633e86a59dd5ebc5e9a8a98ca3"} Jul 05 13:43:53.655096 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:43:53.654951 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-82ec44a3-cff9f5cdc-w6g8m" event={"ID":"d0ebad49-145c-4d6e-9613-216c8f3b3e59","Type":"ContainerDied","Data":"08fc8622d6af564434ee6f7ab8c69f74836d5f0a07d3c8a9f9954fc94c5a94fa"} Jul 05 13:43:53.655096 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:43:53.654978 2578 scope.go:117] "RemoveContainer" containerID="98084850d8f096f123d193dd5336fe6c9359f4633e86a59dd5ebc5e9a8a98ca3" Jul 05 13:43:53.664649 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:43:53.664625 2578 scope.go:117] "RemoveContainer" containerID="98084850d8f096f123d193dd5336fe6c9359f4633e86a59dd5ebc5e9a8a98ca3" Jul 05 13:43:53.664922 ip-10-0-130-191 kubenswrapper[2578]: E0705 13:43:53.664901 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"98084850d8f096f123d193dd5336fe6c9359f4633e86a59dd5ebc5e9a8a98ca3\": container with ID starting with 98084850d8f096f123d193dd5336fe6c9359f4633e86a59dd5ebc5e9a8a98ca3 not found: ID does not exist" containerID="98084850d8f096f123d193dd5336fe6c9359f4633e86a59dd5ebc5e9a8a98ca3" Jul 05 13:43:53.664981 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:43:53.664931 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"98084850d8f096f123d193dd5336fe6c9359f4633e86a59dd5ebc5e9a8a98ca3"} err="failed to get container status \"98084850d8f096f123d193dd5336fe6c9359f4633e86a59dd5ebc5e9a8a98ca3\": rpc error: code = NotFound desc = could not find container \"98084850d8f096f123d193dd5336fe6c9359f4633e86a59dd5ebc5e9a8a98ca3\": container with ID starting with 98084850d8f096f123d193dd5336fe6c9359f4633e86a59dd5ebc5e9a8a98ca3 not found: ID does not exist" Jul 05 13:43:53.678553 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:43:53.678530 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-ait-82ec44a3-cff9f5cdc-w6g8m"] Jul 05 13:43:53.686213 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:43:53.686190 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-ait-82ec44a3-cff9f5cdc-w6g8m"] Jul 05 13:43:54.009552 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:43:54.009518 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d0ebad49-145c-4d6e-9613-216c8f3b3e59" path="/var/lib/kubelet/pods/d0ebad49-145c-4d6e-9613-216c8f3b3e59/volumes" Jul 05 13:44:00.534701 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:44:00.534665 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-ait-986e3717-767944947-clg4x"] Jul 05 13:44:00.535219 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:44:00.535201 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d0ebad49-145c-4d6e-9613-216c8f3b3e59" containerName="maas-api" Jul 05 13:44:00.535304 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:44:00.535223 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="d0ebad49-145c-4d6e-9613-216c8f3b3e59" containerName="maas-api" Jul 05 13:44:00.535362 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:44:00.535346 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="d0ebad49-145c-4d6e-9613-216c8f3b3e59" containerName="maas-api" Jul 05 13:44:00.539943 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:44:00.539918 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-986e3717-767944947-clg4x" Jul 05 13:44:00.545083 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:44:00.545059 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-ait-986e3717-serving-cert\"" Jul 05 13:44:00.549691 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:44:00.549661 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-ait-986e3717-767944947-clg4x"] Jul 05 13:44:00.659787 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:44:00.659731 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nr2b6\" (UniqueName: \"kubernetes.io/projected/75120880-b004-4b10-810d-9aad03697654-kube-api-access-nr2b6\") pod \"maas-api-e2e-ait-986e3717-767944947-clg4x\" (UID: \"75120880-b004-4b10-810d-9aad03697654\") " pod="opendatahub/maas-api-e2e-ait-986e3717-767944947-clg4x" Jul 05 13:44:00.659787 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:44:00.659792 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/75120880-b004-4b10-810d-9aad03697654-maas-api-tls\") pod \"maas-api-e2e-ait-986e3717-767944947-clg4x\" (UID: \"75120880-b004-4b10-810d-9aad03697654\") " pod="opendatahub/maas-api-e2e-ait-986e3717-767944947-clg4x" Jul 05 13:44:00.660012 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:44:00.659876 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/75120880-b004-4b10-810d-9aad03697654-openshift-service-ca\") pod \"maas-api-e2e-ait-986e3717-767944947-clg4x\" (UID: \"75120880-b004-4b10-810d-9aad03697654\") " pod="opendatahub/maas-api-e2e-ait-986e3717-767944947-clg4x" Jul 05 13:44:00.760547 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:44:00.760503 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/75120880-b004-4b10-810d-9aad03697654-openshift-service-ca\") pod \"maas-api-e2e-ait-986e3717-767944947-clg4x\" (UID: \"75120880-b004-4b10-810d-9aad03697654\") " pod="opendatahub/maas-api-e2e-ait-986e3717-767944947-clg4x" Jul 05 13:44:00.760769 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:44:00.760635 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nr2b6\" (UniqueName: \"kubernetes.io/projected/75120880-b004-4b10-810d-9aad03697654-kube-api-access-nr2b6\") pod \"maas-api-e2e-ait-986e3717-767944947-clg4x\" (UID: \"75120880-b004-4b10-810d-9aad03697654\") " pod="opendatahub/maas-api-e2e-ait-986e3717-767944947-clg4x" Jul 05 13:44:00.760769 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:44:00.760664 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/75120880-b004-4b10-810d-9aad03697654-maas-api-tls\") pod \"maas-api-e2e-ait-986e3717-767944947-clg4x\" (UID: \"75120880-b004-4b10-810d-9aad03697654\") " pod="opendatahub/maas-api-e2e-ait-986e3717-767944947-clg4x" Jul 05 13:44:00.761255 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:44:00.761231 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/75120880-b004-4b10-810d-9aad03697654-openshift-service-ca\") pod \"maas-api-e2e-ait-986e3717-767944947-clg4x\" (UID: \"75120880-b004-4b10-810d-9aad03697654\") " pod="opendatahub/maas-api-e2e-ait-986e3717-767944947-clg4x" Jul 05 13:44:00.763127 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:44:00.763107 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/75120880-b004-4b10-810d-9aad03697654-maas-api-tls\") pod \"maas-api-e2e-ait-986e3717-767944947-clg4x\" (UID: \"75120880-b004-4b10-810d-9aad03697654\") " pod="opendatahub/maas-api-e2e-ait-986e3717-767944947-clg4x" Jul 05 13:44:00.768960 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:44:00.768938 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nr2b6\" (UniqueName: \"kubernetes.io/projected/75120880-b004-4b10-810d-9aad03697654-kube-api-access-nr2b6\") pod \"maas-api-e2e-ait-986e3717-767944947-clg4x\" (UID: \"75120880-b004-4b10-810d-9aad03697654\") " pod="opendatahub/maas-api-e2e-ait-986e3717-767944947-clg4x" Jul 05 13:44:00.851753 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:44:00.851659 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-986e3717-767944947-clg4x" Jul 05 13:44:00.997306 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:44:00.997273 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-ait-986e3717-767944947-clg4x"] Jul 05 13:44:00.999444 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:44:00.999416 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod75120880_b004_4b10_810d_9aad03697654.slice/crio-23b44cf7f05acada24c325a980c580ab52913cd8563d611337cc3c94cd342355 WatchSource:0}: Error finding container 23b44cf7f05acada24c325a980c580ab52913cd8563d611337cc3c94cd342355: Status 404 returned error can't find the container with id 23b44cf7f05acada24c325a980c580ab52913cd8563d611337cc3c94cd342355 Jul 05 13:44:01.701259 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:44:01.701228 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-986e3717-767944947-clg4x" event={"ID":"75120880-b004-4b10-810d-9aad03697654","Type":"ContainerStarted","Data":"57ed734f98ae3f47421a57ba2102434198f90f6210af05707515c6a6f2072613"} Jul 05 13:44:01.701691 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:44:01.701276 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-986e3717-767944947-clg4x" event={"ID":"75120880-b004-4b10-810d-9aad03697654","Type":"ContainerStarted","Data":"23b44cf7f05acada24c325a980c580ab52913cd8563d611337cc3c94cd342355"} Jul 05 13:44:01.701691 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:44:01.701318 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-ait-986e3717-767944947-clg4x" Jul 05 13:44:01.721059 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:44:01.721009 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-ait-986e3717-767944947-clg4x" podStartSLOduration=1.23875479 podStartE2EDuration="1.720992782s" totalImagesPullingTime="482.237992ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 13:44:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 13:44:01.718628383 +0000 UTC m=+1726.177191489" watchObservedRunningTime="2026-07-05 13:44:01.720992782 +0000 UTC m=+1726.179555886" Jul 05 13:44:06.020833 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:44:06.020792 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-ait-986e3717-767944947-clg4x"] Jul 05 13:44:06.021311 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:44:06.021011 2578 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-ait-986e3717-767944947-clg4x" podUID="75120880-b004-4b10-810d-9aad03697654" containerName="maas-api" containerID="cri-o://57ed734f98ae3f47421a57ba2102434198f90f6210af05707515c6a6f2072613" gracePeriod=30 Jul 05 13:44:06.027192 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:44:06.027164 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-ait-986e3717-767944947-clg4x" Jul 05 13:44:06.275081 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:44:06.275013 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-986e3717-767944947-clg4x" Jul 05 13:44:06.310700 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:44:06.310662 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/75120880-b004-4b10-810d-9aad03697654-openshift-service-ca\") pod \"75120880-b004-4b10-810d-9aad03697654\" (UID: \"75120880-b004-4b10-810d-9aad03697654\") " Jul 05 13:44:06.310909 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:44:06.310730 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/75120880-b004-4b10-810d-9aad03697654-maas-api-tls\") pod \"75120880-b004-4b10-810d-9aad03697654\" (UID: \"75120880-b004-4b10-810d-9aad03697654\") " Jul 05 13:44:06.310909 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:44:06.310798 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nr2b6\" (UniqueName: \"kubernetes.io/projected/75120880-b004-4b10-810d-9aad03697654-kube-api-access-nr2b6\") pod \"75120880-b004-4b10-810d-9aad03697654\" (UID: \"75120880-b004-4b10-810d-9aad03697654\") " Jul 05 13:44:06.311373 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:44:06.311008 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/75120880-b004-4b10-810d-9aad03697654-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "75120880-b004-4b10-810d-9aad03697654" (UID: "75120880-b004-4b10-810d-9aad03697654"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 05 13:44:06.313125 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:44:06.313092 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/75120880-b004-4b10-810d-9aad03697654-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "75120880-b004-4b10-810d-9aad03697654" (UID: "75120880-b004-4b10-810d-9aad03697654"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 05 13:44:06.313284 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:44:06.313258 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/75120880-b004-4b10-810d-9aad03697654-kube-api-access-nr2b6" (OuterVolumeSpecName: "kube-api-access-nr2b6") pod "75120880-b004-4b10-810d-9aad03697654" (UID: "75120880-b004-4b10-810d-9aad03697654"). InnerVolumeSpecName "kube-api-access-nr2b6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 05 13:44:06.412447 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:44:06.412410 2578 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/75120880-b004-4b10-810d-9aad03697654-openshift-service-ca\") on node \"ip-10-0-130-191.ec2.internal\" DevicePath \"\"" Jul 05 13:44:06.412447 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:44:06.412439 2578 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/75120880-b004-4b10-810d-9aad03697654-maas-api-tls\") on node \"ip-10-0-130-191.ec2.internal\" DevicePath \"\"" Jul 05 13:44:06.412447 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:44:06.412449 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nr2b6\" (UniqueName: \"kubernetes.io/projected/75120880-b004-4b10-810d-9aad03697654-kube-api-access-nr2b6\") on node \"ip-10-0-130-191.ec2.internal\" DevicePath \"\"" Jul 05 13:44:06.729469 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:44:06.729374 2578 generic.go:356] "Generic (PLEG): container finished" podID="75120880-b004-4b10-810d-9aad03697654" containerID="57ed734f98ae3f47421a57ba2102434198f90f6210af05707515c6a6f2072613" exitCode=0 Jul 05 13:44:06.729469 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:44:06.729422 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-986e3717-767944947-clg4x" Jul 05 13:44:06.729469 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:44:06.729458 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-986e3717-767944947-clg4x" event={"ID":"75120880-b004-4b10-810d-9aad03697654","Type":"ContainerDied","Data":"57ed734f98ae3f47421a57ba2102434198f90f6210af05707515c6a6f2072613"} Jul 05 13:44:06.729815 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:44:06.729506 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-986e3717-767944947-clg4x" event={"ID":"75120880-b004-4b10-810d-9aad03697654","Type":"ContainerDied","Data":"23b44cf7f05acada24c325a980c580ab52913cd8563d611337cc3c94cd342355"} Jul 05 13:44:06.729815 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:44:06.729529 2578 scope.go:117] "RemoveContainer" containerID="57ed734f98ae3f47421a57ba2102434198f90f6210af05707515c6a6f2072613" Jul 05 13:44:06.743526 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:44:06.743507 2578 scope.go:117] "RemoveContainer" containerID="57ed734f98ae3f47421a57ba2102434198f90f6210af05707515c6a6f2072613" Jul 05 13:44:06.743858 ip-10-0-130-191 kubenswrapper[2578]: E0705 13:44:06.743840 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"57ed734f98ae3f47421a57ba2102434198f90f6210af05707515c6a6f2072613\": container with ID starting with 57ed734f98ae3f47421a57ba2102434198f90f6210af05707515c6a6f2072613 not found: ID does not exist" containerID="57ed734f98ae3f47421a57ba2102434198f90f6210af05707515c6a6f2072613" Jul 05 13:44:06.743920 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:44:06.743867 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"57ed734f98ae3f47421a57ba2102434198f90f6210af05707515c6a6f2072613"} err="failed to get container status \"57ed734f98ae3f47421a57ba2102434198f90f6210af05707515c6a6f2072613\": rpc error: code = NotFound desc = could not find container \"57ed734f98ae3f47421a57ba2102434198f90f6210af05707515c6a6f2072613\": container with ID starting with 57ed734f98ae3f47421a57ba2102434198f90f6210af05707515c6a6f2072613 not found: ID does not exist" Jul 05 13:44:06.753766 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:44:06.753740 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-ait-986e3717-767944947-clg4x"] Jul 05 13:44:06.757266 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:44:06.757243 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-ait-986e3717-767944947-clg4x"] Jul 05 13:44:08.009663 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:44:08.009633 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="75120880-b004-4b10-810d-9aad03697654" path="/var/lib/kubelet/pods/75120880-b004-4b10-810d-9aad03697654/volumes" Jul 05 13:44:19.655422 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:44:19.655386 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-derive-7b43b0ed-84876b4497-9sn42"] Jul 05 13:44:19.656027 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:44:19.656007 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="75120880-b004-4b10-810d-9aad03697654" containerName="maas-api" Jul 05 13:44:19.656075 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:44:19.656033 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="75120880-b004-4b10-810d-9aad03697654" containerName="maas-api" Jul 05 13:44:19.656172 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:44:19.656160 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="75120880-b004-4b10-810d-9aad03697654" containerName="maas-api" Jul 05 13:44:19.660554 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:44:19.660526 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-derive-7b43b0ed-84876b4497-9sn42" Jul 05 13:44:19.665495 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:44:19.665469 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-derive-7b43b0ed-serving-cert\"" Jul 05 13:44:19.673000 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:44:19.672972 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-derive-7b43b0ed-84876b4497-9sn42"] Jul 05 13:44:19.737991 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:44:19.737955 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/506b2aea-700a-4ff1-b1ac-43a533cd70ed-maas-api-tls\") pod \"maas-api-e2e-derive-7b43b0ed-84876b4497-9sn42\" (UID: \"506b2aea-700a-4ff1-b1ac-43a533cd70ed\") " pod="opendatahub/maas-api-e2e-derive-7b43b0ed-84876b4497-9sn42" Jul 05 13:44:19.737991 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:44:19.738004 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kkjfz\" (UniqueName: \"kubernetes.io/projected/506b2aea-700a-4ff1-b1ac-43a533cd70ed-kube-api-access-kkjfz\") pod \"maas-api-e2e-derive-7b43b0ed-84876b4497-9sn42\" (UID: \"506b2aea-700a-4ff1-b1ac-43a533cd70ed\") " pod="opendatahub/maas-api-e2e-derive-7b43b0ed-84876b4497-9sn42" Jul 05 13:44:19.738244 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:44:19.738085 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/506b2aea-700a-4ff1-b1ac-43a533cd70ed-openshift-service-ca\") pod \"maas-api-e2e-derive-7b43b0ed-84876b4497-9sn42\" (UID: \"506b2aea-700a-4ff1-b1ac-43a533cd70ed\") " pod="opendatahub/maas-api-e2e-derive-7b43b0ed-84876b4497-9sn42" Jul 05 13:44:19.839139 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:44:19.839097 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/506b2aea-700a-4ff1-b1ac-43a533cd70ed-maas-api-tls\") pod \"maas-api-e2e-derive-7b43b0ed-84876b4497-9sn42\" (UID: \"506b2aea-700a-4ff1-b1ac-43a533cd70ed\") " pod="opendatahub/maas-api-e2e-derive-7b43b0ed-84876b4497-9sn42" Jul 05 13:44:19.839351 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:44:19.839150 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kkjfz\" (UniqueName: \"kubernetes.io/projected/506b2aea-700a-4ff1-b1ac-43a533cd70ed-kube-api-access-kkjfz\") pod \"maas-api-e2e-derive-7b43b0ed-84876b4497-9sn42\" (UID: \"506b2aea-700a-4ff1-b1ac-43a533cd70ed\") " pod="opendatahub/maas-api-e2e-derive-7b43b0ed-84876b4497-9sn42" Jul 05 13:44:19.839351 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:44:19.839189 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/506b2aea-700a-4ff1-b1ac-43a533cd70ed-openshift-service-ca\") pod \"maas-api-e2e-derive-7b43b0ed-84876b4497-9sn42\" (UID: \"506b2aea-700a-4ff1-b1ac-43a533cd70ed\") " pod="opendatahub/maas-api-e2e-derive-7b43b0ed-84876b4497-9sn42" Jul 05 13:44:19.839913 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:44:19.839893 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/506b2aea-700a-4ff1-b1ac-43a533cd70ed-openshift-service-ca\") pod \"maas-api-e2e-derive-7b43b0ed-84876b4497-9sn42\" (UID: \"506b2aea-700a-4ff1-b1ac-43a533cd70ed\") " pod="opendatahub/maas-api-e2e-derive-7b43b0ed-84876b4497-9sn42" Jul 05 13:44:19.841842 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:44:19.841818 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/506b2aea-700a-4ff1-b1ac-43a533cd70ed-maas-api-tls\") pod \"maas-api-e2e-derive-7b43b0ed-84876b4497-9sn42\" (UID: \"506b2aea-700a-4ff1-b1ac-43a533cd70ed\") " pod="opendatahub/maas-api-e2e-derive-7b43b0ed-84876b4497-9sn42" Jul 05 13:44:19.852383 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:44:19.852347 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kkjfz\" (UniqueName: \"kubernetes.io/projected/506b2aea-700a-4ff1-b1ac-43a533cd70ed-kube-api-access-kkjfz\") pod \"maas-api-e2e-derive-7b43b0ed-84876b4497-9sn42\" (UID: \"506b2aea-700a-4ff1-b1ac-43a533cd70ed\") " pod="opendatahub/maas-api-e2e-derive-7b43b0ed-84876b4497-9sn42" Jul 05 13:44:19.975889 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:44:19.975798 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-derive-7b43b0ed-84876b4497-9sn42" Jul 05 13:44:20.137229 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:44:20.137195 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-derive-7b43b0ed-84876b4497-9sn42"] Jul 05 13:44:20.138961 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:44:20.138932 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod506b2aea_700a_4ff1_b1ac_43a533cd70ed.slice/crio-436ed3fba01a60ab3cd6453b25a5f07fb6fbca998e64d92acdd55e288d237ded WatchSource:0}: Error finding container 436ed3fba01a60ab3cd6453b25a5f07fb6fbca998e64d92acdd55e288d237ded: Status 404 returned error can't find the container with id 436ed3fba01a60ab3cd6453b25a5f07fb6fbca998e64d92acdd55e288d237ded Jul 05 13:44:20.807150 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:44:20.807042 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-derive-7b43b0ed-84876b4497-9sn42" event={"ID":"506b2aea-700a-4ff1-b1ac-43a533cd70ed","Type":"ContainerStarted","Data":"100f4392e001ede8cf8a55c004057d646918d2a947935a22156e6036f8341c2f"} Jul 05 13:44:20.807537 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:44:20.807233 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-derive-7b43b0ed-84876b4497-9sn42" Jul 05 13:44:20.807537 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:44:20.807262 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-derive-7b43b0ed-84876b4497-9sn42" event={"ID":"506b2aea-700a-4ff1-b1ac-43a533cd70ed","Type":"ContainerStarted","Data":"436ed3fba01a60ab3cd6453b25a5f07fb6fbca998e64d92acdd55e288d237ded"} Jul 05 13:44:20.829476 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:44:20.829422 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-derive-7b43b0ed-84876b4497-9sn42" podStartSLOduration=1.42410106 podStartE2EDuration="1.829406948s" totalImagesPullingTime="405.305888ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 13:44:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 13:44:20.826765003 +0000 UTC m=+1745.285328105" watchObservedRunningTime="2026-07-05 13:44:20.829406948 +0000 UTC m=+1745.287970052" Jul 05 13:44:24.558457 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:44:24.558418 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-derive-7b43b0ed-84876b4497-9sn42"] Jul 05 13:44:24.558920 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:44:24.558630 2578 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-derive-7b43b0ed-84876b4497-9sn42" podUID="506b2aea-700a-4ff1-b1ac-43a533cd70ed" containerName="maas-api" containerID="cri-o://100f4392e001ede8cf8a55c004057d646918d2a947935a22156e6036f8341c2f" gracePeriod=30 Jul 05 13:44:24.811731 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:44:24.811664 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-derive-7b43b0ed-84876b4497-9sn42" Jul 05 13:44:24.833935 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:44:24.833903 2578 generic.go:356] "Generic (PLEG): container finished" podID="506b2aea-700a-4ff1-b1ac-43a533cd70ed" containerID="100f4392e001ede8cf8a55c004057d646918d2a947935a22156e6036f8341c2f" exitCode=0 Jul 05 13:44:24.834123 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:44:24.833950 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-derive-7b43b0ed-84876b4497-9sn42" Jul 05 13:44:24.834123 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:44:24.833988 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-derive-7b43b0ed-84876b4497-9sn42" event={"ID":"506b2aea-700a-4ff1-b1ac-43a533cd70ed","Type":"ContainerDied","Data":"100f4392e001ede8cf8a55c004057d646918d2a947935a22156e6036f8341c2f"} Jul 05 13:44:24.834123 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:44:24.834052 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-derive-7b43b0ed-84876b4497-9sn42" event={"ID":"506b2aea-700a-4ff1-b1ac-43a533cd70ed","Type":"ContainerDied","Data":"436ed3fba01a60ab3cd6453b25a5f07fb6fbca998e64d92acdd55e288d237ded"} Jul 05 13:44:24.834123 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:44:24.834081 2578 scope.go:117] "RemoveContainer" containerID="100f4392e001ede8cf8a55c004057d646918d2a947935a22156e6036f8341c2f" Jul 05 13:44:24.844032 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:44:24.844012 2578 scope.go:117] "RemoveContainer" containerID="100f4392e001ede8cf8a55c004057d646918d2a947935a22156e6036f8341c2f" Jul 05 13:44:24.844331 ip-10-0-130-191 kubenswrapper[2578]: E0705 13:44:24.844308 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"100f4392e001ede8cf8a55c004057d646918d2a947935a22156e6036f8341c2f\": container with ID starting with 100f4392e001ede8cf8a55c004057d646918d2a947935a22156e6036f8341c2f not found: ID does not exist" containerID="100f4392e001ede8cf8a55c004057d646918d2a947935a22156e6036f8341c2f" Jul 05 13:44:24.844428 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:44:24.844336 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"100f4392e001ede8cf8a55c004057d646918d2a947935a22156e6036f8341c2f"} err="failed to get container status \"100f4392e001ede8cf8a55c004057d646918d2a947935a22156e6036f8341c2f\": rpc error: code = NotFound desc = could not find container \"100f4392e001ede8cf8a55c004057d646918d2a947935a22156e6036f8341c2f\": container with ID starting with 100f4392e001ede8cf8a55c004057d646918d2a947935a22156e6036f8341c2f not found: ID does not exist" Jul 05 13:44:24.888539 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:44:24.888505 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/506b2aea-700a-4ff1-b1ac-43a533cd70ed-maas-api-tls\") pod \"506b2aea-700a-4ff1-b1ac-43a533cd70ed\" (UID: \"506b2aea-700a-4ff1-b1ac-43a533cd70ed\") " Jul 05 13:44:24.888724 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:44:24.888600 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kkjfz\" (UniqueName: \"kubernetes.io/projected/506b2aea-700a-4ff1-b1ac-43a533cd70ed-kube-api-access-kkjfz\") pod \"506b2aea-700a-4ff1-b1ac-43a533cd70ed\" (UID: \"506b2aea-700a-4ff1-b1ac-43a533cd70ed\") " Jul 05 13:44:24.888724 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:44:24.888652 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/506b2aea-700a-4ff1-b1ac-43a533cd70ed-openshift-service-ca\") pod \"506b2aea-700a-4ff1-b1ac-43a533cd70ed\" (UID: \"506b2aea-700a-4ff1-b1ac-43a533cd70ed\") " Jul 05 13:44:24.889053 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:44:24.889029 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/506b2aea-700a-4ff1-b1ac-43a533cd70ed-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "506b2aea-700a-4ff1-b1ac-43a533cd70ed" (UID: "506b2aea-700a-4ff1-b1ac-43a533cd70ed"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 05 13:44:24.890861 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:44:24.890837 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/506b2aea-700a-4ff1-b1ac-43a533cd70ed-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "506b2aea-700a-4ff1-b1ac-43a533cd70ed" (UID: "506b2aea-700a-4ff1-b1ac-43a533cd70ed"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 05 13:44:24.890957 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:44:24.890866 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/506b2aea-700a-4ff1-b1ac-43a533cd70ed-kube-api-access-kkjfz" (OuterVolumeSpecName: "kube-api-access-kkjfz") pod "506b2aea-700a-4ff1-b1ac-43a533cd70ed" (UID: "506b2aea-700a-4ff1-b1ac-43a533cd70ed"). InnerVolumeSpecName "kube-api-access-kkjfz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 05 13:44:24.990082 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:44:24.990046 2578 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/506b2aea-700a-4ff1-b1ac-43a533cd70ed-maas-api-tls\") on node \"ip-10-0-130-191.ec2.internal\" DevicePath \"\"" Jul 05 13:44:24.990082 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:44:24.990077 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kkjfz\" (UniqueName: \"kubernetes.io/projected/506b2aea-700a-4ff1-b1ac-43a533cd70ed-kube-api-access-kkjfz\") on node \"ip-10-0-130-191.ec2.internal\" DevicePath \"\"" Jul 05 13:44:24.990082 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:44:24.990090 2578 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/506b2aea-700a-4ff1-b1ac-43a533cd70ed-openshift-service-ca\") on node \"ip-10-0-130-191.ec2.internal\" DevicePath \"\"" Jul 05 13:44:25.161745 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:44:25.161711 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-derive-7b43b0ed-84876b4497-9sn42"] Jul 05 13:44:25.164466 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:44:25.164439 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-derive-7b43b0ed-84876b4497-9sn42"] Jul 05 13:44:26.009370 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:44:26.009336 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="506b2aea-700a-4ff1-b1ac-43a533cd70ed" path="/var/lib/kubelet/pods/506b2aea-700a-4ff1-b1ac-43a533cd70ed/volumes" Jul 05 13:45:16.112438 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:45:16.112402 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-tg2xx_1c214417-9bef-4007-ab01-e61b84d1b35e/console-operator/1.log" Jul 05 13:45:16.118053 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:45:16.118028 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-tg2xx_1c214417-9bef-4007-ab01-e61b84d1b35e/console-operator/1.log" Jul 05 13:45:16.121027 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:45:16.121005 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-656m6_7bdbd204-0fb7-4ebf-af05-432c370a814b/ovn-acl-logging/0.log" Jul 05 13:45:16.127416 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:45:16.127396 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-656m6_7bdbd204-0fb7-4ebf-af05-432c370a814b/ovn-acl-logging/0.log" Jul 05 13:46:10.388571 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:46:10.388531 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-mt-5fbdd8c5-698f8bf65-lvzpf"] Jul 05 13:46:10.389138 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:46:10.388994 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="506b2aea-700a-4ff1-b1ac-43a533cd70ed" containerName="maas-api" Jul 05 13:46:10.389138 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:46:10.389011 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="506b2aea-700a-4ff1-b1ac-43a533cd70ed" containerName="maas-api" Jul 05 13:46:10.389138 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:46:10.389075 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="506b2aea-700a-4ff1-b1ac-43a533cd70ed" containerName="maas-api" Jul 05 13:46:10.392295 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:46:10.392268 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-5fbdd8c5-698f8bf65-lvzpf" Jul 05 13:46:10.395061 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:46:10.395026 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-mt-5fbdd8c5-serving-cert\"" Jul 05 13:46:10.407303 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:46:10.407279 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-mt-5fbdd8c5-698f8bf65-lvzpf"] Jul 05 13:46:10.454624 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:46:10.454571 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4jpxp\" (UniqueName: \"kubernetes.io/projected/bd13a4f0-2736-4b07-a3e8-ad4edc1fc74d-kube-api-access-4jpxp\") pod \"maas-api-e2e-mt-5fbdd8c5-698f8bf65-lvzpf\" (UID: \"bd13a4f0-2736-4b07-a3e8-ad4edc1fc74d\") " pod="opendatahub/maas-api-e2e-mt-5fbdd8c5-698f8bf65-lvzpf" Jul 05 13:46:10.454811 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:46:10.454717 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bd13a4f0-2736-4b07-a3e8-ad4edc1fc74d-maas-api-tls\") pod \"maas-api-e2e-mt-5fbdd8c5-698f8bf65-lvzpf\" (UID: \"bd13a4f0-2736-4b07-a3e8-ad4edc1fc74d\") " pod="opendatahub/maas-api-e2e-mt-5fbdd8c5-698f8bf65-lvzpf" Jul 05 13:46:10.454811 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:46:10.454748 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bd13a4f0-2736-4b07-a3e8-ad4edc1fc74d-openshift-service-ca\") pod \"maas-api-e2e-mt-5fbdd8c5-698f8bf65-lvzpf\" (UID: \"bd13a4f0-2736-4b07-a3e8-ad4edc1fc74d\") " pod="opendatahub/maas-api-e2e-mt-5fbdd8c5-698f8bf65-lvzpf" Jul 05 13:46:10.555390 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:46:10.555355 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bd13a4f0-2736-4b07-a3e8-ad4edc1fc74d-maas-api-tls\") pod \"maas-api-e2e-mt-5fbdd8c5-698f8bf65-lvzpf\" (UID: \"bd13a4f0-2736-4b07-a3e8-ad4edc1fc74d\") " pod="opendatahub/maas-api-e2e-mt-5fbdd8c5-698f8bf65-lvzpf" Jul 05 13:46:10.555574 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:46:10.555398 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bd13a4f0-2736-4b07-a3e8-ad4edc1fc74d-openshift-service-ca\") pod \"maas-api-e2e-mt-5fbdd8c5-698f8bf65-lvzpf\" (UID: \"bd13a4f0-2736-4b07-a3e8-ad4edc1fc74d\") " pod="opendatahub/maas-api-e2e-mt-5fbdd8c5-698f8bf65-lvzpf" Jul 05 13:46:10.555574 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:46:10.555434 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4jpxp\" (UniqueName: \"kubernetes.io/projected/bd13a4f0-2736-4b07-a3e8-ad4edc1fc74d-kube-api-access-4jpxp\") pod \"maas-api-e2e-mt-5fbdd8c5-698f8bf65-lvzpf\" (UID: \"bd13a4f0-2736-4b07-a3e8-ad4edc1fc74d\") " pod="opendatahub/maas-api-e2e-mt-5fbdd8c5-698f8bf65-lvzpf" Jul 05 13:46:10.556131 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:46:10.556111 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bd13a4f0-2736-4b07-a3e8-ad4edc1fc74d-openshift-service-ca\") pod \"maas-api-e2e-mt-5fbdd8c5-698f8bf65-lvzpf\" (UID: \"bd13a4f0-2736-4b07-a3e8-ad4edc1fc74d\") " pod="opendatahub/maas-api-e2e-mt-5fbdd8c5-698f8bf65-lvzpf" Jul 05 13:46:10.557993 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:46:10.557973 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bd13a4f0-2736-4b07-a3e8-ad4edc1fc74d-maas-api-tls\") pod \"maas-api-e2e-mt-5fbdd8c5-698f8bf65-lvzpf\" (UID: \"bd13a4f0-2736-4b07-a3e8-ad4edc1fc74d\") " pod="opendatahub/maas-api-e2e-mt-5fbdd8c5-698f8bf65-lvzpf" Jul 05 13:46:10.577330 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:46:10.577307 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4jpxp\" (UniqueName: \"kubernetes.io/projected/bd13a4f0-2736-4b07-a3e8-ad4edc1fc74d-kube-api-access-4jpxp\") pod \"maas-api-e2e-mt-5fbdd8c5-698f8bf65-lvzpf\" (UID: \"bd13a4f0-2736-4b07-a3e8-ad4edc1fc74d\") " pod="opendatahub/maas-api-e2e-mt-5fbdd8c5-698f8bf65-lvzpf" Jul 05 13:46:10.703258 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:46:10.703166 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-5fbdd8c5-698f8bf65-lvzpf" Jul 05 13:46:10.878619 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:46:10.878573 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-mt-5fbdd8c5-698f8bf65-lvzpf"] Jul 05 13:46:10.880288 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:46:10.880261 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbd13a4f0_2736_4b07_a3e8_ad4edc1fc74d.slice/crio-f8ec1500b0bc5bd7679124fcd27bd76dafcb0211944499dd81166f3703e8ea2c WatchSource:0}: Error finding container f8ec1500b0bc5bd7679124fcd27bd76dafcb0211944499dd81166f3703e8ea2c: Status 404 returned error can't find the container with id f8ec1500b0bc5bd7679124fcd27bd76dafcb0211944499dd81166f3703e8ea2c Jul 05 13:46:11.419428 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:46:11.419392 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-5fbdd8c5-698f8bf65-lvzpf" event={"ID":"bd13a4f0-2736-4b07-a3e8-ad4edc1fc74d","Type":"ContainerStarted","Data":"f8ec1500b0bc5bd7679124fcd27bd76dafcb0211944499dd81166f3703e8ea2c"} Jul 05 13:46:12.429901 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:46:12.429866 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-5fbdd8c5-698f8bf65-lvzpf" event={"ID":"bd13a4f0-2736-4b07-a3e8-ad4edc1fc74d","Type":"ContainerStarted","Data":"f1e914e862437abf7cd1876e8cdaf6cbe974b5356710dd93f97c2975f8360736"} Jul 05 13:46:12.430297 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:46:12.429923 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-mt-5fbdd8c5-698f8bf65-lvzpf" Jul 05 13:46:12.461131 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:46:12.461086 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-mt-5fbdd8c5-698f8bf65-lvzpf" podStartSLOduration=1.9548149540000002 podStartE2EDuration="2.461070693s" totalImagesPullingTime="506.255739ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 13:46:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 13:46:12.460918648 +0000 UTC m=+1856.919481754" watchObservedRunningTime="2026-07-05 13:46:12.461070693 +0000 UTC m=+1856.919633799" Jul 05 13:46:18.440698 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:46:18.440669 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-mt-5fbdd8c5-698f8bf65-lvzpf" Jul 05 13:46:22.829719 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:46:22.829681 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-mt-5fbdd8c5-698f8bf65-lvzpf"] Jul 05 13:46:22.830201 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:46:22.829916 2578 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-mt-5fbdd8c5-698f8bf65-lvzpf" podUID="bd13a4f0-2736-4b07-a3e8-ad4edc1fc74d" containerName="maas-api" containerID="cri-o://f1e914e862437abf7cd1876e8cdaf6cbe974b5356710dd93f97c2975f8360736" gracePeriod=30 Jul 05 13:46:23.084113 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:46:23.084057 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-5fbdd8c5-698f8bf65-lvzpf" Jul 05 13:46:23.153606 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:46:23.153565 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4jpxp\" (UniqueName: \"kubernetes.io/projected/bd13a4f0-2736-4b07-a3e8-ad4edc1fc74d-kube-api-access-4jpxp\") pod \"bd13a4f0-2736-4b07-a3e8-ad4edc1fc74d\" (UID: \"bd13a4f0-2736-4b07-a3e8-ad4edc1fc74d\") " Jul 05 13:46:23.153805 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:46:23.153623 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bd13a4f0-2736-4b07-a3e8-ad4edc1fc74d-openshift-service-ca\") pod \"bd13a4f0-2736-4b07-a3e8-ad4edc1fc74d\" (UID: \"bd13a4f0-2736-4b07-a3e8-ad4edc1fc74d\") " Jul 05 13:46:23.153805 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:46:23.153659 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bd13a4f0-2736-4b07-a3e8-ad4edc1fc74d-maas-api-tls\") pod \"bd13a4f0-2736-4b07-a3e8-ad4edc1fc74d\" (UID: \"bd13a4f0-2736-4b07-a3e8-ad4edc1fc74d\") " Jul 05 13:46:23.154018 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:46:23.153993 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bd13a4f0-2736-4b07-a3e8-ad4edc1fc74d-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "bd13a4f0-2736-4b07-a3e8-ad4edc1fc74d" (UID: "bd13a4f0-2736-4b07-a3e8-ad4edc1fc74d"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 05 13:46:23.155787 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:46:23.155758 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bd13a4f0-2736-4b07-a3e8-ad4edc1fc74d-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "bd13a4f0-2736-4b07-a3e8-ad4edc1fc74d" (UID: "bd13a4f0-2736-4b07-a3e8-ad4edc1fc74d"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 05 13:46:23.155906 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:46:23.155884 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bd13a4f0-2736-4b07-a3e8-ad4edc1fc74d-kube-api-access-4jpxp" (OuterVolumeSpecName: "kube-api-access-4jpxp") pod "bd13a4f0-2736-4b07-a3e8-ad4edc1fc74d" (UID: "bd13a4f0-2736-4b07-a3e8-ad4edc1fc74d"). InnerVolumeSpecName "kube-api-access-4jpxp". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 05 13:46:23.254592 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:46:23.254544 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4jpxp\" (UniqueName: \"kubernetes.io/projected/bd13a4f0-2736-4b07-a3e8-ad4edc1fc74d-kube-api-access-4jpxp\") on node \"ip-10-0-130-191.ec2.internal\" DevicePath \"\"" Jul 05 13:46:23.254816 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:46:23.254613 2578 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bd13a4f0-2736-4b07-a3e8-ad4edc1fc74d-openshift-service-ca\") on node \"ip-10-0-130-191.ec2.internal\" DevicePath \"\"" Jul 05 13:46:23.254816 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:46:23.254624 2578 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bd13a4f0-2736-4b07-a3e8-ad4edc1fc74d-maas-api-tls\") on node \"ip-10-0-130-191.ec2.internal\" DevicePath \"\"" Jul 05 13:46:23.494199 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:46:23.494109 2578 generic.go:356] "Generic (PLEG): container finished" podID="bd13a4f0-2736-4b07-a3e8-ad4edc1fc74d" containerID="f1e914e862437abf7cd1876e8cdaf6cbe974b5356710dd93f97c2975f8360736" exitCode=0 Jul 05 13:46:23.494199 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:46:23.494154 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-5fbdd8c5-698f8bf65-lvzpf" Jul 05 13:46:23.494199 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:46:23.494174 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-5fbdd8c5-698f8bf65-lvzpf" event={"ID":"bd13a4f0-2736-4b07-a3e8-ad4edc1fc74d","Type":"ContainerDied","Data":"f1e914e862437abf7cd1876e8cdaf6cbe974b5356710dd93f97c2975f8360736"} Jul 05 13:46:23.494423 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:46:23.494217 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-5fbdd8c5-698f8bf65-lvzpf" event={"ID":"bd13a4f0-2736-4b07-a3e8-ad4edc1fc74d","Type":"ContainerDied","Data":"f8ec1500b0bc5bd7679124fcd27bd76dafcb0211944499dd81166f3703e8ea2c"} Jul 05 13:46:23.494423 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:46:23.494239 2578 scope.go:117] "RemoveContainer" containerID="f1e914e862437abf7cd1876e8cdaf6cbe974b5356710dd93f97c2975f8360736" Jul 05 13:46:23.503920 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:46:23.503903 2578 scope.go:117] "RemoveContainer" containerID="f1e914e862437abf7cd1876e8cdaf6cbe974b5356710dd93f97c2975f8360736" Jul 05 13:46:23.504228 ip-10-0-130-191 kubenswrapper[2578]: E0705 13:46:23.504208 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f1e914e862437abf7cd1876e8cdaf6cbe974b5356710dd93f97c2975f8360736\": container with ID starting with f1e914e862437abf7cd1876e8cdaf6cbe974b5356710dd93f97c2975f8360736 not found: ID does not exist" containerID="f1e914e862437abf7cd1876e8cdaf6cbe974b5356710dd93f97c2975f8360736" Jul 05 13:46:23.504272 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:46:23.504238 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f1e914e862437abf7cd1876e8cdaf6cbe974b5356710dd93f97c2975f8360736"} err="failed to get container status \"f1e914e862437abf7cd1876e8cdaf6cbe974b5356710dd93f97c2975f8360736\": rpc error: code = NotFound desc = could not find container \"f1e914e862437abf7cd1876e8cdaf6cbe974b5356710dd93f97c2975f8360736\": container with ID starting with f1e914e862437abf7cd1876e8cdaf6cbe974b5356710dd93f97c2975f8360736 not found: ID does not exist" Jul 05 13:46:23.520979 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:46:23.520948 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-mt-5fbdd8c5-698f8bf65-lvzpf"] Jul 05 13:46:23.524445 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:46:23.524426 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-mt-5fbdd8c5-698f8bf65-lvzpf"] Jul 05 13:46:24.009838 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:46:24.009809 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bd13a4f0-2736-4b07-a3e8-ad4edc1fc74d" path="/var/lib/kubelet/pods/bd13a4f0-2736-4b07-a3e8-ad4edc1fc74d/volumes" Jul 05 13:46:52.560468 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:46:52.560436 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-mt-6522870d-76fd6bcbb-6452r"] Jul 05 13:46:52.560963 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:46:52.560850 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bd13a4f0-2736-4b07-a3e8-ad4edc1fc74d" containerName="maas-api" Jul 05 13:46:52.560963 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:46:52.560864 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="bd13a4f0-2736-4b07-a3e8-ad4edc1fc74d" containerName="maas-api" Jul 05 13:46:52.560963 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:46:52.560928 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="bd13a4f0-2736-4b07-a3e8-ad4edc1fc74d" containerName="maas-api" Jul 05 13:46:52.564020 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:46:52.564002 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-6522870d-76fd6bcbb-6452r" Jul 05 13:46:52.566910 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:46:52.566887 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-mt-6522870d-serving-cert\"" Jul 05 13:46:52.587245 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:46:52.587213 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-mt-6522870d-76fd6bcbb-6452r"] Jul 05 13:46:52.731733 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:46:52.731684 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3d08f4a6-9396-48ce-a820-2122350e82db-openshift-service-ca\") pod \"maas-api-e2e-mt-6522870d-76fd6bcbb-6452r\" (UID: \"3d08f4a6-9396-48ce-a820-2122350e82db\") " pod="opendatahub/maas-api-e2e-mt-6522870d-76fd6bcbb-6452r" Jul 05 13:46:52.731733 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:46:52.731737 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3d08f4a6-9396-48ce-a820-2122350e82db-maas-api-tls\") pod \"maas-api-e2e-mt-6522870d-76fd6bcbb-6452r\" (UID: \"3d08f4a6-9396-48ce-a820-2122350e82db\") " pod="opendatahub/maas-api-e2e-mt-6522870d-76fd6bcbb-6452r" Jul 05 13:46:52.731964 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:46:52.731756 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n5sfb\" (UniqueName: \"kubernetes.io/projected/3d08f4a6-9396-48ce-a820-2122350e82db-kube-api-access-n5sfb\") pod \"maas-api-e2e-mt-6522870d-76fd6bcbb-6452r\" (UID: \"3d08f4a6-9396-48ce-a820-2122350e82db\") " pod="opendatahub/maas-api-e2e-mt-6522870d-76fd6bcbb-6452r" Jul 05 13:46:52.833096 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:46:52.832987 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3d08f4a6-9396-48ce-a820-2122350e82db-openshift-service-ca\") pod \"maas-api-e2e-mt-6522870d-76fd6bcbb-6452r\" (UID: \"3d08f4a6-9396-48ce-a820-2122350e82db\") " pod="opendatahub/maas-api-e2e-mt-6522870d-76fd6bcbb-6452r" Jul 05 13:46:52.833096 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:46:52.833048 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3d08f4a6-9396-48ce-a820-2122350e82db-maas-api-tls\") pod \"maas-api-e2e-mt-6522870d-76fd6bcbb-6452r\" (UID: \"3d08f4a6-9396-48ce-a820-2122350e82db\") " pod="opendatahub/maas-api-e2e-mt-6522870d-76fd6bcbb-6452r" Jul 05 13:46:52.833096 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:46:52.833071 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n5sfb\" (UniqueName: \"kubernetes.io/projected/3d08f4a6-9396-48ce-a820-2122350e82db-kube-api-access-n5sfb\") pod \"maas-api-e2e-mt-6522870d-76fd6bcbb-6452r\" (UID: \"3d08f4a6-9396-48ce-a820-2122350e82db\") " pod="opendatahub/maas-api-e2e-mt-6522870d-76fd6bcbb-6452r" Jul 05 13:46:52.833397 ip-10-0-130-191 kubenswrapper[2578]: E0705 13:46:52.833200 2578 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-mt-6522870d-serving-cert: secret "maas-api-e2e-mt-6522870d-serving-cert" not found Jul 05 13:46:52.833397 ip-10-0-130-191 kubenswrapper[2578]: E0705 13:46:52.833276 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3d08f4a6-9396-48ce-a820-2122350e82db-maas-api-tls podName:3d08f4a6-9396-48ce-a820-2122350e82db nodeName:}" failed. No retries permitted until 2026-07-05 13:46:53.333256547 +0000 UTC m=+1897.791819647 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/3d08f4a6-9396-48ce-a820-2122350e82db-maas-api-tls") pod "maas-api-e2e-mt-6522870d-76fd6bcbb-6452r" (UID: "3d08f4a6-9396-48ce-a820-2122350e82db") : secret "maas-api-e2e-mt-6522870d-serving-cert" not found Jul 05 13:46:52.833777 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:46:52.833752 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3d08f4a6-9396-48ce-a820-2122350e82db-openshift-service-ca\") pod \"maas-api-e2e-mt-6522870d-76fd6bcbb-6452r\" (UID: \"3d08f4a6-9396-48ce-a820-2122350e82db\") " pod="opendatahub/maas-api-e2e-mt-6522870d-76fd6bcbb-6452r" Jul 05 13:46:52.848054 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:46:52.848021 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n5sfb\" (UniqueName: \"kubernetes.io/projected/3d08f4a6-9396-48ce-a820-2122350e82db-kube-api-access-n5sfb\") pod \"maas-api-e2e-mt-6522870d-76fd6bcbb-6452r\" (UID: \"3d08f4a6-9396-48ce-a820-2122350e82db\") " pod="opendatahub/maas-api-e2e-mt-6522870d-76fd6bcbb-6452r" Jul 05 13:46:53.338370 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:46:53.338330 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3d08f4a6-9396-48ce-a820-2122350e82db-maas-api-tls\") pod \"maas-api-e2e-mt-6522870d-76fd6bcbb-6452r\" (UID: \"3d08f4a6-9396-48ce-a820-2122350e82db\") " pod="opendatahub/maas-api-e2e-mt-6522870d-76fd6bcbb-6452r" Jul 05 13:46:53.340967 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:46:53.340941 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3d08f4a6-9396-48ce-a820-2122350e82db-maas-api-tls\") pod \"maas-api-e2e-mt-6522870d-76fd6bcbb-6452r\" (UID: \"3d08f4a6-9396-48ce-a820-2122350e82db\") " pod="opendatahub/maas-api-e2e-mt-6522870d-76fd6bcbb-6452r" Jul 05 13:46:53.474256 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:46:53.474222 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-6522870d-76fd6bcbb-6452r" Jul 05 13:46:53.626093 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:46:53.626075 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-mt-6522870d-76fd6bcbb-6452r"] Jul 05 13:46:53.629001 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:46:53.628972 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3d08f4a6_9396_48ce_a820_2122350e82db.slice/crio-7dc53b97ee85e8a4c28df44ee515d9d584b8d93f407632c303c17c5e98ac3d61 WatchSource:0}: Error finding container 7dc53b97ee85e8a4c28df44ee515d9d584b8d93f407632c303c17c5e98ac3d61: Status 404 returned error can't find the container with id 7dc53b97ee85e8a4c28df44ee515d9d584b8d93f407632c303c17c5e98ac3d61 Jul 05 13:46:53.658203 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:46:53.658171 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-6522870d-76fd6bcbb-6452r" event={"ID":"3d08f4a6-9396-48ce-a820-2122350e82db","Type":"ContainerStarted","Data":"7dc53b97ee85e8a4c28df44ee515d9d584b8d93f407632c303c17c5e98ac3d61"} Jul 05 13:46:54.665942 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:46:54.665904 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-6522870d-76fd6bcbb-6452r" event={"ID":"3d08f4a6-9396-48ce-a820-2122350e82db","Type":"ContainerStarted","Data":"c1dcd4b70bb9cf75681e9a95f8e1d4e0ed008d8acded96216d5d236686fe599e"} Jul 05 13:46:54.666333 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:46:54.666001 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-mt-6522870d-76fd6bcbb-6452r" Jul 05 13:46:54.685844 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:46:54.685792 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-mt-6522870d-76fd6bcbb-6452r" podStartSLOduration=2.315692866 podStartE2EDuration="2.685773851s" totalImagesPullingTime="370.080985ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 13:46:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 13:46:54.683608882 +0000 UTC m=+1899.142171983" watchObservedRunningTime="2026-07-05 13:46:54.685773851 +0000 UTC m=+1899.144336955" Jul 05 13:47:00.678326 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:47:00.678295 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-mt-6522870d-76fd6bcbb-6452r" Jul 05 13:47:24.919949 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:47:24.919912 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-mt-6522870d-76fd6bcbb-6452r"] Jul 05 13:47:24.920470 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:47:24.920125 2578 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-mt-6522870d-76fd6bcbb-6452r" podUID="3d08f4a6-9396-48ce-a820-2122350e82db" containerName="maas-api" containerID="cri-o://c1dcd4b70bb9cf75681e9a95f8e1d4e0ed008d8acded96216d5d236686fe599e" gracePeriod=30 Jul 05 13:47:25.162853 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:47:25.162826 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-6522870d-76fd6bcbb-6452r" Jul 05 13:47:25.231689 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:47:25.231605 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3d08f4a6-9396-48ce-a820-2122350e82db-maas-api-tls\") pod \"3d08f4a6-9396-48ce-a820-2122350e82db\" (UID: \"3d08f4a6-9396-48ce-a820-2122350e82db\") " Jul 05 13:47:25.231830 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:47:25.231735 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n5sfb\" (UniqueName: \"kubernetes.io/projected/3d08f4a6-9396-48ce-a820-2122350e82db-kube-api-access-n5sfb\") pod \"3d08f4a6-9396-48ce-a820-2122350e82db\" (UID: \"3d08f4a6-9396-48ce-a820-2122350e82db\") " Jul 05 13:47:25.231830 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:47:25.231804 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3d08f4a6-9396-48ce-a820-2122350e82db-openshift-service-ca\") pod \"3d08f4a6-9396-48ce-a820-2122350e82db\" (UID: \"3d08f4a6-9396-48ce-a820-2122350e82db\") " Jul 05 13:47:25.232241 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:47:25.232211 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3d08f4a6-9396-48ce-a820-2122350e82db-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "3d08f4a6-9396-48ce-a820-2122350e82db" (UID: "3d08f4a6-9396-48ce-a820-2122350e82db"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 05 13:47:25.233757 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:47:25.233726 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3d08f4a6-9396-48ce-a820-2122350e82db-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "3d08f4a6-9396-48ce-a820-2122350e82db" (UID: "3d08f4a6-9396-48ce-a820-2122350e82db"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 05 13:47:25.234011 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:47:25.233988 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3d08f4a6-9396-48ce-a820-2122350e82db-kube-api-access-n5sfb" (OuterVolumeSpecName: "kube-api-access-n5sfb") pod "3d08f4a6-9396-48ce-a820-2122350e82db" (UID: "3d08f4a6-9396-48ce-a820-2122350e82db"). InnerVolumeSpecName "kube-api-access-n5sfb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 05 13:47:25.333317 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:47:25.333261 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n5sfb\" (UniqueName: \"kubernetes.io/projected/3d08f4a6-9396-48ce-a820-2122350e82db-kube-api-access-n5sfb\") on node \"ip-10-0-130-191.ec2.internal\" DevicePath \"\"" Jul 05 13:47:25.333317 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:47:25.333309 2578 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3d08f4a6-9396-48ce-a820-2122350e82db-openshift-service-ca\") on node \"ip-10-0-130-191.ec2.internal\" DevicePath \"\"" Jul 05 13:47:25.333317 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:47:25.333319 2578 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3d08f4a6-9396-48ce-a820-2122350e82db-maas-api-tls\") on node \"ip-10-0-130-191.ec2.internal\" DevicePath \"\"" Jul 05 13:47:25.843547 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:47:25.843515 2578 generic.go:356] "Generic (PLEG): container finished" podID="3d08f4a6-9396-48ce-a820-2122350e82db" containerID="c1dcd4b70bb9cf75681e9a95f8e1d4e0ed008d8acded96216d5d236686fe599e" exitCode=0 Jul 05 13:47:25.843775 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:47:25.843598 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-6522870d-76fd6bcbb-6452r" Jul 05 13:47:25.843775 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:47:25.843608 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-6522870d-76fd6bcbb-6452r" event={"ID":"3d08f4a6-9396-48ce-a820-2122350e82db","Type":"ContainerDied","Data":"c1dcd4b70bb9cf75681e9a95f8e1d4e0ed008d8acded96216d5d236686fe599e"} Jul 05 13:47:25.843775 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:47:25.843701 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-6522870d-76fd6bcbb-6452r" event={"ID":"3d08f4a6-9396-48ce-a820-2122350e82db","Type":"ContainerDied","Data":"7dc53b97ee85e8a4c28df44ee515d9d584b8d93f407632c303c17c5e98ac3d61"} Jul 05 13:47:25.843775 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:47:25.843733 2578 scope.go:117] "RemoveContainer" containerID="c1dcd4b70bb9cf75681e9a95f8e1d4e0ed008d8acded96216d5d236686fe599e" Jul 05 13:47:25.854098 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:47:25.854077 2578 scope.go:117] "RemoveContainer" containerID="c1dcd4b70bb9cf75681e9a95f8e1d4e0ed008d8acded96216d5d236686fe599e" Jul 05 13:47:25.854371 ip-10-0-130-191 kubenswrapper[2578]: E0705 13:47:25.854355 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c1dcd4b70bb9cf75681e9a95f8e1d4e0ed008d8acded96216d5d236686fe599e\": container with ID starting with c1dcd4b70bb9cf75681e9a95f8e1d4e0ed008d8acded96216d5d236686fe599e not found: ID does not exist" containerID="c1dcd4b70bb9cf75681e9a95f8e1d4e0ed008d8acded96216d5d236686fe599e" Jul 05 13:47:25.854421 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:47:25.854379 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c1dcd4b70bb9cf75681e9a95f8e1d4e0ed008d8acded96216d5d236686fe599e"} err="failed to get container status \"c1dcd4b70bb9cf75681e9a95f8e1d4e0ed008d8acded96216d5d236686fe599e\": rpc error: code = NotFound desc = could not find container \"c1dcd4b70bb9cf75681e9a95f8e1d4e0ed008d8acded96216d5d236686fe599e\": container with ID starting with c1dcd4b70bb9cf75681e9a95f8e1d4e0ed008d8acded96216d5d236686fe599e not found: ID does not exist" Jul 05 13:47:25.869164 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:47:25.869131 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-mt-6522870d-76fd6bcbb-6452r"] Jul 05 13:47:25.873297 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:47:25.873273 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-mt-6522870d-76fd6bcbb-6452r"] Jul 05 13:47:26.009160 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:47:26.009124 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3d08f4a6-9396-48ce-a820-2122350e82db" path="/var/lib/kubelet/pods/3d08f4a6-9396-48ce-a820-2122350e82db/volumes" Jul 05 13:49:20.223382 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:49:20.223343 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-inf-a-84bdd7-67fd7ff4c9-vw4j6"] Jul 05 13:49:20.223882 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:49:20.223756 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3d08f4a6-9396-48ce-a820-2122350e82db" containerName="maas-api" Jul 05 13:49:20.223882 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:49:20.223770 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="3d08f4a6-9396-48ce-a820-2122350e82db" containerName="maas-api" Jul 05 13:49:20.223882 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:49:20.223830 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="3d08f4a6-9396-48ce-a820-2122350e82db" containerName="maas-api" Jul 05 13:49:20.226790 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:49:20.226772 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-a-84bdd7-67fd7ff4c9-vw4j6" Jul 05 13:49:20.229852 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:49:20.229834 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-inf-a-84bdd7-serving-cert\"" Jul 05 13:49:20.247188 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:49:20.247153 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-inf-a-84bdd7-67fd7ff4c9-vw4j6"] Jul 05 13:49:20.378238 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:49:20.378197 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/81409f58-4e9d-48b8-b2ed-c08304b9c9d8-openshift-service-ca\") pod \"maas-api-e2e-inf-a-84bdd7-67fd7ff4c9-vw4j6\" (UID: \"81409f58-4e9d-48b8-b2ed-c08304b9c9d8\") " pod="opendatahub/maas-api-e2e-inf-a-84bdd7-67fd7ff4c9-vw4j6" Jul 05 13:49:20.378482 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:49:20.378260 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/81409f58-4e9d-48b8-b2ed-c08304b9c9d8-maas-api-tls\") pod \"maas-api-e2e-inf-a-84bdd7-67fd7ff4c9-vw4j6\" (UID: \"81409f58-4e9d-48b8-b2ed-c08304b9c9d8\") " pod="opendatahub/maas-api-e2e-inf-a-84bdd7-67fd7ff4c9-vw4j6" Jul 05 13:49:20.378482 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:49:20.378287 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fx4h5\" (UniqueName: \"kubernetes.io/projected/81409f58-4e9d-48b8-b2ed-c08304b9c9d8-kube-api-access-fx4h5\") pod \"maas-api-e2e-inf-a-84bdd7-67fd7ff4c9-vw4j6\" (UID: \"81409f58-4e9d-48b8-b2ed-c08304b9c9d8\") " pod="opendatahub/maas-api-e2e-inf-a-84bdd7-67fd7ff4c9-vw4j6" Jul 05 13:49:20.478935 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:49:20.478832 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/81409f58-4e9d-48b8-b2ed-c08304b9c9d8-openshift-service-ca\") pod \"maas-api-e2e-inf-a-84bdd7-67fd7ff4c9-vw4j6\" (UID: \"81409f58-4e9d-48b8-b2ed-c08304b9c9d8\") " pod="opendatahub/maas-api-e2e-inf-a-84bdd7-67fd7ff4c9-vw4j6" Jul 05 13:49:20.478935 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:49:20.478875 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/81409f58-4e9d-48b8-b2ed-c08304b9c9d8-maas-api-tls\") pod \"maas-api-e2e-inf-a-84bdd7-67fd7ff4c9-vw4j6\" (UID: \"81409f58-4e9d-48b8-b2ed-c08304b9c9d8\") " pod="opendatahub/maas-api-e2e-inf-a-84bdd7-67fd7ff4c9-vw4j6" Jul 05 13:49:20.478935 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:49:20.478893 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fx4h5\" (UniqueName: \"kubernetes.io/projected/81409f58-4e9d-48b8-b2ed-c08304b9c9d8-kube-api-access-fx4h5\") pod \"maas-api-e2e-inf-a-84bdd7-67fd7ff4c9-vw4j6\" (UID: \"81409f58-4e9d-48b8-b2ed-c08304b9c9d8\") " pod="opendatahub/maas-api-e2e-inf-a-84bdd7-67fd7ff4c9-vw4j6" Jul 05 13:49:20.479237 ip-10-0-130-191 kubenswrapper[2578]: E0705 13:49:20.479000 2578 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-inf-a-84bdd7-serving-cert: secret "maas-api-e2e-inf-a-84bdd7-serving-cert" not found Jul 05 13:49:20.479237 ip-10-0-130-191 kubenswrapper[2578]: E0705 13:49:20.479083 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/81409f58-4e9d-48b8-b2ed-c08304b9c9d8-maas-api-tls podName:81409f58-4e9d-48b8-b2ed-c08304b9c9d8 nodeName:}" failed. No retries permitted until 2026-07-05 13:49:20.979061297 +0000 UTC m=+2045.437624395 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/81409f58-4e9d-48b8-b2ed-c08304b9c9d8-maas-api-tls") pod "maas-api-e2e-inf-a-84bdd7-67fd7ff4c9-vw4j6" (UID: "81409f58-4e9d-48b8-b2ed-c08304b9c9d8") : secret "maas-api-e2e-inf-a-84bdd7-serving-cert" not found Jul 05 13:49:20.479666 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:49:20.479643 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/81409f58-4e9d-48b8-b2ed-c08304b9c9d8-openshift-service-ca\") pod \"maas-api-e2e-inf-a-84bdd7-67fd7ff4c9-vw4j6\" (UID: \"81409f58-4e9d-48b8-b2ed-c08304b9c9d8\") " pod="opendatahub/maas-api-e2e-inf-a-84bdd7-67fd7ff4c9-vw4j6" Jul 05 13:49:20.492223 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:49:20.492197 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fx4h5\" (UniqueName: \"kubernetes.io/projected/81409f58-4e9d-48b8-b2ed-c08304b9c9d8-kube-api-access-fx4h5\") pod \"maas-api-e2e-inf-a-84bdd7-67fd7ff4c9-vw4j6\" (UID: \"81409f58-4e9d-48b8-b2ed-c08304b9c9d8\") " pod="opendatahub/maas-api-e2e-inf-a-84bdd7-67fd7ff4c9-vw4j6" Jul 05 13:49:20.983156 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:49:20.983118 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/81409f58-4e9d-48b8-b2ed-c08304b9c9d8-maas-api-tls\") pod \"maas-api-e2e-inf-a-84bdd7-67fd7ff4c9-vw4j6\" (UID: \"81409f58-4e9d-48b8-b2ed-c08304b9c9d8\") " pod="opendatahub/maas-api-e2e-inf-a-84bdd7-67fd7ff4c9-vw4j6" Jul 05 13:49:20.985871 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:49:20.985847 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/81409f58-4e9d-48b8-b2ed-c08304b9c9d8-maas-api-tls\") pod \"maas-api-e2e-inf-a-84bdd7-67fd7ff4c9-vw4j6\" (UID: \"81409f58-4e9d-48b8-b2ed-c08304b9c9d8\") " pod="opendatahub/maas-api-e2e-inf-a-84bdd7-67fd7ff4c9-vw4j6" Jul 05 13:49:21.137378 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:49:21.137337 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-a-84bdd7-67fd7ff4c9-vw4j6" Jul 05 13:49:21.287133 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:49:21.287099 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-inf-a-84bdd7-67fd7ff4c9-vw4j6"] Jul 05 13:49:21.288438 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:49:21.288407 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod81409f58_4e9d_48b8_b2ed_c08304b9c9d8.slice/crio-5d20c06e00ef069c10ce1984db4de25f4030f7a0cf6b70eedfbda95f84347299 WatchSource:0}: Error finding container 5d20c06e00ef069c10ce1984db4de25f4030f7a0cf6b70eedfbda95f84347299: Status 404 returned error can't find the container with id 5d20c06e00ef069c10ce1984db4de25f4030f7a0cf6b70eedfbda95f84347299 Jul 05 13:49:21.290121 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:49:21.290103 2578 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 05 13:49:21.475047 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:49:21.475006 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-a-84bdd7-67fd7ff4c9-vw4j6" event={"ID":"81409f58-4e9d-48b8-b2ed-c08304b9c9d8","Type":"ContainerStarted","Data":"5d20c06e00ef069c10ce1984db4de25f4030f7a0cf6b70eedfbda95f84347299"} Jul 05 13:49:22.482105 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:49:22.482064 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-a-84bdd7-67fd7ff4c9-vw4j6" event={"ID":"81409f58-4e9d-48b8-b2ed-c08304b9c9d8","Type":"ContainerStarted","Data":"0780fd99fa543551f3740a811ec318f26bc9f458d37518be994a2428e4148dd4"} Jul 05 13:49:22.482500 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:49:22.482144 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-inf-a-84bdd7-67fd7ff4c9-vw4j6" Jul 05 13:49:22.502430 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:49:22.502371 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-inf-a-84bdd7-67fd7ff4c9-vw4j6" podStartSLOduration=1.941914315 podStartE2EDuration="2.50235463s" totalImagesPullingTime="560.440315ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 13:49:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 13:49:22.501207866 +0000 UTC m=+2046.959770972" watchObservedRunningTime="2026-07-05 13:49:22.50235463 +0000 UTC m=+2046.960917734" Jul 05 13:49:28.310688 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:49:28.310642 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-inf-b-9521b3-c6bdccdff-nkkfw"] Jul 05 13:49:28.314351 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:49:28.314325 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-b-9521b3-c6bdccdff-nkkfw" Jul 05 13:49:28.317112 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:49:28.317088 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-inf-b-9521b3-serving-cert\"" Jul 05 13:49:28.336748 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:49:28.336717 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-inf-b-9521b3-c6bdccdff-nkkfw"] Jul 05 13:49:28.449424 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:49:28.449379 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8ad7e96b-80c1-4814-913b-e879550d8156-openshift-service-ca\") pod \"maas-api-e2e-inf-b-9521b3-c6bdccdff-nkkfw\" (UID: \"8ad7e96b-80c1-4814-913b-e879550d8156\") " pod="opendatahub/maas-api-e2e-inf-b-9521b3-c6bdccdff-nkkfw" Jul 05 13:49:28.449424 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:49:28.449434 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8ad7e96b-80c1-4814-913b-e879550d8156-maas-api-tls\") pod \"maas-api-e2e-inf-b-9521b3-c6bdccdff-nkkfw\" (UID: \"8ad7e96b-80c1-4814-913b-e879550d8156\") " pod="opendatahub/maas-api-e2e-inf-b-9521b3-c6bdccdff-nkkfw" Jul 05 13:49:28.449696 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:49:28.449464 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n4twf\" (UniqueName: \"kubernetes.io/projected/8ad7e96b-80c1-4814-913b-e879550d8156-kube-api-access-n4twf\") pod \"maas-api-e2e-inf-b-9521b3-c6bdccdff-nkkfw\" (UID: \"8ad7e96b-80c1-4814-913b-e879550d8156\") " pod="opendatahub/maas-api-e2e-inf-b-9521b3-c6bdccdff-nkkfw" Jul 05 13:49:28.493552 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:49:28.493521 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-inf-a-84bdd7-67fd7ff4c9-vw4j6" Jul 05 13:49:28.550642 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:49:28.550607 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8ad7e96b-80c1-4814-913b-e879550d8156-openshift-service-ca\") pod \"maas-api-e2e-inf-b-9521b3-c6bdccdff-nkkfw\" (UID: \"8ad7e96b-80c1-4814-913b-e879550d8156\") " pod="opendatahub/maas-api-e2e-inf-b-9521b3-c6bdccdff-nkkfw" Jul 05 13:49:28.550822 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:49:28.550677 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8ad7e96b-80c1-4814-913b-e879550d8156-maas-api-tls\") pod \"maas-api-e2e-inf-b-9521b3-c6bdccdff-nkkfw\" (UID: \"8ad7e96b-80c1-4814-913b-e879550d8156\") " pod="opendatahub/maas-api-e2e-inf-b-9521b3-c6bdccdff-nkkfw" Jul 05 13:49:28.550822 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:49:28.550718 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n4twf\" (UniqueName: \"kubernetes.io/projected/8ad7e96b-80c1-4814-913b-e879550d8156-kube-api-access-n4twf\") pod \"maas-api-e2e-inf-b-9521b3-c6bdccdff-nkkfw\" (UID: \"8ad7e96b-80c1-4814-913b-e879550d8156\") " pod="opendatahub/maas-api-e2e-inf-b-9521b3-c6bdccdff-nkkfw" Jul 05 13:49:28.550944 ip-10-0-130-191 kubenswrapper[2578]: E0705 13:49:28.550853 2578 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-inf-b-9521b3-serving-cert: secret "maas-api-e2e-inf-b-9521b3-serving-cert" not found Jul 05 13:49:28.550944 ip-10-0-130-191 kubenswrapper[2578]: E0705 13:49:28.550936 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8ad7e96b-80c1-4814-913b-e879550d8156-maas-api-tls podName:8ad7e96b-80c1-4814-913b-e879550d8156 nodeName:}" failed. No retries permitted until 2026-07-05 13:49:29.050914069 +0000 UTC m=+2053.509477177 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/8ad7e96b-80c1-4814-913b-e879550d8156-maas-api-tls") pod "maas-api-e2e-inf-b-9521b3-c6bdccdff-nkkfw" (UID: "8ad7e96b-80c1-4814-913b-e879550d8156") : secret "maas-api-e2e-inf-b-9521b3-serving-cert" not found Jul 05 13:49:28.551290 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:49:28.551272 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8ad7e96b-80c1-4814-913b-e879550d8156-openshift-service-ca\") pod \"maas-api-e2e-inf-b-9521b3-c6bdccdff-nkkfw\" (UID: \"8ad7e96b-80c1-4814-913b-e879550d8156\") " pod="opendatahub/maas-api-e2e-inf-b-9521b3-c6bdccdff-nkkfw" Jul 05 13:49:28.574439 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:49:28.574361 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n4twf\" (UniqueName: \"kubernetes.io/projected/8ad7e96b-80c1-4814-913b-e879550d8156-kube-api-access-n4twf\") pod \"maas-api-e2e-inf-b-9521b3-c6bdccdff-nkkfw\" (UID: \"8ad7e96b-80c1-4814-913b-e879550d8156\") " pod="opendatahub/maas-api-e2e-inf-b-9521b3-c6bdccdff-nkkfw" Jul 05 13:49:29.055451 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:49:29.055414 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8ad7e96b-80c1-4814-913b-e879550d8156-maas-api-tls\") pod \"maas-api-e2e-inf-b-9521b3-c6bdccdff-nkkfw\" (UID: \"8ad7e96b-80c1-4814-913b-e879550d8156\") " pod="opendatahub/maas-api-e2e-inf-b-9521b3-c6bdccdff-nkkfw" Jul 05 13:49:29.057976 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:49:29.057947 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8ad7e96b-80c1-4814-913b-e879550d8156-maas-api-tls\") pod \"maas-api-e2e-inf-b-9521b3-c6bdccdff-nkkfw\" (UID: \"8ad7e96b-80c1-4814-913b-e879550d8156\") " pod="opendatahub/maas-api-e2e-inf-b-9521b3-c6bdccdff-nkkfw" Jul 05 13:49:29.225034 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:49:29.224995 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-b-9521b3-c6bdccdff-nkkfw" Jul 05 13:49:29.382902 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:49:29.382874 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-inf-b-9521b3-c6bdccdff-nkkfw"] Jul 05 13:49:29.384629 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:49:29.384603 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8ad7e96b_80c1_4814_913b_e879550d8156.slice/crio-84ca94869962df3a410581c692c0517a12c1cf75f968263a2b531e44ccfb7ab2 WatchSource:0}: Error finding container 84ca94869962df3a410581c692c0517a12c1cf75f968263a2b531e44ccfb7ab2: Status 404 returned error can't find the container with id 84ca94869962df3a410581c692c0517a12c1cf75f968263a2b531e44ccfb7ab2 Jul 05 13:49:29.528999 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:49:29.528968 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-b-9521b3-c6bdccdff-nkkfw" event={"ID":"8ad7e96b-80c1-4814-913b-e879550d8156","Type":"ContainerStarted","Data":"84ca94869962df3a410581c692c0517a12c1cf75f968263a2b531e44ccfb7ab2"} Jul 05 13:49:30.536975 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:49:30.536931 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-b-9521b3-c6bdccdff-nkkfw" event={"ID":"8ad7e96b-80c1-4814-913b-e879550d8156","Type":"ContainerStarted","Data":"8e3728174300e9d9e3a2dc27df2849765ef0251da751a3c11ef906a9b6784e63"} Jul 05 13:49:30.537389 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:49:30.537136 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-inf-b-9521b3-c6bdccdff-nkkfw" Jul 05 13:49:30.558535 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:49:30.558476 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-inf-b-9521b3-c6bdccdff-nkkfw" podStartSLOduration=2.252785281 podStartE2EDuration="2.558458857s" totalImagesPullingTime="305.673576ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 13:49:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 13:49:30.555991574 +0000 UTC m=+2055.014554678" watchObservedRunningTime="2026-07-05 13:49:30.558458857 +0000 UTC m=+2055.017021976" Jul 05 13:49:36.548511 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:49:36.548479 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-inf-b-9521b3-c6bdccdff-nkkfw" Jul 05 13:50:16.151884 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:50:16.151855 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-tg2xx_1c214417-9bef-4007-ab01-e61b84d1b35e/console-operator/1.log" Jul 05 13:50:16.159096 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:50:16.159071 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-656m6_7bdbd204-0fb7-4ebf-af05-432c370a814b/ovn-acl-logging/0.log" Jul 05 13:50:16.161242 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:50:16.161221 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-tg2xx_1c214417-9bef-4007-ab01-e61b84d1b35e/console-operator/1.log" Jul 05 13:50:16.168038 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:50:16.168020 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-656m6_7bdbd204-0fb7-4ebf-af05-432c370a814b/ovn-acl-logging/0.log" Jul 05 13:50:38.366845 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:50:38.366809 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-inf-a-84bdd7-67fd7ff4c9-vw4j6"] Jul 05 13:50:38.367377 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:50:38.367025 2578 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-inf-a-84bdd7-67fd7ff4c9-vw4j6" podUID="81409f58-4e9d-48b8-b2ed-c08304b9c9d8" containerName="maas-api" containerID="cri-o://0780fd99fa543551f3740a811ec318f26bc9f458d37518be994a2428e4148dd4" gracePeriod=30 Jul 05 13:50:38.488204 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:50:38.488158 2578 prober.go:120] "Probe failed" probeType="Readiness" pod="opendatahub/maas-api-e2e-inf-a-84bdd7-67fd7ff4c9-vw4j6" podUID="81409f58-4e9d-48b8-b2ed-c08304b9c9d8" containerName="maas-api" probeResult="failure" output="Get \"https://10.132.0.58:8443/health\": dial tcp 10.132.0.58:8443: connect: connection refused" Jul 05 13:50:38.623269 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:50:38.623189 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-a-84bdd7-67fd7ff4c9-vw4j6" Jul 05 13:50:38.658889 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:50:38.658857 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/81409f58-4e9d-48b8-b2ed-c08304b9c9d8-openshift-service-ca\") pod \"81409f58-4e9d-48b8-b2ed-c08304b9c9d8\" (UID: \"81409f58-4e9d-48b8-b2ed-c08304b9c9d8\") " Jul 05 13:50:38.659084 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:50:38.658916 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/81409f58-4e9d-48b8-b2ed-c08304b9c9d8-maas-api-tls\") pod \"81409f58-4e9d-48b8-b2ed-c08304b9c9d8\" (UID: \"81409f58-4e9d-48b8-b2ed-c08304b9c9d8\") " Jul 05 13:50:38.659084 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:50:38.658968 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fx4h5\" (UniqueName: \"kubernetes.io/projected/81409f58-4e9d-48b8-b2ed-c08304b9c9d8-kube-api-access-fx4h5\") pod \"81409f58-4e9d-48b8-b2ed-c08304b9c9d8\" (UID: \"81409f58-4e9d-48b8-b2ed-c08304b9c9d8\") " Jul 05 13:50:38.659239 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:50:38.659217 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/81409f58-4e9d-48b8-b2ed-c08304b9c9d8-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "81409f58-4e9d-48b8-b2ed-c08304b9c9d8" (UID: "81409f58-4e9d-48b8-b2ed-c08304b9c9d8"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 05 13:50:38.661110 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:50:38.661081 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/81409f58-4e9d-48b8-b2ed-c08304b9c9d8-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "81409f58-4e9d-48b8-b2ed-c08304b9c9d8" (UID: "81409f58-4e9d-48b8-b2ed-c08304b9c9d8"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 05 13:50:38.661223 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:50:38.661145 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/81409f58-4e9d-48b8-b2ed-c08304b9c9d8-kube-api-access-fx4h5" (OuterVolumeSpecName: "kube-api-access-fx4h5") pod "81409f58-4e9d-48b8-b2ed-c08304b9c9d8" (UID: "81409f58-4e9d-48b8-b2ed-c08304b9c9d8"). InnerVolumeSpecName "kube-api-access-fx4h5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 05 13:50:38.760251 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:50:38.760207 2578 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/81409f58-4e9d-48b8-b2ed-c08304b9c9d8-openshift-service-ca\") on node \"ip-10-0-130-191.ec2.internal\" DevicePath \"\"" Jul 05 13:50:38.760251 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:50:38.760241 2578 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/81409f58-4e9d-48b8-b2ed-c08304b9c9d8-maas-api-tls\") on node \"ip-10-0-130-191.ec2.internal\" DevicePath \"\"" Jul 05 13:50:38.760251 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:50:38.760252 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fx4h5\" (UniqueName: \"kubernetes.io/projected/81409f58-4e9d-48b8-b2ed-c08304b9c9d8-kube-api-access-fx4h5\") on node \"ip-10-0-130-191.ec2.internal\" DevicePath \"\"" Jul 05 13:50:38.918803 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:50:38.918703 2578 generic.go:356] "Generic (PLEG): container finished" podID="81409f58-4e9d-48b8-b2ed-c08304b9c9d8" containerID="0780fd99fa543551f3740a811ec318f26bc9f458d37518be994a2428e4148dd4" exitCode=0 Jul 05 13:50:38.918803 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:50:38.918769 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-a-84bdd7-67fd7ff4c9-vw4j6" Jul 05 13:50:38.918803 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:50:38.918788 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-a-84bdd7-67fd7ff4c9-vw4j6" event={"ID":"81409f58-4e9d-48b8-b2ed-c08304b9c9d8","Type":"ContainerDied","Data":"0780fd99fa543551f3740a811ec318f26bc9f458d37518be994a2428e4148dd4"} Jul 05 13:50:38.919093 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:50:38.918842 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-a-84bdd7-67fd7ff4c9-vw4j6" event={"ID":"81409f58-4e9d-48b8-b2ed-c08304b9c9d8","Type":"ContainerDied","Data":"5d20c06e00ef069c10ce1984db4de25f4030f7a0cf6b70eedfbda95f84347299"} Jul 05 13:50:38.919093 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:50:38.918863 2578 scope.go:117] "RemoveContainer" containerID="0780fd99fa543551f3740a811ec318f26bc9f458d37518be994a2428e4148dd4" Jul 05 13:50:38.929156 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:50:38.929136 2578 scope.go:117] "RemoveContainer" containerID="0780fd99fa543551f3740a811ec318f26bc9f458d37518be994a2428e4148dd4" Jul 05 13:50:38.929482 ip-10-0-130-191 kubenswrapper[2578]: E0705 13:50:38.929461 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0780fd99fa543551f3740a811ec318f26bc9f458d37518be994a2428e4148dd4\": container with ID starting with 0780fd99fa543551f3740a811ec318f26bc9f458d37518be994a2428e4148dd4 not found: ID does not exist" containerID="0780fd99fa543551f3740a811ec318f26bc9f458d37518be994a2428e4148dd4" Jul 05 13:50:38.929534 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:50:38.929492 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0780fd99fa543551f3740a811ec318f26bc9f458d37518be994a2428e4148dd4"} err="failed to get container status \"0780fd99fa543551f3740a811ec318f26bc9f458d37518be994a2428e4148dd4\": rpc error: code = NotFound desc = could not find container \"0780fd99fa543551f3740a811ec318f26bc9f458d37518be994a2428e4148dd4\": container with ID starting with 0780fd99fa543551f3740a811ec318f26bc9f458d37518be994a2428e4148dd4 not found: ID does not exist" Jul 05 13:50:38.943833 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:50:38.943805 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-inf-a-84bdd7-67fd7ff4c9-vw4j6"] Jul 05 13:50:38.947489 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:50:38.947455 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-inf-a-84bdd7-67fd7ff4c9-vw4j6"] Jul 05 13:50:40.008839 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:50:40.008801 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="81409f58-4e9d-48b8-b2ed-c08304b9c9d8" path="/var/lib/kubelet/pods/81409f58-4e9d-48b8-b2ed-c08304b9c9d8/volumes" Jul 05 13:51:21.843453 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:51:21.843413 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-inf-b-9521b3-c6bdccdff-nkkfw"] Jul 05 13:51:21.844034 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:51:21.843726 2578 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-inf-b-9521b3-c6bdccdff-nkkfw" podUID="8ad7e96b-80c1-4814-913b-e879550d8156" containerName="maas-api" containerID="cri-o://8e3728174300e9d9e3a2dc27df2849765ef0251da751a3c11ef906a9b6784e63" gracePeriod=30 Jul 05 13:51:22.085945 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:51:22.085916 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-b-9521b3-c6bdccdff-nkkfw" Jul 05 13:51:22.145663 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:51:22.145552 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8ad7e96b-80c1-4814-913b-e879550d8156-openshift-service-ca\") pod \"8ad7e96b-80c1-4814-913b-e879550d8156\" (UID: \"8ad7e96b-80c1-4814-913b-e879550d8156\") " Jul 05 13:51:22.145663 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:51:22.145610 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n4twf\" (UniqueName: \"kubernetes.io/projected/8ad7e96b-80c1-4814-913b-e879550d8156-kube-api-access-n4twf\") pod \"8ad7e96b-80c1-4814-913b-e879550d8156\" (UID: \"8ad7e96b-80c1-4814-913b-e879550d8156\") " Jul 05 13:51:22.145896 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:51:22.145708 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8ad7e96b-80c1-4814-913b-e879550d8156-maas-api-tls\") pod \"8ad7e96b-80c1-4814-913b-e879550d8156\" (UID: \"8ad7e96b-80c1-4814-913b-e879550d8156\") " Jul 05 13:51:22.146000 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:51:22.145978 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8ad7e96b-80c1-4814-913b-e879550d8156-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "8ad7e96b-80c1-4814-913b-e879550d8156" (UID: "8ad7e96b-80c1-4814-913b-e879550d8156"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 05 13:51:22.147817 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:51:22.147789 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8ad7e96b-80c1-4814-913b-e879550d8156-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "8ad7e96b-80c1-4814-913b-e879550d8156" (UID: "8ad7e96b-80c1-4814-913b-e879550d8156"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 05 13:51:22.147891 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:51:22.147842 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8ad7e96b-80c1-4814-913b-e879550d8156-kube-api-access-n4twf" (OuterVolumeSpecName: "kube-api-access-n4twf") pod "8ad7e96b-80c1-4814-913b-e879550d8156" (UID: "8ad7e96b-80c1-4814-913b-e879550d8156"). InnerVolumeSpecName "kube-api-access-n4twf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 05 13:51:22.160160 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:51:22.160133 2578 generic.go:356] "Generic (PLEG): container finished" podID="8ad7e96b-80c1-4814-913b-e879550d8156" containerID="8e3728174300e9d9e3a2dc27df2849765ef0251da751a3c11ef906a9b6784e63" exitCode=0 Jul 05 13:51:22.160269 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:51:22.160175 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-b-9521b3-c6bdccdff-nkkfw" Jul 05 13:51:22.160269 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:51:22.160205 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-b-9521b3-c6bdccdff-nkkfw" event={"ID":"8ad7e96b-80c1-4814-913b-e879550d8156","Type":"ContainerDied","Data":"8e3728174300e9d9e3a2dc27df2849765ef0251da751a3c11ef906a9b6784e63"} Jul 05 13:51:22.160269 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:51:22.160257 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-b-9521b3-c6bdccdff-nkkfw" event={"ID":"8ad7e96b-80c1-4814-913b-e879550d8156","Type":"ContainerDied","Data":"84ca94869962df3a410581c692c0517a12c1cf75f968263a2b531e44ccfb7ab2"} Jul 05 13:51:22.160397 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:51:22.160279 2578 scope.go:117] "RemoveContainer" containerID="8e3728174300e9d9e3a2dc27df2849765ef0251da751a3c11ef906a9b6784e63" Jul 05 13:51:22.171752 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:51:22.171164 2578 scope.go:117] "RemoveContainer" containerID="8e3728174300e9d9e3a2dc27df2849765ef0251da751a3c11ef906a9b6784e63" Jul 05 13:51:22.171752 ip-10-0-130-191 kubenswrapper[2578]: E0705 13:51:22.171557 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8e3728174300e9d9e3a2dc27df2849765ef0251da751a3c11ef906a9b6784e63\": container with ID starting with 8e3728174300e9d9e3a2dc27df2849765ef0251da751a3c11ef906a9b6784e63 not found: ID does not exist" containerID="8e3728174300e9d9e3a2dc27df2849765ef0251da751a3c11ef906a9b6784e63" Jul 05 13:51:22.171752 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:51:22.171614 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8e3728174300e9d9e3a2dc27df2849765ef0251da751a3c11ef906a9b6784e63"} err="failed to get container status \"8e3728174300e9d9e3a2dc27df2849765ef0251da751a3c11ef906a9b6784e63\": rpc error: code = NotFound desc = could not find container \"8e3728174300e9d9e3a2dc27df2849765ef0251da751a3c11ef906a9b6784e63\": container with ID starting with 8e3728174300e9d9e3a2dc27df2849765ef0251da751a3c11ef906a9b6784e63 not found: ID does not exist" Jul 05 13:51:22.196843 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:51:22.196802 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-inf-b-9521b3-c6bdccdff-nkkfw"] Jul 05 13:51:22.205138 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:51:22.205109 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-inf-b-9521b3-c6bdccdff-nkkfw"] Jul 05 13:51:22.247238 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:51:22.247212 2578 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8ad7e96b-80c1-4814-913b-e879550d8156-maas-api-tls\") on node \"ip-10-0-130-191.ec2.internal\" DevicePath \"\"" Jul 05 13:51:22.247238 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:51:22.247237 2578 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8ad7e96b-80c1-4814-913b-e879550d8156-openshift-service-ca\") on node \"ip-10-0-130-191.ec2.internal\" DevicePath \"\"" Jul 05 13:51:22.247412 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:51:22.247247 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n4twf\" (UniqueName: \"kubernetes.io/projected/8ad7e96b-80c1-4814-913b-e879550d8156-kube-api-access-n4twf\") on node \"ip-10-0-130-191.ec2.internal\" DevicePath \"\"" Jul 05 13:51:24.009982 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:51:24.009948 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8ad7e96b-80c1-4814-913b-e879550d8156" path="/var/lib/kubelet/pods/8ad7e96b-80c1-4814-913b-e879550d8156/volumes" Jul 05 13:51:32.684074 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:51:32.684038 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-api-a-e55a60-ffb74dc47-r5m87"] Jul 05 13:51:32.684462 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:51:32.684390 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="81409f58-4e9d-48b8-b2ed-c08304b9c9d8" containerName="maas-api" Jul 05 13:51:32.684462 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:51:32.684401 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="81409f58-4e9d-48b8-b2ed-c08304b9c9d8" containerName="maas-api" Jul 05 13:51:32.684462 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:51:32.684413 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8ad7e96b-80c1-4814-913b-e879550d8156" containerName="maas-api" Jul 05 13:51:32.684462 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:51:32.684418 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="8ad7e96b-80c1-4814-913b-e879550d8156" containerName="maas-api" Jul 05 13:51:32.684626 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:51:32.684481 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="8ad7e96b-80c1-4814-913b-e879550d8156" containerName="maas-api" Jul 05 13:51:32.684626 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:51:32.684492 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="81409f58-4e9d-48b8-b2ed-c08304b9c9d8" containerName="maas-api" Jul 05 13:51:32.689370 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:51:32.689351 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-a-e55a60-ffb74dc47-r5m87" Jul 05 13:51:32.692069 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:51:32.692049 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-api-a-e55a60-serving-cert\"" Jul 05 13:51:32.699947 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:51:32.699919 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-api-a-e55a60-ffb74dc47-r5m87"] Jul 05 13:51:32.745407 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:51:32.745370 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a5ddf5f0-16bd-41a9-91a0-e743a47add6d-openshift-service-ca\") pod \"maas-api-e2e-api-a-e55a60-ffb74dc47-r5m87\" (UID: \"a5ddf5f0-16bd-41a9-91a0-e743a47add6d\") " pod="opendatahub/maas-api-e2e-api-a-e55a60-ffb74dc47-r5m87" Jul 05 13:51:32.745610 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:51:32.745456 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4ft2b\" (UniqueName: \"kubernetes.io/projected/a5ddf5f0-16bd-41a9-91a0-e743a47add6d-kube-api-access-4ft2b\") pod \"maas-api-e2e-api-a-e55a60-ffb74dc47-r5m87\" (UID: \"a5ddf5f0-16bd-41a9-91a0-e743a47add6d\") " pod="opendatahub/maas-api-e2e-api-a-e55a60-ffb74dc47-r5m87" Jul 05 13:51:32.745610 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:51:32.745501 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a5ddf5f0-16bd-41a9-91a0-e743a47add6d-maas-api-tls\") pod \"maas-api-e2e-api-a-e55a60-ffb74dc47-r5m87\" (UID: \"a5ddf5f0-16bd-41a9-91a0-e743a47add6d\") " pod="opendatahub/maas-api-e2e-api-a-e55a60-ffb74dc47-r5m87" Jul 05 13:51:32.846729 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:51:32.846689 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4ft2b\" (UniqueName: \"kubernetes.io/projected/a5ddf5f0-16bd-41a9-91a0-e743a47add6d-kube-api-access-4ft2b\") pod \"maas-api-e2e-api-a-e55a60-ffb74dc47-r5m87\" (UID: \"a5ddf5f0-16bd-41a9-91a0-e743a47add6d\") " pod="opendatahub/maas-api-e2e-api-a-e55a60-ffb74dc47-r5m87" Jul 05 13:51:32.846937 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:51:32.846748 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a5ddf5f0-16bd-41a9-91a0-e743a47add6d-maas-api-tls\") pod \"maas-api-e2e-api-a-e55a60-ffb74dc47-r5m87\" (UID: \"a5ddf5f0-16bd-41a9-91a0-e743a47add6d\") " pod="opendatahub/maas-api-e2e-api-a-e55a60-ffb74dc47-r5m87" Jul 05 13:51:32.846937 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:51:32.846782 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a5ddf5f0-16bd-41a9-91a0-e743a47add6d-openshift-service-ca\") pod \"maas-api-e2e-api-a-e55a60-ffb74dc47-r5m87\" (UID: \"a5ddf5f0-16bd-41a9-91a0-e743a47add6d\") " pod="opendatahub/maas-api-e2e-api-a-e55a60-ffb74dc47-r5m87" Jul 05 13:51:32.846937 ip-10-0-130-191 kubenswrapper[2578]: E0705 13:51:32.846912 2578 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-api-a-e55a60-serving-cert: secret "maas-api-e2e-api-a-e55a60-serving-cert" not found Jul 05 13:51:32.847119 ip-10-0-130-191 kubenswrapper[2578]: E0705 13:51:32.846982 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a5ddf5f0-16bd-41a9-91a0-e743a47add6d-maas-api-tls podName:a5ddf5f0-16bd-41a9-91a0-e743a47add6d nodeName:}" failed. No retries permitted until 2026-07-05 13:51:33.346963167 +0000 UTC m=+2177.805526249 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/a5ddf5f0-16bd-41a9-91a0-e743a47add6d-maas-api-tls") pod "maas-api-e2e-api-a-e55a60-ffb74dc47-r5m87" (UID: "a5ddf5f0-16bd-41a9-91a0-e743a47add6d") : secret "maas-api-e2e-api-a-e55a60-serving-cert" not found Jul 05 13:51:32.847415 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:51:32.847396 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a5ddf5f0-16bd-41a9-91a0-e743a47add6d-openshift-service-ca\") pod \"maas-api-e2e-api-a-e55a60-ffb74dc47-r5m87\" (UID: \"a5ddf5f0-16bd-41a9-91a0-e743a47add6d\") " pod="opendatahub/maas-api-e2e-api-a-e55a60-ffb74dc47-r5m87" Jul 05 13:51:32.860007 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:51:32.859971 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4ft2b\" (UniqueName: \"kubernetes.io/projected/a5ddf5f0-16bd-41a9-91a0-e743a47add6d-kube-api-access-4ft2b\") pod \"maas-api-e2e-api-a-e55a60-ffb74dc47-r5m87\" (UID: \"a5ddf5f0-16bd-41a9-91a0-e743a47add6d\") " pod="opendatahub/maas-api-e2e-api-a-e55a60-ffb74dc47-r5m87" Jul 05 13:51:33.351200 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:51:33.351165 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a5ddf5f0-16bd-41a9-91a0-e743a47add6d-maas-api-tls\") pod \"maas-api-e2e-api-a-e55a60-ffb74dc47-r5m87\" (UID: \"a5ddf5f0-16bd-41a9-91a0-e743a47add6d\") " pod="opendatahub/maas-api-e2e-api-a-e55a60-ffb74dc47-r5m87" Jul 05 13:51:33.353718 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:51:33.353690 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a5ddf5f0-16bd-41a9-91a0-e743a47add6d-maas-api-tls\") pod \"maas-api-e2e-api-a-e55a60-ffb74dc47-r5m87\" (UID: \"a5ddf5f0-16bd-41a9-91a0-e743a47add6d\") " pod="opendatahub/maas-api-e2e-api-a-e55a60-ffb74dc47-r5m87" Jul 05 13:51:33.602064 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:51:33.601934 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-a-e55a60-ffb74dc47-r5m87" Jul 05 13:51:33.753259 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:51:33.753232 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-api-a-e55a60-ffb74dc47-r5m87"] Jul 05 13:51:33.755913 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:51:33.755886 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda5ddf5f0_16bd_41a9_91a0_e743a47add6d.slice/crio-3856471562c6a50b620bef8558af9c2e49d8d53670e39a50dfd907cc3998c03b WatchSource:0}: Error finding container 3856471562c6a50b620bef8558af9c2e49d8d53670e39a50dfd907cc3998c03b: Status 404 returned error can't find the container with id 3856471562c6a50b620bef8558af9c2e49d8d53670e39a50dfd907cc3998c03b Jul 05 13:51:34.226074 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:51:34.226041 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-a-e55a60-ffb74dc47-r5m87" event={"ID":"a5ddf5f0-16bd-41a9-91a0-e743a47add6d","Type":"ContainerStarted","Data":"3856471562c6a50b620bef8558af9c2e49d8d53670e39a50dfd907cc3998c03b"} Jul 05 13:51:35.233914 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:51:35.233874 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-a-e55a60-ffb74dc47-r5m87" event={"ID":"a5ddf5f0-16bd-41a9-91a0-e743a47add6d","Type":"ContainerStarted","Data":"dd7e173377137a4eca9236ea30df4d1177a01d137c274d72eb9111e9c0de1d82"} Jul 05 13:51:35.234421 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:51:35.234007 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-api-a-e55a60-ffb74dc47-r5m87" Jul 05 13:51:35.260900 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:51:35.260854 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-api-a-e55a60-ffb74dc47-r5m87" podStartSLOduration=2.755701095 podStartE2EDuration="3.260839956s" totalImagesPullingTime="505.138861ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 13:51:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 13:51:35.258446343 +0000 UTC m=+2179.717009447" watchObservedRunningTime="2026-07-05 13:51:35.260839956 +0000 UTC m=+2179.719403060" Jul 05 13:51:40.793896 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:51:40.793852 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-api-b-853a17-677d5fdb4f-jkj8c"] Jul 05 13:51:40.797672 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:51:40.797647 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-b-853a17-677d5fdb4f-jkj8c" Jul 05 13:51:40.800365 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:51:40.800340 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-api-b-853a17-serving-cert\"" Jul 05 13:51:40.815051 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:51:40.815027 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-api-b-853a17-677d5fdb4f-jkj8c"] Jul 05 13:51:40.923650 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:51:40.923610 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5wbdn\" (UniqueName: \"kubernetes.io/projected/583e1d75-628d-4e7d-8f10-24cddbeac35c-kube-api-access-5wbdn\") pod \"maas-api-e2e-api-b-853a17-677d5fdb4f-jkj8c\" (UID: \"583e1d75-628d-4e7d-8f10-24cddbeac35c\") " pod="opendatahub/maas-api-e2e-api-b-853a17-677d5fdb4f-jkj8c" Jul 05 13:51:40.923831 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:51:40.923745 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/583e1d75-628d-4e7d-8f10-24cddbeac35c-maas-api-tls\") pod \"maas-api-e2e-api-b-853a17-677d5fdb4f-jkj8c\" (UID: \"583e1d75-628d-4e7d-8f10-24cddbeac35c\") " pod="opendatahub/maas-api-e2e-api-b-853a17-677d5fdb4f-jkj8c" Jul 05 13:51:40.923831 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:51:40.923803 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/583e1d75-628d-4e7d-8f10-24cddbeac35c-openshift-service-ca\") pod \"maas-api-e2e-api-b-853a17-677d5fdb4f-jkj8c\" (UID: \"583e1d75-628d-4e7d-8f10-24cddbeac35c\") " pod="opendatahub/maas-api-e2e-api-b-853a17-677d5fdb4f-jkj8c" Jul 05 13:51:41.024468 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:51:41.024433 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/583e1d75-628d-4e7d-8f10-24cddbeac35c-maas-api-tls\") pod \"maas-api-e2e-api-b-853a17-677d5fdb4f-jkj8c\" (UID: \"583e1d75-628d-4e7d-8f10-24cddbeac35c\") " pod="opendatahub/maas-api-e2e-api-b-853a17-677d5fdb4f-jkj8c" Jul 05 13:51:41.024711 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:51:41.024487 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/583e1d75-628d-4e7d-8f10-24cddbeac35c-openshift-service-ca\") pod \"maas-api-e2e-api-b-853a17-677d5fdb4f-jkj8c\" (UID: \"583e1d75-628d-4e7d-8f10-24cddbeac35c\") " pod="opendatahub/maas-api-e2e-api-b-853a17-677d5fdb4f-jkj8c" Jul 05 13:51:41.024711 ip-10-0-130-191 kubenswrapper[2578]: E0705 13:51:41.024633 2578 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-api-b-853a17-serving-cert: secret "maas-api-e2e-api-b-853a17-serving-cert" not found Jul 05 13:51:41.024855 ip-10-0-130-191 kubenswrapper[2578]: E0705 13:51:41.024723 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/583e1d75-628d-4e7d-8f10-24cddbeac35c-maas-api-tls podName:583e1d75-628d-4e7d-8f10-24cddbeac35c nodeName:}" failed. No retries permitted until 2026-07-05 13:51:41.524700224 +0000 UTC m=+2185.983263307 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/583e1d75-628d-4e7d-8f10-24cddbeac35c-maas-api-tls") pod "maas-api-e2e-api-b-853a17-677d5fdb4f-jkj8c" (UID: "583e1d75-628d-4e7d-8f10-24cddbeac35c") : secret "maas-api-e2e-api-b-853a17-serving-cert" not found Jul 05 13:51:41.024855 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:51:41.024630 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5wbdn\" (UniqueName: \"kubernetes.io/projected/583e1d75-628d-4e7d-8f10-24cddbeac35c-kube-api-access-5wbdn\") pod \"maas-api-e2e-api-b-853a17-677d5fdb4f-jkj8c\" (UID: \"583e1d75-628d-4e7d-8f10-24cddbeac35c\") " pod="opendatahub/maas-api-e2e-api-b-853a17-677d5fdb4f-jkj8c" Jul 05 13:51:41.025216 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:51:41.025198 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/583e1d75-628d-4e7d-8f10-24cddbeac35c-openshift-service-ca\") pod \"maas-api-e2e-api-b-853a17-677d5fdb4f-jkj8c\" (UID: \"583e1d75-628d-4e7d-8f10-24cddbeac35c\") " pod="opendatahub/maas-api-e2e-api-b-853a17-677d5fdb4f-jkj8c" Jul 05 13:51:41.038239 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:51:41.038207 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5wbdn\" (UniqueName: \"kubernetes.io/projected/583e1d75-628d-4e7d-8f10-24cddbeac35c-kube-api-access-5wbdn\") pod \"maas-api-e2e-api-b-853a17-677d5fdb4f-jkj8c\" (UID: \"583e1d75-628d-4e7d-8f10-24cddbeac35c\") " pod="opendatahub/maas-api-e2e-api-b-853a17-677d5fdb4f-jkj8c" Jul 05 13:51:41.245317 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:51:41.245293 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-api-a-e55a60-ffb74dc47-r5m87" Jul 05 13:51:41.530599 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:51:41.530471 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/583e1d75-628d-4e7d-8f10-24cddbeac35c-maas-api-tls\") pod \"maas-api-e2e-api-b-853a17-677d5fdb4f-jkj8c\" (UID: \"583e1d75-628d-4e7d-8f10-24cddbeac35c\") " pod="opendatahub/maas-api-e2e-api-b-853a17-677d5fdb4f-jkj8c" Jul 05 13:51:41.533782 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:51:41.533755 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/583e1d75-628d-4e7d-8f10-24cddbeac35c-maas-api-tls\") pod \"maas-api-e2e-api-b-853a17-677d5fdb4f-jkj8c\" (UID: \"583e1d75-628d-4e7d-8f10-24cddbeac35c\") " pod="opendatahub/maas-api-e2e-api-b-853a17-677d5fdb4f-jkj8c" Jul 05 13:51:41.708735 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:51:41.708696 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-b-853a17-677d5fdb4f-jkj8c" Jul 05 13:51:41.853022 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:51:41.852994 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-api-b-853a17-677d5fdb4f-jkj8c"] Jul 05 13:51:41.854858 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:51:41.854825 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod583e1d75_628d_4e7d_8f10_24cddbeac35c.slice/crio-b568dd8dcd6da66732a1018269703c0e8a71aa9be14743f268b1f2bb51558334 WatchSource:0}: Error finding container b568dd8dcd6da66732a1018269703c0e8a71aa9be14743f268b1f2bb51558334: Status 404 returned error can't find the container with id b568dd8dcd6da66732a1018269703c0e8a71aa9be14743f268b1f2bb51558334 Jul 05 13:51:42.275813 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:51:42.275691 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-b-853a17-677d5fdb4f-jkj8c" event={"ID":"583e1d75-628d-4e7d-8f10-24cddbeac35c","Type":"ContainerStarted","Data":"b568dd8dcd6da66732a1018269703c0e8a71aa9be14743f268b1f2bb51558334"} Jul 05 13:51:43.283782 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:51:43.283736 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-b-853a17-677d5fdb4f-jkj8c" event={"ID":"583e1d75-628d-4e7d-8f10-24cddbeac35c","Type":"ContainerStarted","Data":"155323d1b50f1649a5a2324874d9c26f8f69789a546f11b37b2005e81a79838d"} Jul 05 13:51:43.284189 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:51:43.283816 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-api-b-853a17-677d5fdb4f-jkj8c" Jul 05 13:51:43.302130 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:51:43.302073 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-api-b-853a17-677d5fdb4f-jkj8c" podStartSLOduration=2.883316528 podStartE2EDuration="3.30205525s" totalImagesPullingTime="418.738722ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 13:51:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 13:51:43.301153932 +0000 UTC m=+2187.759717039" watchObservedRunningTime="2026-07-05 13:51:43.30205525 +0000 UTC m=+2187.760618403" Jul 05 13:51:49.295944 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:51:49.295913 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-api-b-853a17-677d5fdb4f-jkj8c" Jul 05 13:51:54.178775 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:51:54.178732 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-api-a-e55a60-ffb74dc47-r5m87"] Jul 05 13:51:54.179260 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:51:54.179007 2578 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-api-a-e55a60-ffb74dc47-r5m87" podUID="a5ddf5f0-16bd-41a9-91a0-e743a47add6d" containerName="maas-api" containerID="cri-o://dd7e173377137a4eca9236ea30df4d1177a01d137c274d72eb9111e9c0de1d82" gracePeriod=30 Jul 05 13:51:54.347727 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:51:54.347698 2578 generic.go:356] "Generic (PLEG): container finished" podID="a5ddf5f0-16bd-41a9-91a0-e743a47add6d" containerID="dd7e173377137a4eca9236ea30df4d1177a01d137c274d72eb9111e9c0de1d82" exitCode=0 Jul 05 13:51:54.347872 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:51:54.347790 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-a-e55a60-ffb74dc47-r5m87" event={"ID":"a5ddf5f0-16bd-41a9-91a0-e743a47add6d","Type":"ContainerDied","Data":"dd7e173377137a4eca9236ea30df4d1177a01d137c274d72eb9111e9c0de1d82"} Jul 05 13:51:54.431202 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:51:54.431136 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-a-e55a60-ffb74dc47-r5m87" Jul 05 13:51:54.555860 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:51:54.555820 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a5ddf5f0-16bd-41a9-91a0-e743a47add6d-openshift-service-ca\") pod \"a5ddf5f0-16bd-41a9-91a0-e743a47add6d\" (UID: \"a5ddf5f0-16bd-41a9-91a0-e743a47add6d\") " Jul 05 13:51:54.556048 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:51:54.555879 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a5ddf5f0-16bd-41a9-91a0-e743a47add6d-maas-api-tls\") pod \"a5ddf5f0-16bd-41a9-91a0-e743a47add6d\" (UID: \"a5ddf5f0-16bd-41a9-91a0-e743a47add6d\") " Jul 05 13:51:54.556048 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:51:54.555959 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4ft2b\" (UniqueName: \"kubernetes.io/projected/a5ddf5f0-16bd-41a9-91a0-e743a47add6d-kube-api-access-4ft2b\") pod \"a5ddf5f0-16bd-41a9-91a0-e743a47add6d\" (UID: \"a5ddf5f0-16bd-41a9-91a0-e743a47add6d\") " Jul 05 13:51:54.556275 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:51:54.556246 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a5ddf5f0-16bd-41a9-91a0-e743a47add6d-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "a5ddf5f0-16bd-41a9-91a0-e743a47add6d" (UID: "a5ddf5f0-16bd-41a9-91a0-e743a47add6d"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 05 13:51:54.558184 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:51:54.558160 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a5ddf5f0-16bd-41a9-91a0-e743a47add6d-kube-api-access-4ft2b" (OuterVolumeSpecName: "kube-api-access-4ft2b") pod "a5ddf5f0-16bd-41a9-91a0-e743a47add6d" (UID: "a5ddf5f0-16bd-41a9-91a0-e743a47add6d"). InnerVolumeSpecName "kube-api-access-4ft2b". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 05 13:51:54.558184 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:51:54.558161 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a5ddf5f0-16bd-41a9-91a0-e743a47add6d-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "a5ddf5f0-16bd-41a9-91a0-e743a47add6d" (UID: "a5ddf5f0-16bd-41a9-91a0-e743a47add6d"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 05 13:51:54.656635 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:51:54.656563 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4ft2b\" (UniqueName: \"kubernetes.io/projected/a5ddf5f0-16bd-41a9-91a0-e743a47add6d-kube-api-access-4ft2b\") on node \"ip-10-0-130-191.ec2.internal\" DevicePath \"\"" Jul 05 13:51:54.656635 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:51:54.656628 2578 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a5ddf5f0-16bd-41a9-91a0-e743a47add6d-openshift-service-ca\") on node \"ip-10-0-130-191.ec2.internal\" DevicePath \"\"" Jul 05 13:51:54.656635 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:51:54.656639 2578 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a5ddf5f0-16bd-41a9-91a0-e743a47add6d-maas-api-tls\") on node \"ip-10-0-130-191.ec2.internal\" DevicePath \"\"" Jul 05 13:51:55.355074 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:51:55.355046 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-a-e55a60-ffb74dc47-r5m87" Jul 05 13:51:55.355490 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:51:55.355090 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-a-e55a60-ffb74dc47-r5m87" event={"ID":"a5ddf5f0-16bd-41a9-91a0-e743a47add6d","Type":"ContainerDied","Data":"3856471562c6a50b620bef8558af9c2e49d8d53670e39a50dfd907cc3998c03b"} Jul 05 13:51:55.355490 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:51:55.355152 2578 scope.go:117] "RemoveContainer" containerID="dd7e173377137a4eca9236ea30df4d1177a01d137c274d72eb9111e9c0de1d82" Jul 05 13:51:55.379013 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:51:55.378986 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-api-a-e55a60-ffb74dc47-r5m87"] Jul 05 13:51:55.384382 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:51:55.384353 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-api-a-e55a60-ffb74dc47-r5m87"] Jul 05 13:51:56.009765 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:51:56.009732 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a5ddf5f0-16bd-41a9-91a0-e743a47add6d" path="/var/lib/kubelet/pods/a5ddf5f0-16bd-41a9-91a0-e743a47add6d/volumes" Jul 05 13:52:01.039077 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:52:01.039037 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-api-b-853a17-677d5fdb4f-jkj8c"] Jul 05 13:52:01.039608 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:52:01.039283 2578 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-api-b-853a17-677d5fdb4f-jkj8c" podUID="583e1d75-628d-4e7d-8f10-24cddbeac35c" containerName="maas-api" containerID="cri-o://155323d1b50f1649a5a2324874d9c26f8f69789a546f11b37b2005e81a79838d" gracePeriod=30 Jul 05 13:52:01.296050 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:52:01.295990 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-b-853a17-677d5fdb4f-jkj8c" Jul 05 13:52:01.320076 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:52:01.320050 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/583e1d75-628d-4e7d-8f10-24cddbeac35c-maas-api-tls\") pod \"583e1d75-628d-4e7d-8f10-24cddbeac35c\" (UID: \"583e1d75-628d-4e7d-8f10-24cddbeac35c\") " Jul 05 13:52:01.320245 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:52:01.320085 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5wbdn\" (UniqueName: \"kubernetes.io/projected/583e1d75-628d-4e7d-8f10-24cddbeac35c-kube-api-access-5wbdn\") pod \"583e1d75-628d-4e7d-8f10-24cddbeac35c\" (UID: \"583e1d75-628d-4e7d-8f10-24cddbeac35c\") " Jul 05 13:52:01.320245 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:52:01.320109 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/583e1d75-628d-4e7d-8f10-24cddbeac35c-openshift-service-ca\") pod \"583e1d75-628d-4e7d-8f10-24cddbeac35c\" (UID: \"583e1d75-628d-4e7d-8f10-24cddbeac35c\") " Jul 05 13:52:01.320562 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:52:01.320536 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/583e1d75-628d-4e7d-8f10-24cddbeac35c-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "583e1d75-628d-4e7d-8f10-24cddbeac35c" (UID: "583e1d75-628d-4e7d-8f10-24cddbeac35c"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 05 13:52:01.322458 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:52:01.322425 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/583e1d75-628d-4e7d-8f10-24cddbeac35c-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "583e1d75-628d-4e7d-8f10-24cddbeac35c" (UID: "583e1d75-628d-4e7d-8f10-24cddbeac35c"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 05 13:52:01.322622 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:52:01.322463 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/583e1d75-628d-4e7d-8f10-24cddbeac35c-kube-api-access-5wbdn" (OuterVolumeSpecName: "kube-api-access-5wbdn") pod "583e1d75-628d-4e7d-8f10-24cddbeac35c" (UID: "583e1d75-628d-4e7d-8f10-24cddbeac35c"). InnerVolumeSpecName "kube-api-access-5wbdn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 05 13:52:01.389485 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:52:01.389453 2578 generic.go:356] "Generic (PLEG): container finished" podID="583e1d75-628d-4e7d-8f10-24cddbeac35c" containerID="155323d1b50f1649a5a2324874d9c26f8f69789a546f11b37b2005e81a79838d" exitCode=0 Jul 05 13:52:01.389689 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:52:01.389495 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-b-853a17-677d5fdb4f-jkj8c" Jul 05 13:52:01.389689 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:52:01.389538 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-b-853a17-677d5fdb4f-jkj8c" event={"ID":"583e1d75-628d-4e7d-8f10-24cddbeac35c","Type":"ContainerDied","Data":"155323d1b50f1649a5a2324874d9c26f8f69789a546f11b37b2005e81a79838d"} Jul 05 13:52:01.389689 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:52:01.389620 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-b-853a17-677d5fdb4f-jkj8c" event={"ID":"583e1d75-628d-4e7d-8f10-24cddbeac35c","Type":"ContainerDied","Data":"b568dd8dcd6da66732a1018269703c0e8a71aa9be14743f268b1f2bb51558334"} Jul 05 13:52:01.389689 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:52:01.389647 2578 scope.go:117] "RemoveContainer" containerID="155323d1b50f1649a5a2324874d9c26f8f69789a546f11b37b2005e81a79838d" Jul 05 13:52:01.399564 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:52:01.399547 2578 scope.go:117] "RemoveContainer" containerID="155323d1b50f1649a5a2324874d9c26f8f69789a546f11b37b2005e81a79838d" Jul 05 13:52:01.399891 ip-10-0-130-191 kubenswrapper[2578]: E0705 13:52:01.399870 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"155323d1b50f1649a5a2324874d9c26f8f69789a546f11b37b2005e81a79838d\": container with ID starting with 155323d1b50f1649a5a2324874d9c26f8f69789a546f11b37b2005e81a79838d not found: ID does not exist" containerID="155323d1b50f1649a5a2324874d9c26f8f69789a546f11b37b2005e81a79838d" Jul 05 13:52:01.399932 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:52:01.399902 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"155323d1b50f1649a5a2324874d9c26f8f69789a546f11b37b2005e81a79838d"} err="failed to get container status \"155323d1b50f1649a5a2324874d9c26f8f69789a546f11b37b2005e81a79838d\": rpc error: code = NotFound desc = could not find container \"155323d1b50f1649a5a2324874d9c26f8f69789a546f11b37b2005e81a79838d\": container with ID starting with 155323d1b50f1649a5a2324874d9c26f8f69789a546f11b37b2005e81a79838d not found: ID does not exist" Jul 05 13:52:01.412179 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:52:01.412139 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-api-b-853a17-677d5fdb4f-jkj8c"] Jul 05 13:52:01.416342 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:52:01.416316 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-api-b-853a17-677d5fdb4f-jkj8c"] Jul 05 13:52:01.420973 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:52:01.420953 2578 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/583e1d75-628d-4e7d-8f10-24cddbeac35c-maas-api-tls\") on node \"ip-10-0-130-191.ec2.internal\" DevicePath \"\"" Jul 05 13:52:01.421042 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:52:01.420977 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5wbdn\" (UniqueName: \"kubernetes.io/projected/583e1d75-628d-4e7d-8f10-24cddbeac35c-kube-api-access-5wbdn\") on node \"ip-10-0-130-191.ec2.internal\" DevicePath \"\"" Jul 05 13:52:01.421042 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:52:01.420986 2578 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/583e1d75-628d-4e7d-8f10-24cddbeac35c-openshift-service-ca\") on node \"ip-10-0-130-191.ec2.internal\" DevicePath \"\"" Jul 05 13:52:02.008927 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:52:02.008894 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="583e1d75-628d-4e7d-8f10-24cddbeac35c" path="/var/lib/kubelet/pods/583e1d75-628d-4e7d-8f10-24cddbeac35c/volumes" Jul 05 13:52:10.803990 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:52:10.803952 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-shared-a-94581b-6676f8b4bf-jwknr"] Jul 05 13:52:10.804497 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:52:10.804482 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a5ddf5f0-16bd-41a9-91a0-e743a47add6d" containerName="maas-api" Jul 05 13:52:10.804540 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:52:10.804501 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="a5ddf5f0-16bd-41a9-91a0-e743a47add6d" containerName="maas-api" Jul 05 13:52:10.804540 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:52:10.804537 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="583e1d75-628d-4e7d-8f10-24cddbeac35c" containerName="maas-api" Jul 05 13:52:10.804628 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:52:10.804546 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="583e1d75-628d-4e7d-8f10-24cddbeac35c" containerName="maas-api" Jul 05 13:52:10.804664 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:52:10.804654 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="583e1d75-628d-4e7d-8f10-24cddbeac35c" containerName="maas-api" Jul 05 13:52:10.804702 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:52:10.804670 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="a5ddf5f0-16bd-41a9-91a0-e743a47add6d" containerName="maas-api" Jul 05 13:52:10.809319 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:52:10.809302 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-a-94581b-6676f8b4bf-jwknr" Jul 05 13:52:10.811834 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:52:10.811813 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-shared-a-94581b-serving-cert\"" Jul 05 13:52:10.819376 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:52:10.819299 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-shared-a-94581b-6676f8b4bf-jwknr"] Jul 05 13:52:10.907132 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:52:10.907090 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/466d9763-b7d9-4e40-b241-f10b7b94c481-openshift-service-ca\") pod \"maas-api-e2e-shared-a-94581b-6676f8b4bf-jwknr\" (UID: \"466d9763-b7d9-4e40-b241-f10b7b94c481\") " pod="opendatahub/maas-api-e2e-shared-a-94581b-6676f8b4bf-jwknr" Jul 05 13:52:10.907132 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:52:10.907131 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/466d9763-b7d9-4e40-b241-f10b7b94c481-maas-api-tls\") pod \"maas-api-e2e-shared-a-94581b-6676f8b4bf-jwknr\" (UID: \"466d9763-b7d9-4e40-b241-f10b7b94c481\") " pod="opendatahub/maas-api-e2e-shared-a-94581b-6676f8b4bf-jwknr" Jul 05 13:52:10.907342 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:52:10.907155 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7vr6b\" (UniqueName: \"kubernetes.io/projected/466d9763-b7d9-4e40-b241-f10b7b94c481-kube-api-access-7vr6b\") pod \"maas-api-e2e-shared-a-94581b-6676f8b4bf-jwknr\" (UID: \"466d9763-b7d9-4e40-b241-f10b7b94c481\") " pod="opendatahub/maas-api-e2e-shared-a-94581b-6676f8b4bf-jwknr" Jul 05 13:52:11.007818 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:52:11.007784 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7vr6b\" (UniqueName: \"kubernetes.io/projected/466d9763-b7d9-4e40-b241-f10b7b94c481-kube-api-access-7vr6b\") pod \"maas-api-e2e-shared-a-94581b-6676f8b4bf-jwknr\" (UID: \"466d9763-b7d9-4e40-b241-f10b7b94c481\") " pod="opendatahub/maas-api-e2e-shared-a-94581b-6676f8b4bf-jwknr" Jul 05 13:52:11.008005 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:52:11.007903 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/466d9763-b7d9-4e40-b241-f10b7b94c481-openshift-service-ca\") pod \"maas-api-e2e-shared-a-94581b-6676f8b4bf-jwknr\" (UID: \"466d9763-b7d9-4e40-b241-f10b7b94c481\") " pod="opendatahub/maas-api-e2e-shared-a-94581b-6676f8b4bf-jwknr" Jul 05 13:52:11.008005 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:52:11.007942 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/466d9763-b7d9-4e40-b241-f10b7b94c481-maas-api-tls\") pod \"maas-api-e2e-shared-a-94581b-6676f8b4bf-jwknr\" (UID: \"466d9763-b7d9-4e40-b241-f10b7b94c481\") " pod="opendatahub/maas-api-e2e-shared-a-94581b-6676f8b4bf-jwknr" Jul 05 13:52:11.008532 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:52:11.008511 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/466d9763-b7d9-4e40-b241-f10b7b94c481-openshift-service-ca\") pod \"maas-api-e2e-shared-a-94581b-6676f8b4bf-jwknr\" (UID: \"466d9763-b7d9-4e40-b241-f10b7b94c481\") " pod="opendatahub/maas-api-e2e-shared-a-94581b-6676f8b4bf-jwknr" Jul 05 13:52:11.010475 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:52:11.010454 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/466d9763-b7d9-4e40-b241-f10b7b94c481-maas-api-tls\") pod \"maas-api-e2e-shared-a-94581b-6676f8b4bf-jwknr\" (UID: \"466d9763-b7d9-4e40-b241-f10b7b94c481\") " pod="opendatahub/maas-api-e2e-shared-a-94581b-6676f8b4bf-jwknr" Jul 05 13:52:11.016204 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:52:11.016180 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7vr6b\" (UniqueName: \"kubernetes.io/projected/466d9763-b7d9-4e40-b241-f10b7b94c481-kube-api-access-7vr6b\") pod \"maas-api-e2e-shared-a-94581b-6676f8b4bf-jwknr\" (UID: \"466d9763-b7d9-4e40-b241-f10b7b94c481\") " pod="opendatahub/maas-api-e2e-shared-a-94581b-6676f8b4bf-jwknr" Jul 05 13:52:11.120716 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:52:11.120612 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-a-94581b-6676f8b4bf-jwknr" Jul 05 13:52:11.267223 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:52:11.267195 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-shared-a-94581b-6676f8b4bf-jwknr"] Jul 05 13:52:11.268996 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:52:11.268966 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod466d9763_b7d9_4e40_b241_f10b7b94c481.slice/crio-bbfd39a838069a933f7eda05b36d2f5cea962b689314e5cb63937aab4fee04e3 WatchSource:0}: Error finding container bbfd39a838069a933f7eda05b36d2f5cea962b689314e5cb63937aab4fee04e3: Status 404 returned error can't find the container with id bbfd39a838069a933f7eda05b36d2f5cea962b689314e5cb63937aab4fee04e3 Jul 05 13:52:11.449785 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:52:11.449696 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-a-94581b-6676f8b4bf-jwknr" event={"ID":"466d9763-b7d9-4e40-b241-f10b7b94c481","Type":"ContainerStarted","Data":"bbfd39a838069a933f7eda05b36d2f5cea962b689314e5cb63937aab4fee04e3"} Jul 05 13:52:12.456559 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:52:12.456524 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-a-94581b-6676f8b4bf-jwknr" event={"ID":"466d9763-b7d9-4e40-b241-f10b7b94c481","Type":"ContainerStarted","Data":"3b92476fccf1f9c15ea8322bc2fee2c91a404eaa3ea7b1b70c0cc586128b6bd8"} Jul 05 13:52:12.457083 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:52:12.456731 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-shared-a-94581b-6676f8b4bf-jwknr" Jul 05 13:52:12.475624 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:52:12.475563 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-shared-a-94581b-6676f8b4bf-jwknr" podStartSLOduration=2.17696533 podStartE2EDuration="2.4755509s" totalImagesPullingTime="298.58557ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 13:52:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 13:52:12.473540203 +0000 UTC m=+2216.932103308" watchObservedRunningTime="2026-07-05 13:52:12.4755509 +0000 UTC m=+2216.934114005" Jul 05 13:52:18.467844 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:52:18.467815 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-shared-a-94581b-6676f8b4bf-jwknr" Jul 05 13:52:18.888541 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:52:18.888508 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-shared-b-aa2b8d-85959d48f5-g9c79"] Jul 05 13:52:18.892974 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:52:18.892958 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-b-aa2b8d-85959d48f5-g9c79" Jul 05 13:52:18.896389 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:52:18.896361 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-shared-b-aa2b8d-serving-cert\"" Jul 05 13:52:18.907742 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:52:18.907710 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-shared-b-aa2b8d-85959d48f5-g9c79"] Jul 05 13:52:18.981372 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:52:18.981336 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/80f41632-fc45-460f-a5b8-3ff9e9dd9e50-openshift-service-ca\") pod \"maas-api-e2e-shared-b-aa2b8d-85959d48f5-g9c79\" (UID: \"80f41632-fc45-460f-a5b8-3ff9e9dd9e50\") " pod="opendatahub/maas-api-e2e-shared-b-aa2b8d-85959d48f5-g9c79" Jul 05 13:52:18.981547 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:52:18.981383 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rqbxn\" (UniqueName: \"kubernetes.io/projected/80f41632-fc45-460f-a5b8-3ff9e9dd9e50-kube-api-access-rqbxn\") pod \"maas-api-e2e-shared-b-aa2b8d-85959d48f5-g9c79\" (UID: \"80f41632-fc45-460f-a5b8-3ff9e9dd9e50\") " pod="opendatahub/maas-api-e2e-shared-b-aa2b8d-85959d48f5-g9c79" Jul 05 13:52:18.981547 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:52:18.981433 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/80f41632-fc45-460f-a5b8-3ff9e9dd9e50-maas-api-tls\") pod \"maas-api-e2e-shared-b-aa2b8d-85959d48f5-g9c79\" (UID: \"80f41632-fc45-460f-a5b8-3ff9e9dd9e50\") " pod="opendatahub/maas-api-e2e-shared-b-aa2b8d-85959d48f5-g9c79" Jul 05 13:52:19.081866 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:52:19.081821 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/80f41632-fc45-460f-a5b8-3ff9e9dd9e50-maas-api-tls\") pod \"maas-api-e2e-shared-b-aa2b8d-85959d48f5-g9c79\" (UID: \"80f41632-fc45-460f-a5b8-3ff9e9dd9e50\") " pod="opendatahub/maas-api-e2e-shared-b-aa2b8d-85959d48f5-g9c79" Jul 05 13:52:19.082071 ip-10-0-130-191 kubenswrapper[2578]: E0705 13:52:19.081927 2578 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-shared-b-aa2b8d-serving-cert: secret "maas-api-e2e-shared-b-aa2b8d-serving-cert" not found Jul 05 13:52:19.082071 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:52:19.081973 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/80f41632-fc45-460f-a5b8-3ff9e9dd9e50-openshift-service-ca\") pod \"maas-api-e2e-shared-b-aa2b8d-85959d48f5-g9c79\" (UID: \"80f41632-fc45-460f-a5b8-3ff9e9dd9e50\") " pod="opendatahub/maas-api-e2e-shared-b-aa2b8d-85959d48f5-g9c79" Jul 05 13:52:19.082071 ip-10-0-130-191 kubenswrapper[2578]: E0705 13:52:19.081990 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/80f41632-fc45-460f-a5b8-3ff9e9dd9e50-maas-api-tls podName:80f41632-fc45-460f-a5b8-3ff9e9dd9e50 nodeName:}" failed. No retries permitted until 2026-07-05 13:52:19.581971752 +0000 UTC m=+2224.040534835 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/80f41632-fc45-460f-a5b8-3ff9e9dd9e50-maas-api-tls") pod "maas-api-e2e-shared-b-aa2b8d-85959d48f5-g9c79" (UID: "80f41632-fc45-460f-a5b8-3ff9e9dd9e50") : secret "maas-api-e2e-shared-b-aa2b8d-serving-cert" not found Jul 05 13:52:19.082071 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:52:19.082045 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rqbxn\" (UniqueName: \"kubernetes.io/projected/80f41632-fc45-460f-a5b8-3ff9e9dd9e50-kube-api-access-rqbxn\") pod \"maas-api-e2e-shared-b-aa2b8d-85959d48f5-g9c79\" (UID: \"80f41632-fc45-460f-a5b8-3ff9e9dd9e50\") " pod="opendatahub/maas-api-e2e-shared-b-aa2b8d-85959d48f5-g9c79" Jul 05 13:52:19.082640 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:52:19.082619 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/80f41632-fc45-460f-a5b8-3ff9e9dd9e50-openshift-service-ca\") pod \"maas-api-e2e-shared-b-aa2b8d-85959d48f5-g9c79\" (UID: \"80f41632-fc45-460f-a5b8-3ff9e9dd9e50\") " pod="opendatahub/maas-api-e2e-shared-b-aa2b8d-85959d48f5-g9c79" Jul 05 13:52:19.113049 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:52:19.113015 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rqbxn\" (UniqueName: \"kubernetes.io/projected/80f41632-fc45-460f-a5b8-3ff9e9dd9e50-kube-api-access-rqbxn\") pod \"maas-api-e2e-shared-b-aa2b8d-85959d48f5-g9c79\" (UID: \"80f41632-fc45-460f-a5b8-3ff9e9dd9e50\") " pod="opendatahub/maas-api-e2e-shared-b-aa2b8d-85959d48f5-g9c79" Jul 05 13:52:19.586778 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:52:19.586741 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/80f41632-fc45-460f-a5b8-3ff9e9dd9e50-maas-api-tls\") pod \"maas-api-e2e-shared-b-aa2b8d-85959d48f5-g9c79\" (UID: \"80f41632-fc45-460f-a5b8-3ff9e9dd9e50\") " pod="opendatahub/maas-api-e2e-shared-b-aa2b8d-85959d48f5-g9c79" Jul 05 13:52:19.589370 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:52:19.589338 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/80f41632-fc45-460f-a5b8-3ff9e9dd9e50-maas-api-tls\") pod \"maas-api-e2e-shared-b-aa2b8d-85959d48f5-g9c79\" (UID: \"80f41632-fc45-460f-a5b8-3ff9e9dd9e50\") " pod="opendatahub/maas-api-e2e-shared-b-aa2b8d-85959d48f5-g9c79" Jul 05 13:52:19.803820 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:52:19.803777 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-b-aa2b8d-85959d48f5-g9c79" Jul 05 13:52:19.948721 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:52:19.948691 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-shared-b-aa2b8d-85959d48f5-g9c79"] Jul 05 13:52:19.949314 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:52:19.949289 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod80f41632_fc45_460f_a5b8_3ff9e9dd9e50.slice/crio-bf4708b5a14eb54d0ce1cdb3e5aef15ff836fb71caf080923176929e233679f1 WatchSource:0}: Error finding container bf4708b5a14eb54d0ce1cdb3e5aef15ff836fb71caf080923176929e233679f1: Status 404 returned error can't find the container with id bf4708b5a14eb54d0ce1cdb3e5aef15ff836fb71caf080923176929e233679f1 Jul 05 13:52:20.503078 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:52:20.503035 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-b-aa2b8d-85959d48f5-g9c79" event={"ID":"80f41632-fc45-460f-a5b8-3ff9e9dd9e50","Type":"ContainerStarted","Data":"40b9ef48fb11a8b50d35402126cb86ab31826f1fe44a78654609e790366cbfcf"} Jul 05 13:52:20.503275 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:52:20.503131 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-b-aa2b8d-85959d48f5-g9c79" event={"ID":"80f41632-fc45-460f-a5b8-3ff9e9dd9e50","Type":"ContainerStarted","Data":"bf4708b5a14eb54d0ce1cdb3e5aef15ff836fb71caf080923176929e233679f1"} Jul 05 13:52:20.503275 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:52:20.503152 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-shared-b-aa2b8d-85959d48f5-g9c79" Jul 05 13:52:20.524552 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:52:20.524500 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-shared-b-aa2b8d-85959d48f5-g9c79" podStartSLOduration=2.19781261 podStartE2EDuration="2.524485583s" totalImagesPullingTime="326.672973ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 13:52:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 13:52:20.521978474 +0000 UTC m=+2224.980541579" watchObservedRunningTime="2026-07-05 13:52:20.524485583 +0000 UTC m=+2224.983048685" Jul 05 13:52:26.518351 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:52:26.518316 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-shared-b-aa2b8d-85959d48f5-g9c79" Jul 05 13:55:16.195792 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:55:16.195684 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-tg2xx_1c214417-9bef-4007-ab01-e61b84d1b35e/console-operator/1.log" Jul 05 13:55:16.202781 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:55:16.202755 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-656m6_7bdbd204-0fb7-4ebf-af05-432c370a814b/ovn-acl-logging/0.log" Jul 05 13:55:16.206674 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:55:16.206646 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-tg2xx_1c214417-9bef-4007-ab01-e61b84d1b35e/console-operator/1.log" Jul 05 13:55:16.213415 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:55:16.213395 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-656m6_7bdbd204-0fb7-4ebf-af05-432c370a814b/ovn-acl-logging/0.log" Jul 05 13:57:23.380983 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:57:23.380941 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-shared-a-94581b-6676f8b4bf-jwknr"] Jul 05 13:57:23.381473 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:57:23.381192 2578 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-shared-a-94581b-6676f8b4bf-jwknr" podUID="466d9763-b7d9-4e40-b241-f10b7b94c481" containerName="maas-api" containerID="cri-o://3b92476fccf1f9c15ea8322bc2fee2c91a404eaa3ea7b1b70c0cc586128b6bd8" gracePeriod=30 Jul 05 13:57:23.463334 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:57:23.463296 2578 prober.go:120] "Probe failed" probeType="Readiness" pod="opendatahub/maas-api-e2e-shared-a-94581b-6676f8b4bf-jwknr" podUID="466d9763-b7d9-4e40-b241-f10b7b94c481" containerName="maas-api" probeResult="failure" output="Get \"https://10.132.0.62:8443/health\": dial tcp 10.132.0.62:8443: connect: connection refused" Jul 05 13:57:23.628888 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:57:23.628863 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-a-94581b-6676f8b4bf-jwknr" Jul 05 13:57:23.755330 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:57:23.755297 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/466d9763-b7d9-4e40-b241-f10b7b94c481-openshift-service-ca\") pod \"466d9763-b7d9-4e40-b241-f10b7b94c481\" (UID: \"466d9763-b7d9-4e40-b241-f10b7b94c481\") " Jul 05 13:57:23.755532 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:57:23.755352 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7vr6b\" (UniqueName: \"kubernetes.io/projected/466d9763-b7d9-4e40-b241-f10b7b94c481-kube-api-access-7vr6b\") pod \"466d9763-b7d9-4e40-b241-f10b7b94c481\" (UID: \"466d9763-b7d9-4e40-b241-f10b7b94c481\") " Jul 05 13:57:23.755532 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:57:23.755434 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/466d9763-b7d9-4e40-b241-f10b7b94c481-maas-api-tls\") pod \"466d9763-b7d9-4e40-b241-f10b7b94c481\" (UID: \"466d9763-b7d9-4e40-b241-f10b7b94c481\") " Jul 05 13:57:23.755792 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:57:23.755765 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/466d9763-b7d9-4e40-b241-f10b7b94c481-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "466d9763-b7d9-4e40-b241-f10b7b94c481" (UID: "466d9763-b7d9-4e40-b241-f10b7b94c481"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 05 13:57:23.757733 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:57:23.757704 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/466d9763-b7d9-4e40-b241-f10b7b94c481-kube-api-access-7vr6b" (OuterVolumeSpecName: "kube-api-access-7vr6b") pod "466d9763-b7d9-4e40-b241-f10b7b94c481" (UID: "466d9763-b7d9-4e40-b241-f10b7b94c481"). InnerVolumeSpecName "kube-api-access-7vr6b". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 05 13:57:23.757733 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:57:23.757716 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/466d9763-b7d9-4e40-b241-f10b7b94c481-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "466d9763-b7d9-4e40-b241-f10b7b94c481" (UID: "466d9763-b7d9-4e40-b241-f10b7b94c481"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 05 13:57:23.856994 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:57:23.856943 2578 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/466d9763-b7d9-4e40-b241-f10b7b94c481-maas-api-tls\") on node \"ip-10-0-130-191.ec2.internal\" DevicePath \"\"" Jul 05 13:57:23.856994 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:57:23.856988 2578 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/466d9763-b7d9-4e40-b241-f10b7b94c481-openshift-service-ca\") on node \"ip-10-0-130-191.ec2.internal\" DevicePath \"\"" Jul 05 13:57:23.856994 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:57:23.857003 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7vr6b\" (UniqueName: \"kubernetes.io/projected/466d9763-b7d9-4e40-b241-f10b7b94c481-kube-api-access-7vr6b\") on node \"ip-10-0-130-191.ec2.internal\" DevicePath \"\"" Jul 05 13:57:24.176523 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:57:24.176484 2578 generic.go:356] "Generic (PLEG): container finished" podID="466d9763-b7d9-4e40-b241-f10b7b94c481" containerID="3b92476fccf1f9c15ea8322bc2fee2c91a404eaa3ea7b1b70c0cc586128b6bd8" exitCode=0 Jul 05 13:57:24.176714 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:57:24.176532 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-a-94581b-6676f8b4bf-jwknr" Jul 05 13:57:24.176714 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:57:24.176570 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-a-94581b-6676f8b4bf-jwknr" event={"ID":"466d9763-b7d9-4e40-b241-f10b7b94c481","Type":"ContainerDied","Data":"3b92476fccf1f9c15ea8322bc2fee2c91a404eaa3ea7b1b70c0cc586128b6bd8"} Jul 05 13:57:24.176714 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:57:24.176650 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-a-94581b-6676f8b4bf-jwknr" event={"ID":"466d9763-b7d9-4e40-b241-f10b7b94c481","Type":"ContainerDied","Data":"bbfd39a838069a933f7eda05b36d2f5cea962b689314e5cb63937aab4fee04e3"} Jul 05 13:57:24.176714 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:57:24.176682 2578 scope.go:117] "RemoveContainer" containerID="3b92476fccf1f9c15ea8322bc2fee2c91a404eaa3ea7b1b70c0cc586128b6bd8" Jul 05 13:57:24.186279 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:57:24.186259 2578 scope.go:117] "RemoveContainer" containerID="3b92476fccf1f9c15ea8322bc2fee2c91a404eaa3ea7b1b70c0cc586128b6bd8" Jul 05 13:57:24.186551 ip-10-0-130-191 kubenswrapper[2578]: E0705 13:57:24.186533 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3b92476fccf1f9c15ea8322bc2fee2c91a404eaa3ea7b1b70c0cc586128b6bd8\": container with ID starting with 3b92476fccf1f9c15ea8322bc2fee2c91a404eaa3ea7b1b70c0cc586128b6bd8 not found: ID does not exist" containerID="3b92476fccf1f9c15ea8322bc2fee2c91a404eaa3ea7b1b70c0cc586128b6bd8" Jul 05 13:57:24.186631 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:57:24.186559 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3b92476fccf1f9c15ea8322bc2fee2c91a404eaa3ea7b1b70c0cc586128b6bd8"} err="failed to get container status \"3b92476fccf1f9c15ea8322bc2fee2c91a404eaa3ea7b1b70c0cc586128b6bd8\": rpc error: code = NotFound desc = could not find container \"3b92476fccf1f9c15ea8322bc2fee2c91a404eaa3ea7b1b70c0cc586128b6bd8\": container with ID starting with 3b92476fccf1f9c15ea8322bc2fee2c91a404eaa3ea7b1b70c0cc586128b6bd8 not found: ID does not exist" Jul 05 13:57:24.194179 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:57:24.194159 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-shared-a-94581b-6676f8b4bf-jwknr"] Jul 05 13:57:24.196346 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:57:24.196329 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-shared-a-94581b-6676f8b4bf-jwknr"] Jul 05 13:57:26.009575 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:57:26.009541 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="466d9763-b7d9-4e40-b241-f10b7b94c481" path="/var/lib/kubelet/pods/466d9763-b7d9-4e40-b241-f10b7b94c481/volumes" Jul 05 13:58:06.065843 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:58:06.065806 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-shared-b-aa2b8d-85959d48f5-g9c79"] Jul 05 13:58:06.066500 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:58:06.066035 2578 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-shared-b-aa2b8d-85959d48f5-g9c79" podUID="80f41632-fc45-460f-a5b8-3ff9e9dd9e50" containerName="maas-api" containerID="cri-o://40b9ef48fb11a8b50d35402126cb86ab31826f1fe44a78654609e790366cbfcf" gracePeriod=30 Jul 05 13:58:06.314549 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:58:06.314524 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-b-aa2b8d-85959d48f5-g9c79" Jul 05 13:58:06.409824 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:58:06.409730 2578 generic.go:356] "Generic (PLEG): container finished" podID="80f41632-fc45-460f-a5b8-3ff9e9dd9e50" containerID="40b9ef48fb11a8b50d35402126cb86ab31826f1fe44a78654609e790366cbfcf" exitCode=0 Jul 05 13:58:06.409824 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:58:06.409782 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-b-aa2b8d-85959d48f5-g9c79" Jul 05 13:58:06.410007 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:58:06.409816 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-b-aa2b8d-85959d48f5-g9c79" event={"ID":"80f41632-fc45-460f-a5b8-3ff9e9dd9e50","Type":"ContainerDied","Data":"40b9ef48fb11a8b50d35402126cb86ab31826f1fe44a78654609e790366cbfcf"} Jul 05 13:58:06.410007 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:58:06.409871 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-b-aa2b8d-85959d48f5-g9c79" event={"ID":"80f41632-fc45-460f-a5b8-3ff9e9dd9e50","Type":"ContainerDied","Data":"bf4708b5a14eb54d0ce1cdb3e5aef15ff836fb71caf080923176929e233679f1"} Jul 05 13:58:06.410007 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:58:06.409893 2578 scope.go:117] "RemoveContainer" containerID="40b9ef48fb11a8b50d35402126cb86ab31826f1fe44a78654609e790366cbfcf" Jul 05 13:58:06.419085 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:58:06.419065 2578 scope.go:117] "RemoveContainer" containerID="40b9ef48fb11a8b50d35402126cb86ab31826f1fe44a78654609e790366cbfcf" Jul 05 13:58:06.419376 ip-10-0-130-191 kubenswrapper[2578]: E0705 13:58:06.419355 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"40b9ef48fb11a8b50d35402126cb86ab31826f1fe44a78654609e790366cbfcf\": container with ID starting with 40b9ef48fb11a8b50d35402126cb86ab31826f1fe44a78654609e790366cbfcf not found: ID does not exist" containerID="40b9ef48fb11a8b50d35402126cb86ab31826f1fe44a78654609e790366cbfcf" Jul 05 13:58:06.419449 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:58:06.419390 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"40b9ef48fb11a8b50d35402126cb86ab31826f1fe44a78654609e790366cbfcf"} err="failed to get container status \"40b9ef48fb11a8b50d35402126cb86ab31826f1fe44a78654609e790366cbfcf\": rpc error: code = NotFound desc = could not find container \"40b9ef48fb11a8b50d35402126cb86ab31826f1fe44a78654609e790366cbfcf\": container with ID starting with 40b9ef48fb11a8b50d35402126cb86ab31826f1fe44a78654609e790366cbfcf not found: ID does not exist" Jul 05 13:58:06.427596 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:58:06.427556 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/80f41632-fc45-460f-a5b8-3ff9e9dd9e50-maas-api-tls\") pod \"80f41632-fc45-460f-a5b8-3ff9e9dd9e50\" (UID: \"80f41632-fc45-460f-a5b8-3ff9e9dd9e50\") " Jul 05 13:58:06.427749 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:58:06.427609 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rqbxn\" (UniqueName: \"kubernetes.io/projected/80f41632-fc45-460f-a5b8-3ff9e9dd9e50-kube-api-access-rqbxn\") pod \"80f41632-fc45-460f-a5b8-3ff9e9dd9e50\" (UID: \"80f41632-fc45-460f-a5b8-3ff9e9dd9e50\") " Jul 05 13:58:06.427749 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:58:06.427665 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/80f41632-fc45-460f-a5b8-3ff9e9dd9e50-openshift-service-ca\") pod \"80f41632-fc45-460f-a5b8-3ff9e9dd9e50\" (UID: \"80f41632-fc45-460f-a5b8-3ff9e9dd9e50\") " Jul 05 13:58:06.428050 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:58:06.428020 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/80f41632-fc45-460f-a5b8-3ff9e9dd9e50-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "80f41632-fc45-460f-a5b8-3ff9e9dd9e50" (UID: "80f41632-fc45-460f-a5b8-3ff9e9dd9e50"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 05 13:58:06.429789 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:58:06.429761 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/80f41632-fc45-460f-a5b8-3ff9e9dd9e50-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "80f41632-fc45-460f-a5b8-3ff9e9dd9e50" (UID: "80f41632-fc45-460f-a5b8-3ff9e9dd9e50"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 05 13:58:06.429877 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:58:06.429806 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/80f41632-fc45-460f-a5b8-3ff9e9dd9e50-kube-api-access-rqbxn" (OuterVolumeSpecName: "kube-api-access-rqbxn") pod "80f41632-fc45-460f-a5b8-3ff9e9dd9e50" (UID: "80f41632-fc45-460f-a5b8-3ff9e9dd9e50"). InnerVolumeSpecName "kube-api-access-rqbxn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 05 13:58:06.529092 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:58:06.529051 2578 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/80f41632-fc45-460f-a5b8-3ff9e9dd9e50-openshift-service-ca\") on node \"ip-10-0-130-191.ec2.internal\" DevicePath \"\"" Jul 05 13:58:06.529092 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:58:06.529085 2578 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/80f41632-fc45-460f-a5b8-3ff9e9dd9e50-maas-api-tls\") on node \"ip-10-0-130-191.ec2.internal\" DevicePath \"\"" Jul 05 13:58:06.529092 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:58:06.529098 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rqbxn\" (UniqueName: \"kubernetes.io/projected/80f41632-fc45-460f-a5b8-3ff9e9dd9e50-kube-api-access-rqbxn\") on node \"ip-10-0-130-191.ec2.internal\" DevicePath \"\"" Jul 05 13:58:06.744025 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:58:06.743989 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-shared-b-aa2b8d-85959d48f5-g9c79"] Jul 05 13:58:06.746352 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:58:06.746326 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-shared-b-aa2b8d-85959d48f5-g9c79"] Jul 05 13:58:08.009593 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:58:08.009557 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="80f41632-fc45-460f-a5b8-3ff9e9dd9e50" path="/var/lib/kubelet/pods/80f41632-fc45-460f-a5b8-3ff9e9dd9e50/volumes" Jul 05 13:58:13.004810 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:58:13.004774 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-7c96d999f9-hdwjz_51f8dd4a-c2ae-423b-93e3-a915c42ce31a/authorino/0.log" Jul 05 13:58:17.694279 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:58:17.694252 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-65c5985bdd-gfz4q_0fd9cbe7-4dfd-4e71-bc32-503f6360d12b/maas-api/0.log" Jul 05 13:58:17.803492 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:58:17.803462 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-key-cleanup-29720970-bbnmt_053537e5-cdbd-476d-a1d5-463b3c95548e/cleanup/0.log" Jul 05 13:58:18.492107 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:58:18.492075 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6674ff8b4f-rxh62_77458566-bec5-4550-8030-bf1f43f00555/copy-manifests/0.log" Jul 05 13:58:18.501076 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:58:18.501052 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6674ff8b4f-rxh62_77458566-bec5-4550-8030-bf1f43f00555/manager/0.log" Jul 05 13:58:19.473801 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:58:19.473772 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759k96jz_bc8169a7-6503-4d40-b6fa-4b225badcda0/extract/0.log" Jul 05 13:58:19.480868 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:58:19.480837 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759k96jz_bc8169a7-6503-4d40-b6fa-4b225badcda0/util/0.log" Jul 05 13:58:19.488009 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:58:19.487987 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759k96jz_bc8169a7-6503-4d40-b6fa-4b225badcda0/pull/0.log" Jul 05 13:58:19.600310 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:58:19.600280 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0nrr9v_5e2a0d79-fc0b-415e-aea9-c5ff86918630/util/0.log" Jul 05 13:58:19.607674 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:58:19.607654 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0nrr9v_5e2a0d79-fc0b-415e-aea9-c5ff86918630/pull/0.log" Jul 05 13:58:19.614423 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:58:19.614406 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0nrr9v_5e2a0d79-fc0b-415e-aea9-c5ff86918630/extract/0.log" Jul 05 13:58:19.727648 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:58:19.727547 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73pfsr7_617eb3c5-e598-495a-bcca-73f0ac7bf7cd/extract/0.log" Jul 05 13:58:19.734265 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:58:19.734248 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73pfsr7_617eb3c5-e598-495a-bcca-73f0ac7bf7cd/util/0.log" Jul 05 13:58:19.740387 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:58:19.740371 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73pfsr7_617eb3c5-e598-495a-bcca-73f0ac7bf7cd/pull/0.log" Jul 05 13:58:19.851214 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:58:19.851186 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qqr9g_44b648ae-87f1-479b-baa3-4124452bf1ab/util/0.log" Jul 05 13:58:19.858269 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:58:19.858244 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qqr9g_44b648ae-87f1-479b-baa3-4124452bf1ab/pull/0.log" Jul 05 13:58:19.864999 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:58:19.864980 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qqr9g_44b648ae-87f1-479b-baa3-4124452bf1ab/extract/0.log" Jul 05 13:58:19.984026 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:58:19.983951 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-7c96d999f9-hdwjz_51f8dd4a-c2ae-423b-93e3-a915c42ce31a/authorino/0.log" Jul 05 13:58:20.320858 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:58:20.320818 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-xf4jh_dc8dab78-d3b4-4e5a-96e7-cf80d35cdc84/kuadrant-console-plugin/0.log" Jul 05 13:58:21.460819 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:58:21.460789 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7bcdf4856b-r2bgp_ceedc936-a8ca-4855-9484-675b40b24330/kube-auth-proxy/0.log" Jul 05 13:58:21.693898 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:58:21.693865 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-wn6j4_0e615cfd-cbda-41d6-af90-ad6958fa40c9/payload-pre-processing/0.log" Jul 05 13:58:22.372004 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:58:22.371975 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-x9kvt_8fd8e4e7-246a-4591-94c9-d7e662169ca8/main/0.log" Jul 05 13:58:22.592026 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:58:22.592000 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbvcspf_155ef003-5f1d-49d9-be0d-48a9942cf9a7/main/0.log" Jul 05 13:58:29.822474 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:58:29.822422 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-72444_4ff28158-61b2-4cb9-8b92-2ac0793c23bb/global-pull-secret-syncer/0.log" Jul 05 13:58:29.983921 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:58:29.983892 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-kpmmd_e8e7535a-8f4e-4fd6-95d2-4cb8888b284b/konnectivity-agent/0.log" Jul 05 13:58:30.059058 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:58:30.059030 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-130-191.ec2.internal_d82e074a4be1637d97b91534bd80eb21/haproxy/0.log" Jul 05 13:58:34.720322 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:58:34.720264 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759k96jz_bc8169a7-6503-4d40-b6fa-4b225badcda0/extract/0.log" Jul 05 13:58:34.756401 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:58:34.756370 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759k96jz_bc8169a7-6503-4d40-b6fa-4b225badcda0/util/0.log" Jul 05 13:58:34.794709 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:58:34.794678 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759k96jz_bc8169a7-6503-4d40-b6fa-4b225badcda0/pull/0.log" Jul 05 13:58:34.844696 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:58:34.844658 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0nrr9v_5e2a0d79-fc0b-415e-aea9-c5ff86918630/extract/0.log" Jul 05 13:58:34.870759 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:58:34.870732 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0nrr9v_5e2a0d79-fc0b-415e-aea9-c5ff86918630/util/0.log" Jul 05 13:58:34.896088 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:58:34.896061 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0nrr9v_5e2a0d79-fc0b-415e-aea9-c5ff86918630/pull/0.log" Jul 05 13:58:34.928092 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:58:34.928067 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73pfsr7_617eb3c5-e598-495a-bcca-73f0ac7bf7cd/extract/0.log" Jul 05 13:58:34.952949 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:58:34.952918 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73pfsr7_617eb3c5-e598-495a-bcca-73f0ac7bf7cd/util/0.log" Jul 05 13:58:34.974703 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:58:34.974627 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73pfsr7_617eb3c5-e598-495a-bcca-73f0ac7bf7cd/pull/0.log" Jul 05 13:58:35.004751 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:58:35.004727 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qqr9g_44b648ae-87f1-479b-baa3-4124452bf1ab/extract/0.log" Jul 05 13:58:35.028500 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:58:35.028475 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qqr9g_44b648ae-87f1-479b-baa3-4124452bf1ab/util/0.log" Jul 05 13:58:35.055599 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:58:35.055556 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qqr9g_44b648ae-87f1-479b-baa3-4124452bf1ab/pull/0.log" Jul 05 13:58:35.334650 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:58:35.334614 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-7c96d999f9-hdwjz_51f8dd4a-c2ae-423b-93e3-a915c42ce31a/authorino/0.log" Jul 05 13:58:35.421785 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:58:35.421753 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-xf4jh_dc8dab78-d3b4-4e5a-96e7-cf80d35cdc84/kuadrant-console-plugin/0.log" Jul 05 13:58:37.523254 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:58:37.523217 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_aab13c90-c88f-42ce-83df-d29f7455f494/alertmanager/0.log" Jul 05 13:58:37.546226 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:58:37.546203 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_aab13c90-c88f-42ce-83df-d29f7455f494/config-reloader/0.log" Jul 05 13:58:37.570573 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:58:37.570542 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_aab13c90-c88f-42ce-83df-d29f7455f494/kube-rbac-proxy-web/0.log" Jul 05 13:58:37.593853 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:58:37.593831 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_aab13c90-c88f-42ce-83df-d29f7455f494/kube-rbac-proxy/0.log" Jul 05 13:58:37.618481 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:58:37.618461 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_aab13c90-c88f-42ce-83df-d29f7455f494/kube-rbac-proxy-metric/0.log" Jul 05 13:58:37.641767 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:58:37.641745 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_aab13c90-c88f-42ce-83df-d29f7455f494/prom-label-proxy/0.log" Jul 05 13:58:37.666680 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:58:37.666651 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_aab13c90-c88f-42ce-83df-d29f7455f494/init-config-reloader/0.log" Jul 05 13:58:37.718023 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:58:37.717992 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-64786d9cbb-d8wzk_91869a1a-47e4-4b83-aac5-5c22528d0940/cluster-monitoring-operator/0.log" Jul 05 13:58:37.987021 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:58:37.986947 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-kghbq_c182307f-a4e2-4f81-97bc-fb7835a2a2ba/node-exporter/0.log" Jul 05 13:58:38.014484 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:58:38.014429 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-kghbq_c182307f-a4e2-4f81-97bc-fb7835a2a2ba/kube-rbac-proxy/0.log" Jul 05 13:58:38.041894 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:58:38.041869 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-kghbq_c182307f-a4e2-4f81-97bc-fb7835a2a2ba/init-textfile/0.log" Jul 05 13:58:38.434173 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:58:38.434141 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-567f7b8b7c-w4t2w_e45113b4-0332-41bf-9b40-5b3be8506ccc/prometheus-operator/0.log" Jul 05 13:58:38.454454 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:58:38.454429 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-567f7b8b7c-w4t2w_e45113b4-0332-41bf-9b40-5b3be8506ccc/kube-rbac-proxy/0.log" Jul 05 13:58:39.827213 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:58:39.827176 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-ks4vm/perf-node-gather-daemonset-v2lxd"] Jul 05 13:58:39.827687 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:58:39.827529 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="80f41632-fc45-460f-a5b8-3ff9e9dd9e50" containerName="maas-api" Jul 05 13:58:39.827687 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:58:39.827540 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="80f41632-fc45-460f-a5b8-3ff9e9dd9e50" containerName="maas-api" Jul 05 13:58:39.827687 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:58:39.827561 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="466d9763-b7d9-4e40-b241-f10b7b94c481" containerName="maas-api" Jul 05 13:58:39.827687 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:58:39.827566 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="466d9763-b7d9-4e40-b241-f10b7b94c481" containerName="maas-api" Jul 05 13:58:39.827687 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:58:39.827662 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="80f41632-fc45-460f-a5b8-3ff9e9dd9e50" containerName="maas-api" Jul 05 13:58:39.827687 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:58:39.827676 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="466d9763-b7d9-4e40-b241-f10b7b94c481" containerName="maas-api" Jul 05 13:58:39.831063 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:58:39.831044 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-ks4vm/perf-node-gather-daemonset-v2lxd" Jul 05 13:58:39.833939 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:58:39.833918 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-ks4vm\"/\"kube-root-ca.crt\"" Jul 05 13:58:39.834056 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:58:39.833966 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-ks4vm\"/\"openshift-service-ca.crt\"" Jul 05 13:58:39.835106 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:58:39.835088 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-ks4vm\"/\"default-dockercfg-q7ls5\"" Jul 05 13:58:39.838882 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:58:39.838860 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-ks4vm/perf-node-gather-daemonset-v2lxd"] Jul 05 13:58:39.881016 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:58:39.880989 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-6bffdf9778-ljfps_30164fef-3f92-4b9d-8c8f-ea7e6a3f1047/networking-console-plugin/0.log" Jul 05 13:58:39.935109 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:58:39.935073 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k78vl\" (UniqueName: \"kubernetes.io/projected/25d84f0f-003a-4909-bd6b-e45dae226e17-kube-api-access-k78vl\") pod \"perf-node-gather-daemonset-v2lxd\" (UID: \"25d84f0f-003a-4909-bd6b-e45dae226e17\") " pod="openshift-must-gather-ks4vm/perf-node-gather-daemonset-v2lxd" Jul 05 13:58:39.935290 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:58:39.935151 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/25d84f0f-003a-4909-bd6b-e45dae226e17-sys\") pod \"perf-node-gather-daemonset-v2lxd\" (UID: \"25d84f0f-003a-4909-bd6b-e45dae226e17\") " pod="openshift-must-gather-ks4vm/perf-node-gather-daemonset-v2lxd" Jul 05 13:58:39.935290 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:58:39.935170 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/25d84f0f-003a-4909-bd6b-e45dae226e17-lib-modules\") pod \"perf-node-gather-daemonset-v2lxd\" (UID: \"25d84f0f-003a-4909-bd6b-e45dae226e17\") " pod="openshift-must-gather-ks4vm/perf-node-gather-daemonset-v2lxd" Jul 05 13:58:39.935290 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:58:39.935189 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/25d84f0f-003a-4909-bd6b-e45dae226e17-podres\") pod \"perf-node-gather-daemonset-v2lxd\" (UID: \"25d84f0f-003a-4909-bd6b-e45dae226e17\") " pod="openshift-must-gather-ks4vm/perf-node-gather-daemonset-v2lxd" Jul 05 13:58:39.935290 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:58:39.935215 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/25d84f0f-003a-4909-bd6b-e45dae226e17-proc\") pod \"perf-node-gather-daemonset-v2lxd\" (UID: \"25d84f0f-003a-4909-bd6b-e45dae226e17\") " pod="openshift-must-gather-ks4vm/perf-node-gather-daemonset-v2lxd" Jul 05 13:58:40.036403 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:58:40.036364 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/25d84f0f-003a-4909-bd6b-e45dae226e17-sys\") pod \"perf-node-gather-daemonset-v2lxd\" (UID: \"25d84f0f-003a-4909-bd6b-e45dae226e17\") " pod="openshift-must-gather-ks4vm/perf-node-gather-daemonset-v2lxd" Jul 05 13:58:40.036403 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:58:40.036402 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/25d84f0f-003a-4909-bd6b-e45dae226e17-lib-modules\") pod \"perf-node-gather-daemonset-v2lxd\" (UID: \"25d84f0f-003a-4909-bd6b-e45dae226e17\") " pod="openshift-must-gather-ks4vm/perf-node-gather-daemonset-v2lxd" Jul 05 13:58:40.036722 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:58:40.036425 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/25d84f0f-003a-4909-bd6b-e45dae226e17-podres\") pod \"perf-node-gather-daemonset-v2lxd\" (UID: \"25d84f0f-003a-4909-bd6b-e45dae226e17\") " pod="openshift-must-gather-ks4vm/perf-node-gather-daemonset-v2lxd" Jul 05 13:58:40.036722 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:58:40.036442 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/25d84f0f-003a-4909-bd6b-e45dae226e17-proc\") pod \"perf-node-gather-daemonset-v2lxd\" (UID: \"25d84f0f-003a-4909-bd6b-e45dae226e17\") " pod="openshift-must-gather-ks4vm/perf-node-gather-daemonset-v2lxd" Jul 05 13:58:40.036722 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:58:40.036491 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/25d84f0f-003a-4909-bd6b-e45dae226e17-sys\") pod \"perf-node-gather-daemonset-v2lxd\" (UID: \"25d84f0f-003a-4909-bd6b-e45dae226e17\") " pod="openshift-must-gather-ks4vm/perf-node-gather-daemonset-v2lxd" Jul 05 13:58:40.036722 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:58:40.036542 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/25d84f0f-003a-4909-bd6b-e45dae226e17-proc\") pod \"perf-node-gather-daemonset-v2lxd\" (UID: \"25d84f0f-003a-4909-bd6b-e45dae226e17\") " pod="openshift-must-gather-ks4vm/perf-node-gather-daemonset-v2lxd" Jul 05 13:58:40.036722 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:58:40.036551 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/25d84f0f-003a-4909-bd6b-e45dae226e17-lib-modules\") pod \"perf-node-gather-daemonset-v2lxd\" (UID: \"25d84f0f-003a-4909-bd6b-e45dae226e17\") " pod="openshift-must-gather-ks4vm/perf-node-gather-daemonset-v2lxd" Jul 05 13:58:40.036722 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:58:40.036567 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/25d84f0f-003a-4909-bd6b-e45dae226e17-podres\") pod \"perf-node-gather-daemonset-v2lxd\" (UID: \"25d84f0f-003a-4909-bd6b-e45dae226e17\") " pod="openshift-must-gather-ks4vm/perf-node-gather-daemonset-v2lxd" Jul 05 13:58:40.036722 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:58:40.036675 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k78vl\" (UniqueName: \"kubernetes.io/projected/25d84f0f-003a-4909-bd6b-e45dae226e17-kube-api-access-k78vl\") pod \"perf-node-gather-daemonset-v2lxd\" (UID: \"25d84f0f-003a-4909-bd6b-e45dae226e17\") " pod="openshift-must-gather-ks4vm/perf-node-gather-daemonset-v2lxd" Jul 05 13:58:40.045366 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:58:40.045342 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k78vl\" (UniqueName: \"kubernetes.io/projected/25d84f0f-003a-4909-bd6b-e45dae226e17-kube-api-access-k78vl\") pod \"perf-node-gather-daemonset-v2lxd\" (UID: \"25d84f0f-003a-4909-bd6b-e45dae226e17\") " pod="openshift-must-gather-ks4vm/perf-node-gather-daemonset-v2lxd" Jul 05 13:58:40.142358 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:58:40.142251 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-ks4vm/perf-node-gather-daemonset-v2lxd" Jul 05 13:58:40.280957 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:58:40.280894 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-ks4vm/perf-node-gather-daemonset-v2lxd"] Jul 05 13:58:40.283741 ip-10-0-130-191 kubenswrapper[2578]: W0705 13:58:40.283711 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod25d84f0f_003a_4909_bd6b_e45dae226e17.slice/crio-6065877a1de20f58050a8c632190af7dbd445b0ff933f03e4acca231a313a33a WatchSource:0}: Error finding container 6065877a1de20f58050a8c632190af7dbd445b0ff933f03e4acca231a313a33a: Status 404 returned error can't find the container with id 6065877a1de20f58050a8c632190af7dbd445b0ff933f03e4acca231a313a33a Jul 05 13:58:40.525414 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:58:40.525381 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-tg2xx_1c214417-9bef-4007-ab01-e61b84d1b35e/console-operator/1.log" Jul 05 13:58:40.530186 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:58:40.530161 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-tg2xx_1c214417-9bef-4007-ab01-e61b84d1b35e/console-operator/2.log" Jul 05 13:58:40.606200 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:58:40.606162 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-ks4vm/perf-node-gather-daemonset-v2lxd" event={"ID":"25d84f0f-003a-4909-bd6b-e45dae226e17","Type":"ContainerStarted","Data":"60ee66a87862e9f9f351fe1133c80c350548697ad47f2a030b80fb299c19fd8b"} Jul 05 13:58:40.606413 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:58:40.606214 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-ks4vm/perf-node-gather-daemonset-v2lxd" event={"ID":"25d84f0f-003a-4909-bd6b-e45dae226e17","Type":"ContainerStarted","Data":"6065877a1de20f58050a8c632190af7dbd445b0ff933f03e4acca231a313a33a"} Jul 05 13:58:40.606413 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:58:40.606237 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-ks4vm/perf-node-gather-daemonset-v2lxd" Jul 05 13:58:40.625217 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:58:40.625168 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-ks4vm/perf-node-gather-daemonset-v2lxd" podStartSLOduration=1.625152003 podStartE2EDuration="1.625152003s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-05 13:58:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 13:58:40.622881703 +0000 UTC m=+2605.081444811" watchObservedRunningTime="2026-07-05 13:58:40.625152003 +0000 UTC m=+2605.083715108" Jul 05 13:58:40.971413 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:58:40.971326 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-d46699747-vtrh8_ce871812-196b-4bc0-ade3-1bbe1e9eafcc/console/0.log" Jul 05 13:58:41.596950 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:58:41.596915 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-5d456fdbcd-x8wmq_ff7bc8c4-d331-46ec-a64e-f32d6ded6d9c/volume-data-source-validator/0.log" Jul 05 13:58:42.703850 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:58:42.703821 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-w8bmv_79875c5a-6f9e-4d4f-9de9-5d974249f92b/dns/0.log" Jul 05 13:58:42.723772 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:58:42.723745 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-w8bmv_79875c5a-6f9e-4d4f-9de9-5d974249f92b/kube-rbac-proxy/0.log" Jul 05 13:58:42.746425 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:58:42.746401 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-btxlt_c01b0489-2ef4-4fbe-9f9e-1870bc580ee9/dns-node-resolver/0.log" Jul 05 13:58:43.390572 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:58:43.390546 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-g7nxb_b14dc74b-da46-4d14-8fd6-fd9357838fe2/node-ca/0.log" Jul 05 13:58:44.551090 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:58:44.551055 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7bcdf4856b-r2bgp_ceedc936-a8ca-4855-9484-675b40b24330/kube-auth-proxy/0.log" Jul 05 13:58:44.658652 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:58:44.658623 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-wn6j4_0e615cfd-cbda-41d6-af90-ad6958fa40c9/payload-pre-processing/0.log" Jul 05 13:58:45.468644 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:58:45.468610 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-qn45h_853f791d-d350-4626-989e-270b15b79453/serve-healthcheck-canary/0.log" Jul 05 13:58:45.935770 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:58:45.935738 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5c847dcfc4-7pd6v_beb3b580-1920-42d3-a152-ac42772476cf/insights-operator/0.log" Jul 05 13:58:45.936247 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:58:45.936191 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5c847dcfc4-7pd6v_beb3b580-1920-42d3-a152-ac42772476cf/insights-operator/1.log" Jul 05 13:58:46.031637 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:58:46.031573 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-djwgd_7bf56499-2b24-4899-9eab-8c809d555735/kube-rbac-proxy/0.log" Jul 05 13:58:46.052619 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:58:46.052568 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-djwgd_7bf56499-2b24-4899-9eab-8c809d555735/exporter/0.log" Jul 05 13:58:46.075093 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:58:46.075063 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-djwgd_7bf56499-2b24-4899-9eab-8c809d555735/extractor/0.log" Jul 05 13:58:46.621852 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:58:46.621819 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-ks4vm/perf-node-gather-daemonset-v2lxd" Jul 05 13:58:48.839935 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:58:48.839901 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-65c5985bdd-gfz4q_0fd9cbe7-4dfd-4e71-bc32-503f6360d12b/maas-api/0.log" Jul 05 13:58:48.862194 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:58:48.862167 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-key-cleanup-29720970-bbnmt_053537e5-cdbd-476d-a1d5-463b3c95548e/cleanup/0.log" Jul 05 13:58:49.211912 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:58:49.211886 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6674ff8b4f-rxh62_77458566-bec5-4550-8030-bf1f43f00555/manager/0.log" Jul 05 13:58:49.238370 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:58:49.238347 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6674ff8b4f-rxh62_77458566-bec5-4550-8030-bf1f43f00555/copy-manifests/0.log" Jul 05 13:58:50.508757 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:58:50.508725 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7bcdf4856b-r2bgp_ceedc936-a8ca-4855-9484-675b40b24330/kube-auth-proxy/0.log" Jul 05 13:58:50.599919 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:58:50.599880 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-wn6j4_0e615cfd-cbda-41d6-af90-ad6958fa40c9/payload-pre-processing/0.log" Jul 05 13:58:50.641880 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:58:50.641852 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-687b6f4bf4-g56mp_e35137df-801a-4327-8fb7-54eb69d4a72d/manager/0.log" Jul 05 13:58:52.168059 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:58:52.168033 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7bcdf4856b-r2bgp_ceedc936-a8ca-4855-9484-675b40b24330/kube-auth-proxy/0.log" Jul 05 13:58:52.252710 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:58:52.252679 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-wn6j4_0e615cfd-cbda-41d6-af90-ad6958fa40c9/payload-pre-processing/0.log" Jul 05 13:58:55.723239 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:58:55.723211 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-d57d58cc9-4thc8_7f87b4fa-b050-448a-a65e-883ab774a48c/migrator/0.log" Jul 05 13:58:55.745024 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:58:55.744996 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-d57d58cc9-4thc8_7f87b4fa-b050-448a-a65e-883ab774a48c/graceful-termination/0.log" Jul 05 13:58:56.163575 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:58:56.163540 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-846bf668f7-wgnw7_b7b1094c-0b91-4c45-aec8-2c0e2d8aed07/kube-storage-version-migrator-operator/1.log" Jul 05 13:58:56.164432 ip-10-0-130-191 kubenswrapper[2578]: I0705 13:58:56.164415 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-846bf668f7-wgnw7_b7b1094c-0b91-4c45-aec8-2c0e2d8aed07/kube-storage-version-migrator-operator/0.log"