Jul 07 14:24:13.956246 ip-10-0-134-42 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Jul 07 14:24:13.956257 ip-10-0-134-42 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Jul 07 14:24:13.956264 ip-10-0-134-42 systemd[1]: kubelet.service: Failed with result 'resources'. Jul 07 14:24:13.956478 ip-10-0-134-42 systemd[1]: Failed to start Kubernetes Kubelet. Jul 07 14:24:25.261761 ip-10-0-134-42 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found. Jul 07 14:24:25.261781 ip-10-0-134-42 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot 83ec6f7d6300481cb2793491ed065824 -- Jul 07 14:26:03.134751 ip-10-0-134-42 systemd[1]: Starting Kubernetes Kubelet... Jul 07 14:26:03.617774 ip-10-0-134-42 kubenswrapper[2557]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 07 14:26:03.617774 ip-10-0-134-42 kubenswrapper[2557]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 07 14:26:03.617774 ip-10-0-134-42 kubenswrapper[2557]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 07 14:26:03.617774 ip-10-0-134-42 kubenswrapper[2557]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 07 14:26:03.617774 ip-10-0-134-42 kubenswrapper[2557]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 07 14:26:03.617774 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:03.602278 2557 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 07 14:26:03.617774 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.609564 2557 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 07 14:26:03.617774 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.609575 2557 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 07 14:26:03.617774 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.609580 2557 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 07 14:26:03.617774 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.609583 2557 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 07 14:26:03.617774 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.609586 2557 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 07 14:26:03.617774 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.609589 2557 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 07 14:26:03.617774 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.609592 2557 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 07 14:26:03.617774 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.609594 2557 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 07 14:26:03.633942 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.609597 2557 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 07 14:26:03.633942 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.609599 2557 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 07 14:26:03.633942 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.609602 2557 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 07 14:26:03.633942 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.609605 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 07 14:26:03.633942 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.609607 2557 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 07 14:26:03.633942 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.609610 2557 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 07 14:26:03.633942 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.609612 2557 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 07 14:26:03.633942 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.609615 2557 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 07 14:26:03.633942 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.609618 2557 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 07 14:26:03.633942 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.609620 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 07 14:26:03.633942 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.609622 2557 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 07 14:26:03.633942 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.609625 2557 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 07 14:26:03.633942 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.609637 2557 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 07 14:26:03.633942 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.609640 2557 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 07 14:26:03.633942 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.609643 2557 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 07 14:26:03.633942 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.609645 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 07 14:26:03.633942 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.609648 2557 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 07 14:26:03.633942 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.609651 2557 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 07 14:26:03.633942 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.609653 2557 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 07 14:26:03.633942 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.609656 2557 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 07 14:26:03.637206 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.609658 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 07 14:26:03.637206 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.609661 2557 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 07 14:26:03.637206 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.609664 2557 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 07 14:26:03.637206 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.609666 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 07 14:26:03.637206 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.609669 2557 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 07 14:26:03.637206 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.609671 2557 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 07 14:26:03.637206 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.609674 2557 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 07 14:26:03.637206 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.609676 2557 feature_gate.go:328] unrecognized feature gate: Example2 Jul 07 14:26:03.637206 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.609679 2557 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 07 14:26:03.637206 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.609682 2557 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 07 14:26:03.637206 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.609685 2557 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 07 14:26:03.637206 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.609688 2557 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 07 14:26:03.637206 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.609690 2557 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 07 14:26:03.637206 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.609693 2557 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 07 14:26:03.637206 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.609695 2557 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 07 14:26:03.637206 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.609698 2557 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 07 14:26:03.637206 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.609700 2557 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 07 14:26:03.637206 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.609704 2557 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 07 14:26:03.637206 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.609708 2557 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 07 14:26:03.637206 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.609711 2557 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 07 14:26:03.650678 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.609713 2557 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 07 14:26:03.650678 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.609716 2557 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 07 14:26:03.650678 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.609720 2557 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 07 14:26:03.650678 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.609722 2557 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 07 14:26:03.650678 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.609725 2557 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 07 14:26:03.650678 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.609728 2557 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 07 14:26:03.650678 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.609732 2557 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 07 14:26:03.650678 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.609734 2557 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 07 14:26:03.650678 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.609737 2557 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 07 14:26:03.650678 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.609739 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 07 14:26:03.650678 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.609742 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 07 14:26:03.650678 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.609744 2557 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 07 14:26:03.650678 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.609747 2557 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 07 14:26:03.650678 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.609751 2557 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 07 14:26:03.650678 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.609755 2557 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 07 14:26:03.650678 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.609757 2557 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 07 14:26:03.650678 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.609760 2557 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 07 14:26:03.650678 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.609762 2557 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 07 14:26:03.650678 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.609765 2557 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 07 14:26:03.652374 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.609767 2557 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 07 14:26:03.652374 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.609770 2557 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 07 14:26:03.652374 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.609772 2557 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 07 14:26:03.652374 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.609775 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 07 14:26:03.652374 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.609778 2557 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 07 14:26:03.652374 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.609782 2557 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 07 14:26:03.652374 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.609784 2557 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 07 14:26:03.652374 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.609787 2557 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 07 14:26:03.652374 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.609789 2557 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 07 14:26:03.652374 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.609792 2557 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 07 14:26:03.652374 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.609795 2557 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 07 14:26:03.652374 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.609798 2557 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 07 14:26:03.652374 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.609801 2557 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 07 14:26:03.652374 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.609803 2557 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 07 14:26:03.652374 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.609806 2557 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 07 14:26:03.652374 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.609809 2557 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 07 14:26:03.652374 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.609811 2557 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 07 14:26:03.652374 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.609814 2557 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 07 14:26:03.652374 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.609816 2557 feature_gate.go:328] unrecognized feature gate: Example Jul 07 14:26:03.652374 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.609820 2557 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 07 14:26:03.665924 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.609823 2557 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 07 14:26:03.665924 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.609827 2557 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 07 14:26:03.665924 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.609830 2557 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 07 14:26:03.665924 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.609832 2557 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 07 14:26:03.665924 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.609835 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 07 14:26:03.665924 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.609838 2557 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 07 14:26:03.665924 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.609840 2557 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 07 14:26:03.665924 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.609843 2557 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 07 14:26:03.665924 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.609845 2557 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 07 14:26:03.665924 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.609847 2557 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 07 14:26:03.665924 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.609850 2557 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 07 14:26:03.665924 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.609852 2557 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 07 14:26:03.665924 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.610982 2557 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 07 14:26:03.665924 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.610995 2557 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 07 14:26:03.665924 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.611000 2557 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 07 14:26:03.665924 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.611005 2557 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 07 14:26:03.665924 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.611009 2557 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 07 14:26:03.665924 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.611014 2557 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 07 14:26:03.665924 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.611018 2557 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 07 14:26:03.669107 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.611022 2557 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 07 14:26:03.669107 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.611027 2557 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 07 14:26:03.669107 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.611032 2557 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 07 14:26:03.669107 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.611042 2557 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 07 14:26:03.669107 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.611046 2557 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 07 14:26:03.669107 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.611050 2557 feature_gate.go:328] unrecognized feature gate: Example2 Jul 07 14:26:03.669107 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.611055 2557 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 07 14:26:03.669107 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.611059 2557 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 07 14:26:03.669107 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.611063 2557 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 07 14:26:03.669107 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.611067 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 07 14:26:03.669107 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.611071 2557 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 07 14:26:03.669107 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.611075 2557 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 07 14:26:03.669107 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.611079 2557 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 07 14:26:03.669107 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.611084 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 07 14:26:03.669107 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.611089 2557 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 07 14:26:03.669107 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.611093 2557 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 07 14:26:03.669107 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.611101 2557 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 07 14:26:03.669107 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.611105 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 07 14:26:03.669107 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.611110 2557 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 07 14:26:03.684156 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.611114 2557 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 07 14:26:03.684156 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.611118 2557 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 07 14:26:03.684156 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.611123 2557 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 07 14:26:03.684156 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.611126 2557 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 07 14:26:03.684156 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.611130 2557 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 07 14:26:03.684156 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.611134 2557 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 07 14:26:03.684156 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.611138 2557 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 07 14:26:03.684156 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.611141 2557 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 07 14:26:03.684156 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.611150 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 07 14:26:03.684156 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.611154 2557 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 07 14:26:03.684156 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.611162 2557 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 07 14:26:03.684156 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.611273 2557 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 07 14:26:03.684156 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.611281 2557 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 07 14:26:03.684156 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.611287 2557 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 07 14:26:03.684156 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.611292 2557 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 07 14:26:03.684156 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.611297 2557 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 07 14:26:03.684156 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.611303 2557 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 07 14:26:03.684156 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.611308 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 07 14:26:03.684156 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.611313 2557 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 07 14:26:03.684156 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.611318 2557 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 07 14:26:03.675557 ip-10-0-134-42 systemd[1]: Started Kubernetes Kubelet. Jul 07 14:26:03.930697 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.611323 2557 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 07 14:26:03.930697 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.611327 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 07 14:26:03.930697 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.611338 2557 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 07 14:26:03.930697 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.611342 2557 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 07 14:26:03.930697 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.611347 2557 feature_gate.go:328] unrecognized feature gate: Example Jul 07 14:26:03.930697 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.611351 2557 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 07 14:26:03.930697 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.611355 2557 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 07 14:26:03.930697 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.611359 2557 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 07 14:26:03.930697 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.611367 2557 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 07 14:26:03.930697 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.611374 2557 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 07 14:26:03.930697 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.611382 2557 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 07 14:26:03.930697 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.611387 2557 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 07 14:26:03.930697 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.611392 2557 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 07 14:26:03.930697 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.611396 2557 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 07 14:26:03.930697 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.611405 2557 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 07 14:26:03.930697 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.611409 2557 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 07 14:26:03.930697 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.611413 2557 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 07 14:26:03.930697 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.611418 2557 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 07 14:26:03.930697 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.611424 2557 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 07 14:26:03.943902 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.611429 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 07 14:26:03.943902 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.611433 2557 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 07 14:26:03.943902 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.611437 2557 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 07 14:26:03.943902 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.611441 2557 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 07 14:26:03.943902 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.611445 2557 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 07 14:26:03.943902 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.611451 2557 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 07 14:26:03.943902 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.611455 2557 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 07 14:26:03.943902 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.611464 2557 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 07 14:26:03.943902 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.611468 2557 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 07 14:26:03.943902 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.611472 2557 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 07 14:26:03.943902 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.611476 2557 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 07 14:26:03.943902 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.611480 2557 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 07 14:26:03.943902 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.611484 2557 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 07 14:26:03.943902 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.611492 2557 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 07 14:26:03.943902 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.611497 2557 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 07 14:26:03.943902 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.611504 2557 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 07 14:26:03.943902 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.611510 2557 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 07 14:26:03.943902 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.611529 2557 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 07 14:26:03.943902 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.611539 2557 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 07 14:26:03.950229 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.611543 2557 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 07 14:26:03.950229 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.611548 2557 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 07 14:26:03.950229 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.611553 2557 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 07 14:26:03.950229 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.611558 2557 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 07 14:26:03.950229 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.611562 2557 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 07 14:26:03.950229 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.611566 2557 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 07 14:26:03.950229 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.611570 2557 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 07 14:26:03.950229 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.611575 2557 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 07 14:26:03.950229 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.611579 2557 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 07 14:26:03.950229 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.611583 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 07 14:26:03.950229 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.611586 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 07 14:26:03.950229 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.611589 2557 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 07 14:26:03.950229 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.611594 2557 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 07 14:26:03.950229 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.611597 2557 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 07 14:26:03.950229 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.611600 2557 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 07 14:26:03.950229 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.612141 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 07 14:26:03.950229 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.612150 2557 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 07 14:26:03.950229 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.612153 2557 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 07 14:26:03.950229 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.612157 2557 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 07 14:26:03.950229 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.612159 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 07 14:26:03.950915 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.612162 2557 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 07 14:26:03.950915 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.612165 2557 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 07 14:26:03.950915 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.612167 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 07 14:26:03.950915 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.612170 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 07 14:26:03.950915 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.612173 2557 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 07 14:26:03.950915 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.612175 2557 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 07 14:26:03.950915 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.612178 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 07 14:26:03.950915 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.612180 2557 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 07 14:26:03.950915 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.612183 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 07 14:26:03.950915 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.612185 2557 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 07 14:26:03.950915 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.612188 2557 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 07 14:26:03.950915 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.612190 2557 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 07 14:26:03.950915 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.612193 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 07 14:26:03.950915 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.612195 2557 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 07 14:26:03.950915 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.612198 2557 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 07 14:26:03.950915 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.612200 2557 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 07 14:26:03.950915 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.612203 2557 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 07 14:26:03.950915 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.612205 2557 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 07 14:26:03.950915 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.612209 2557 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 07 14:26:03.951458 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.612213 2557 feature_gate.go:328] unrecognized feature gate: Example2 Jul 07 14:26:03.951458 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.612216 2557 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 07 14:26:03.951458 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.612219 2557 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 07 14:26:03.951458 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.612221 2557 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 07 14:26:03.951458 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.612224 2557 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 07 14:26:03.951458 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.612226 2557 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 07 14:26:03.951458 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.612229 2557 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 07 14:26:03.951458 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.612232 2557 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 07 14:26:03.951458 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.612235 2557 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 07 14:26:03.951458 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.612237 2557 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 07 14:26:03.951458 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.612240 2557 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 07 14:26:03.951458 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.612242 2557 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 07 14:26:03.951458 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.612245 2557 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 07 14:26:03.951458 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.612247 2557 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 07 14:26:03.951458 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.612250 2557 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 07 14:26:03.951458 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.612253 2557 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 07 14:26:03.951458 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.612257 2557 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 07 14:26:03.951458 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.612260 2557 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 07 14:26:03.951458 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.612262 2557 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 07 14:26:03.952076 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.612265 2557 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 07 14:26:03.952076 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.612267 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 07 14:26:03.952076 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.612269 2557 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 07 14:26:03.952076 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.612272 2557 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 07 14:26:03.952076 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.612274 2557 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 07 14:26:03.952076 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.612277 2557 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 07 14:26:03.952076 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.612279 2557 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 07 14:26:03.952076 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.612282 2557 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 07 14:26:03.952076 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.612286 2557 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 07 14:26:03.952076 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.612289 2557 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 07 14:26:03.952076 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.612292 2557 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 07 14:26:03.952076 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.612294 2557 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 07 14:26:03.952076 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.612297 2557 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 07 14:26:03.952076 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.612299 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 07 14:26:03.952076 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.612304 2557 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 07 14:26:03.952076 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.612307 2557 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 07 14:26:03.952076 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.612310 2557 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 07 14:26:03.952076 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.612312 2557 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 07 14:26:03.952076 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.612315 2557 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 07 14:26:03.952076 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.612317 2557 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 07 14:26:03.952656 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.612320 2557 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 07 14:26:03.952656 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.612322 2557 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 07 14:26:03.952656 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.612325 2557 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 07 14:26:03.952656 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.612327 2557 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 07 14:26:03.952656 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.612329 2557 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 07 14:26:03.952656 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.612332 2557 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 07 14:26:03.952656 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.612335 2557 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 07 14:26:03.952656 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.612337 2557 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 07 14:26:03.952656 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.612340 2557 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 07 14:26:03.952656 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.612343 2557 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 07 14:26:03.952656 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.612346 2557 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 07 14:26:03.952656 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.612348 2557 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 07 14:26:03.952656 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.612351 2557 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 07 14:26:03.952656 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.612354 2557 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 07 14:26:03.952656 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.612356 2557 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 07 14:26:03.952656 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.612358 2557 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 07 14:26:03.952656 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.612361 2557 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 07 14:26:03.952656 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.612363 2557 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 07 14:26:03.952656 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.612365 2557 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 07 14:26:03.953280 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.612368 2557 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 07 14:26:03.953280 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.612370 2557 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 07 14:26:03.953280 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.612373 2557 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 07 14:26:03.953280 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.612375 2557 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 07 14:26:03.953280 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.612378 2557 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 07 14:26:03.953280 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.612380 2557 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 07 14:26:03.953280 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.612383 2557 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 07 14:26:03.953280 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.612387 2557 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 07 14:26:03.953280 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.612389 2557 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 07 14:26:03.953280 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.612392 2557 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 07 14:26:03.953280 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.612394 2557 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 07 14:26:03.953280 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.612397 2557 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 07 14:26:03.953280 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.612399 2557 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 07 14:26:03.953280 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.612402 2557 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 07 14:26:03.953280 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.612404 2557 feature_gate.go:328] unrecognized feature gate: Example Jul 07 14:26:03.953280 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.612407 2557 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 07 14:26:03.953280 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.612409 2557 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 07 14:26:03.953280 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:03.612474 2557 flags.go:64] FLAG: --address="0.0.0.0" Jul 07 14:26:03.953280 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:03.612481 2557 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 07 14:26:03.953280 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:03.613076 2557 flags.go:64] FLAG: --anonymous-auth="true" Jul 07 14:26:03.953280 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:03.613084 2557 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 07 14:26:03.953280 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:03.613089 2557 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 07 14:26:03.953930 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:03.613092 2557 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 07 14:26:03.953930 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:03.613096 2557 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 07 14:26:03.953930 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:03.613101 2557 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 07 14:26:03.953930 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:03.613104 2557 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 07 14:26:03.953930 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:03.613107 2557 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 07 14:26:03.953930 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:03.613110 2557 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 07 14:26:03.953930 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:03.613114 2557 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 07 14:26:03.953930 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:03.613117 2557 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 07 14:26:03.953930 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:03.613119 2557 flags.go:64] FLAG: --cgroup-root="" Jul 07 14:26:03.953930 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:03.613122 2557 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 07 14:26:03.953930 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:03.613125 2557 flags.go:64] FLAG: --client-ca-file="" Jul 07 14:26:03.953930 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:03.613129 2557 flags.go:64] FLAG: --cloud-provider="external" Jul 07 14:26:03.953930 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:03.613132 2557 flags.go:64] FLAG: --cluster-dns="[]" Jul 07 14:26:03.953930 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:03.613138 2557 flags.go:64] FLAG: --cluster-domain="" Jul 07 14:26:03.953930 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:03.613140 2557 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 07 14:26:03.953930 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:03.613144 2557 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 07 14:26:03.953930 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:03.613147 2557 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 07 14:26:03.953930 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:03.613150 2557 flags.go:64] FLAG: --container-log-max-files="5" Jul 07 14:26:03.953930 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:03.613154 2557 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 07 14:26:03.953930 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:03.613157 2557 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 07 14:26:03.953930 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:03.613160 2557 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 07 14:26:03.953930 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:03.613163 2557 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 07 14:26:03.953930 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:03.613166 2557 flags.go:64] FLAG: --contention-profiling="false" Jul 07 14:26:03.953930 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:03.613170 2557 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 07 14:26:03.954643 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:03.613173 2557 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 07 14:26:03.954643 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:03.613176 2557 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 07 14:26:03.954643 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:03.613179 2557 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 07 14:26:03.954643 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:03.613183 2557 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 07 14:26:03.954643 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:03.613188 2557 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 07 14:26:03.954643 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:03.613191 2557 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 07 14:26:03.954643 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:03.613194 2557 flags.go:64] FLAG: --enable-load-reader="false" Jul 07 14:26:03.954643 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:03.613197 2557 flags.go:64] FLAG: --enable-server="true" Jul 07 14:26:03.954643 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:03.613199 2557 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 07 14:26:03.954643 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:03.613204 2557 flags.go:64] FLAG: --event-burst="100" Jul 07 14:26:03.954643 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:03.613207 2557 flags.go:64] FLAG: --event-qps="50" Jul 07 14:26:03.954643 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:03.613210 2557 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 07 14:26:03.954643 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:03.613213 2557 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 07 14:26:03.954643 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:03.613216 2557 flags.go:64] FLAG: --eviction-hard="" Jul 07 14:26:03.954643 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:03.613219 2557 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 07 14:26:03.954643 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:03.613222 2557 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 07 14:26:03.954643 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:03.613225 2557 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 07 14:26:03.954643 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:03.613228 2557 flags.go:64] FLAG: --eviction-soft="" Jul 07 14:26:03.954643 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:03.613231 2557 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 07 14:26:03.954643 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:03.613234 2557 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 07 14:26:03.954643 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:03.613238 2557 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 07 14:26:03.954643 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:03.613241 2557 flags.go:64] FLAG: --experimental-mounter-path="" Jul 07 14:26:03.954643 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:03.613244 2557 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 07 14:26:03.954643 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:03.613246 2557 flags.go:64] FLAG: --fail-swap-on="true" Jul 07 14:26:03.954643 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:03.613249 2557 flags.go:64] FLAG: --feature-gates="" Jul 07 14:26:03.955337 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:03.613253 2557 flags.go:64] FLAG: --file-check-frequency="20s" Jul 07 14:26:03.955337 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:03.613256 2557 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 07 14:26:03.955337 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:03.613259 2557 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 07 14:26:03.955337 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:03.613262 2557 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 07 14:26:03.955337 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:03.613265 2557 flags.go:64] FLAG: --healthz-port="10248" Jul 07 14:26:03.955337 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:03.613268 2557 flags.go:64] FLAG: --help="false" Jul 07 14:26:03.955337 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:03.613271 2557 flags.go:64] FLAG: --hostname-override="ip-10-0-134-42.ec2.internal" Jul 07 14:26:03.955337 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:03.613274 2557 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 07 14:26:03.955337 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:03.613277 2557 flags.go:64] FLAG: --http-check-frequency="20s" Jul 07 14:26:03.955337 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:03.613279 2557 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 07 14:26:03.955337 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:03.613283 2557 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 07 14:26:03.955337 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:03.613287 2557 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 07 14:26:03.955337 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:03.613290 2557 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 07 14:26:03.955337 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:03.613293 2557 flags.go:64] FLAG: --image-service-endpoint="" Jul 07 14:26:03.955337 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:03.613296 2557 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 07 14:26:03.955337 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:03.613299 2557 flags.go:64] FLAG: --kube-api-burst="100" Jul 07 14:26:03.955337 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:03.613302 2557 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 07 14:26:03.955337 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:03.613305 2557 flags.go:64] FLAG: --kube-api-qps="50" Jul 07 14:26:03.955337 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:03.613308 2557 flags.go:64] FLAG: --kube-reserved="" Jul 07 14:26:03.955337 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:03.613310 2557 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 07 14:26:03.955337 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:03.613313 2557 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 07 14:26:03.955337 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:03.613317 2557 flags.go:64] FLAG: --kubelet-cgroups="" Jul 07 14:26:03.955337 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:03.613319 2557 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 07 14:26:03.955337 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:03.613322 2557 flags.go:64] FLAG: --lock-file="" Jul 07 14:26:03.956043 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:03.613325 2557 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 07 14:26:03.956043 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:03.613329 2557 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 07 14:26:03.956043 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:03.613332 2557 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 07 14:26:03.956043 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:03.613337 2557 flags.go:64] FLAG: --log-json-split-stream="false" Jul 07 14:26:03.956043 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:03.613340 2557 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 07 14:26:03.956043 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:03.613343 2557 flags.go:64] FLAG: --log-text-split-stream="false" Jul 07 14:26:03.956043 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:03.613346 2557 flags.go:64] FLAG: --logging-format="text" Jul 07 14:26:03.956043 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:03.613349 2557 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 07 14:26:03.956043 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:03.613352 2557 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 07 14:26:03.956043 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:03.613355 2557 flags.go:64] FLAG: --manifest-url="" Jul 07 14:26:03.956043 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:03.613357 2557 flags.go:64] FLAG: --manifest-url-header="" Jul 07 14:26:03.956043 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:03.613361 2557 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 07 14:26:03.956043 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:03.613364 2557 flags.go:64] FLAG: --max-open-files="1000000" Jul 07 14:26:03.956043 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:03.613368 2557 flags.go:64] FLAG: --max-pods="110" Jul 07 14:26:03.956043 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:03.613371 2557 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 07 14:26:03.956043 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:03.613374 2557 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 07 14:26:03.956043 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:03.613376 2557 flags.go:64] FLAG: --memory-manager-policy="None" Jul 07 14:26:03.956043 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:03.613379 2557 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 07 14:26:03.956043 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:03.613382 2557 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 07 14:26:03.956043 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:03.613386 2557 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 07 14:26:03.956043 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:03.613389 2557 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 07 14:26:03.956043 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:03.613396 2557 flags.go:64] FLAG: --node-status-max-images="50" Jul 07 14:26:03.956043 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:03.613399 2557 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 07 14:26:03.956043 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:03.613402 2557 flags.go:64] FLAG: --oom-score-adj="-999" Jul 07 14:26:03.956043 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:03.613405 2557 flags.go:64] FLAG: --pod-cidr="" Jul 07 14:26:03.956743 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:03.613408 2557 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ce93f6b4197b9c62a26a3131f8057ffe9c9379c5079a4f62e5919b75151394e5" Jul 07 14:26:03.956743 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:03.613413 2557 flags.go:64] FLAG: --pod-manifest-path="" Jul 07 14:26:03.956743 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:03.613417 2557 flags.go:64] FLAG: --pod-max-pids="-1" Jul 07 14:26:03.956743 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:03.613420 2557 flags.go:64] FLAG: --pods-per-core="0" Jul 07 14:26:03.956743 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:03.613422 2557 flags.go:64] FLAG: --port="10250" Jul 07 14:26:03.956743 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:03.613425 2557 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 07 14:26:03.956743 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:03.613428 2557 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0c662a992e79003fd" Jul 07 14:26:03.956743 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:03.613431 2557 flags.go:64] FLAG: --qos-reserved="" Jul 07 14:26:03.956743 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:03.613436 2557 flags.go:64] FLAG: --read-only-port="10255" Jul 07 14:26:03.956743 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:03.613438 2557 flags.go:64] FLAG: --register-node="true" Jul 07 14:26:03.956743 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:03.613441 2557 flags.go:64] FLAG: --register-with-taints="" Jul 07 14:26:03.956743 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:03.613445 2557 flags.go:64] FLAG: --registry-burst="10" Jul 07 14:26:03.956743 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:03.613448 2557 flags.go:64] FLAG: --registry-qps="5" Jul 07 14:26:03.956743 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:03.613451 2557 flags.go:64] FLAG: --reserved-cpus="" Jul 07 14:26:03.956743 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:03.613454 2557 flags.go:64] FLAG: --reserved-memory="" Jul 07 14:26:03.956743 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:03.613460 2557 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 07 14:26:03.956743 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:03.613463 2557 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 07 14:26:03.956743 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:03.613466 2557 flags.go:64] FLAG: --rotate-certificates="false" Jul 07 14:26:03.956743 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:03.613469 2557 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 07 14:26:03.956743 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:03.613472 2557 flags.go:64] FLAG: --runonce="false" Jul 07 14:26:03.956743 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:03.613474 2557 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 07 14:26:03.956743 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:03.613477 2557 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 07 14:26:03.956743 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:03.613480 2557 flags.go:64] FLAG: --seccomp-default="false" Jul 07 14:26:03.956743 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:03.613483 2557 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 07 14:26:03.956743 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:03.613486 2557 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 07 14:26:03.957419 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:03.613489 2557 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 07 14:26:03.957419 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:03.613491 2557 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 07 14:26:03.957419 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:03.613496 2557 flags.go:64] FLAG: --storage-driver-password="root" Jul 07 14:26:03.957419 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:03.613498 2557 flags.go:64] FLAG: --storage-driver-secure="false" Jul 07 14:26:03.957419 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:03.613501 2557 flags.go:64] FLAG: --storage-driver-table="stats" Jul 07 14:26:03.957419 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:03.613504 2557 flags.go:64] FLAG: --storage-driver-user="root" Jul 07 14:26:03.957419 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:03.613506 2557 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 07 14:26:03.957419 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:03.613510 2557 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 07 14:26:03.957419 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:03.613512 2557 flags.go:64] FLAG: --system-cgroups="" Jul 07 14:26:03.957419 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:03.613530 2557 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 07 14:26:03.957419 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:03.613535 2557 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 07 14:26:03.957419 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:03.613538 2557 flags.go:64] FLAG: --tls-cert-file="" Jul 07 14:26:03.957419 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:03.613541 2557 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 07 14:26:03.957419 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:03.613546 2557 flags.go:64] FLAG: --tls-min-version="" Jul 07 14:26:03.957419 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:03.613549 2557 flags.go:64] FLAG: --tls-private-key-file="" Jul 07 14:26:03.957419 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:03.613553 2557 flags.go:64] FLAG: --topology-manager-policy="none" Jul 07 14:26:03.957419 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:03.613555 2557 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 07 14:26:03.957419 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:03.613558 2557 flags.go:64] FLAG: --topology-manager-scope="container" Jul 07 14:26:03.957419 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:03.613561 2557 flags.go:64] FLAG: --v="2" Jul 07 14:26:03.957419 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:03.613565 2557 flags.go:64] FLAG: --version="false" Jul 07 14:26:03.957419 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:03.613570 2557 flags.go:64] FLAG: --vmodule="" Jul 07 14:26:03.957419 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:03.613574 2557 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 07 14:26:03.957419 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:03.613577 2557 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 07 14:26:03.957419 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.613693 2557 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 07 14:26:03.957419 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.613696 2557 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 07 14:26:03.958070 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.613700 2557 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 07 14:26:03.958070 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.613703 2557 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 07 14:26:03.958070 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.613705 2557 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 07 14:26:03.958070 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.613708 2557 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 07 14:26:03.958070 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.613711 2557 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 07 14:26:03.958070 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.613714 2557 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 07 14:26:03.958070 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.613717 2557 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 07 14:26:03.958070 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.613719 2557 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 07 14:26:03.958070 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.613721 2557 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 07 14:26:03.958070 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.613725 2557 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 07 14:26:03.958070 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.613727 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 07 14:26:03.958070 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.613730 2557 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 07 14:26:03.958070 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.613732 2557 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 07 14:26:03.958070 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.613735 2557 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 07 14:26:03.958070 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.613737 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 07 14:26:03.958070 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.613740 2557 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 07 14:26:03.958070 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.613743 2557 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 07 14:26:03.958070 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.613745 2557 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 07 14:26:03.958070 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.613748 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 07 14:26:03.958070 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.613750 2557 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 07 14:26:03.958643 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.613753 2557 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 07 14:26:03.958643 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.613755 2557 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 07 14:26:03.958643 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.613759 2557 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 07 14:26:03.958643 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.613762 2557 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 07 14:26:03.958643 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.613764 2557 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 07 14:26:03.958643 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.613767 2557 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 07 14:26:03.958643 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.613769 2557 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 07 14:26:03.958643 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.613772 2557 feature_gate.go:328] unrecognized feature gate: Example2 Jul 07 14:26:03.958643 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.613776 2557 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 07 14:26:03.958643 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.613778 2557 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 07 14:26:03.958643 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.613781 2557 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 07 14:26:03.958643 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.613785 2557 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 07 14:26:03.958643 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.613788 2557 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 07 14:26:03.958643 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.613791 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 07 14:26:03.958643 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.613794 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 07 14:26:03.958643 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.613796 2557 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 07 14:26:03.958643 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.613799 2557 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 07 14:26:03.958643 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.613801 2557 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 07 14:26:03.958643 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.613804 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 07 14:26:03.959124 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.613806 2557 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 07 14:26:03.959124 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.613808 2557 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 07 14:26:03.959124 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.613812 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 07 14:26:03.959124 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.613815 2557 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 07 14:26:03.959124 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.613817 2557 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 07 14:26:03.959124 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.613820 2557 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 07 14:26:03.959124 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.613823 2557 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 07 14:26:03.959124 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.613826 2557 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 07 14:26:03.959124 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.613828 2557 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 07 14:26:03.959124 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.613832 2557 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 07 14:26:03.959124 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.613834 2557 feature_gate.go:328] unrecognized feature gate: Example Jul 07 14:26:03.959124 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.613836 2557 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 07 14:26:03.959124 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.613839 2557 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 07 14:26:03.959124 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.613841 2557 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 07 14:26:03.959124 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.613844 2557 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 07 14:26:03.959124 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.613850 2557 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 07 14:26:03.959124 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.613853 2557 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 07 14:26:03.959124 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.613855 2557 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 07 14:26:03.959124 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.613858 2557 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 07 14:26:03.959124 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.613860 2557 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 07 14:26:03.959687 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.613863 2557 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 07 14:26:03.959687 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.613865 2557 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 07 14:26:03.959687 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.613868 2557 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 07 14:26:03.959687 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.613870 2557 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 07 14:26:03.959687 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.613873 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 07 14:26:03.959687 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.613876 2557 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 07 14:26:03.959687 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.613878 2557 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 07 14:26:03.959687 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.613880 2557 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 07 14:26:03.959687 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.613883 2557 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 07 14:26:03.959687 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.613885 2557 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 07 14:26:03.959687 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.613888 2557 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 07 14:26:03.959687 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.613890 2557 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 07 14:26:03.959687 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.613892 2557 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 07 14:26:03.959687 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.613895 2557 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 07 14:26:03.959687 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.613898 2557 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 07 14:26:03.959687 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.613901 2557 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 07 14:26:03.959687 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.613903 2557 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 07 14:26:03.959687 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.613906 2557 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 07 14:26:03.959687 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.613908 2557 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 07 14:26:03.959687 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.613911 2557 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 07 14:26:04.122443 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.613913 2557 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 07 14:26:04.122443 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.613916 2557 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 07 14:26:04.122443 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.613918 2557 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 07 14:26:04.122443 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.613920 2557 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 07 14:26:04.122443 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.613923 2557 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 07 14:26:04.122443 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.613926 2557 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 07 14:26:04.122443 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.613928 2557 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 07 14:26:04.122443 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.613932 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 07 14:26:04.122443 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.613934 2557 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 07 14:26:04.122443 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.613937 2557 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 07 14:26:04.122443 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.613939 2557 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 07 14:26:04.122443 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.613941 2557 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 07 14:26:04.122443 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.613945 2557 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 07 14:26:04.122443 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.613948 2557 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 07 14:26:04.122443 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.613951 2557 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 07 14:26:04.122443 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.613955 2557 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 07 14:26:04.122443 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.613959 2557 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 07 14:26:04.122443 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.613961 2557 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 07 14:26:04.123212 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:03.614541 2557 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 07 14:26:04.123212 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:03.623716 2557 server.go:529] "Kubelet version" kubeletVersion="v1.34.8" Jul 07 14:26:04.123212 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:03.623731 2557 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 07 14:26:04.123212 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:03.624608 2557 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 07 14:26:04.123212 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:03.624615 2557 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 07 14:26:04.123212 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.624668 2557 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 07 14:26:04.123212 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.624674 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 07 14:26:04.123212 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.624677 2557 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 07 14:26:04.123212 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.624680 2557 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 07 14:26:04.123212 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.624683 2557 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 07 14:26:04.123212 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.624685 2557 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 07 14:26:04.123212 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.624688 2557 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 07 14:26:04.123212 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.624690 2557 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 07 14:26:04.123212 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.624693 2557 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 07 14:26:04.123212 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.624696 2557 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 07 14:26:04.123819 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.624699 2557 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 07 14:26:04.123819 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.624701 2557 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 07 14:26:04.123819 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.624704 2557 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 07 14:26:04.123819 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.624706 2557 feature_gate.go:328] unrecognized feature gate: Example2 Jul 07 14:26:04.123819 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.624709 2557 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 07 14:26:04.123819 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.624711 2557 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 07 14:26:04.123819 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.624714 2557 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 07 14:26:04.123819 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.624717 2557 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 07 14:26:04.123819 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.624720 2557 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 07 14:26:04.123819 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.624724 2557 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 07 14:26:04.123819 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.624729 2557 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 07 14:26:04.123819 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.624733 2557 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 07 14:26:04.123819 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.624735 2557 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 07 14:26:04.123819 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.624738 2557 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 07 14:26:04.123819 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.624742 2557 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 07 14:26:04.123819 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.624745 2557 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 07 14:26:04.123819 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.624748 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 07 14:26:04.123819 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.624750 2557 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 07 14:26:04.123819 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.624753 2557 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 07 14:26:04.124427 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.624755 2557 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 07 14:26:04.124427 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.624758 2557 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 07 14:26:04.124427 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.624760 2557 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 07 14:26:04.124427 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.624763 2557 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 07 14:26:04.124427 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.624766 2557 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 07 14:26:04.124427 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.624768 2557 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 07 14:26:04.124427 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.624771 2557 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 07 14:26:04.124427 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.624773 2557 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 07 14:26:04.124427 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.624776 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 07 14:26:04.124427 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.624778 2557 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 07 14:26:04.124427 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.624781 2557 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 07 14:26:04.124427 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.624785 2557 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 07 14:26:04.124427 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.624788 2557 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 07 14:26:04.124427 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.624791 2557 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 07 14:26:04.124427 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.624794 2557 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 07 14:26:04.124427 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.624797 2557 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 07 14:26:04.124427 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.624799 2557 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 07 14:26:04.124427 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.624802 2557 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 07 14:26:04.124427 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.624805 2557 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 07 14:26:04.124427 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.624808 2557 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 07 14:26:04.127185 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.624811 2557 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 07 14:26:04.127185 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.624814 2557 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 07 14:26:04.127185 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.624817 2557 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 07 14:26:04.127185 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.624819 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 07 14:26:04.127185 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.624823 2557 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 07 14:26:04.127185 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.624825 2557 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 07 14:26:04.127185 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.624829 2557 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 07 14:26:04.127185 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.624832 2557 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 07 14:26:04.127185 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.624835 2557 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 07 14:26:04.127185 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.624838 2557 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 07 14:26:04.127185 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.624841 2557 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 07 14:26:04.127185 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.624843 2557 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 07 14:26:04.127185 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.624846 2557 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 07 14:26:04.127185 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.624857 2557 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 07 14:26:04.127185 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.624860 2557 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 07 14:26:04.127185 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.624862 2557 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 07 14:26:04.127185 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.624865 2557 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 07 14:26:04.127185 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.624867 2557 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 07 14:26:04.127185 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.624870 2557 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 07 14:26:04.127185 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.624873 2557 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 07 14:26:04.127739 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.624875 2557 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 07 14:26:04.127739 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.624878 2557 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 07 14:26:04.127739 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.624881 2557 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 07 14:26:04.127739 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.624883 2557 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 07 14:26:04.127739 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.624886 2557 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 07 14:26:04.127739 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.624888 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 07 14:26:04.127739 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.624891 2557 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 07 14:26:04.127739 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.624894 2557 feature_gate.go:328] unrecognized feature gate: Example Jul 07 14:26:04.127739 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.624896 2557 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 07 14:26:04.127739 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.624900 2557 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 07 14:26:04.127739 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.624903 2557 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 07 14:26:04.127739 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.624907 2557 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 07 14:26:04.127739 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.624910 2557 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 07 14:26:04.127739 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.624913 2557 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 07 14:26:04.127739 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.624919 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 07 14:26:04.127739 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.624922 2557 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 07 14:26:04.127739 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.624924 2557 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 07 14:26:04.127739 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.624927 2557 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 07 14:26:04.127739 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.624930 2557 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 07 14:26:04.276339 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.624932 2557 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 07 14:26:04.276339 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.624935 2557 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 07 14:26:04.276339 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.624938 2557 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 07 14:26:04.276339 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.624940 2557 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 07 14:26:04.276339 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.624943 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 07 14:26:04.276339 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.624945 2557 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 07 14:26:04.276339 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.624948 2557 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 07 14:26:04.276339 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.624950 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 07 14:26:04.276339 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.624953 2557 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 07 14:26:04.276339 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.624955 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 07 14:26:04.276339 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.624958 2557 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 07 14:26:04.276339 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:03.624964 2557 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 07 14:26:04.276339 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.625069 2557 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 07 14:26:04.276339 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.625073 2557 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 07 14:26:04.276339 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.625077 2557 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 07 14:26:04.276902 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.625080 2557 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 07 14:26:04.276902 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.625082 2557 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 07 14:26:04.276902 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.625085 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 07 14:26:04.276902 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.625088 2557 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 07 14:26:04.276902 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.625092 2557 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 07 14:26:04.276902 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.625094 2557 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 07 14:26:04.276902 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.625097 2557 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 07 14:26:04.276902 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.625099 2557 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 07 14:26:04.276902 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.625102 2557 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 07 14:26:04.276902 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.625104 2557 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 07 14:26:04.276902 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.625107 2557 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 07 14:26:04.276902 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.625109 2557 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 07 14:26:04.276902 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.625111 2557 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 07 14:26:04.276902 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.625115 2557 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 07 14:26:04.276902 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.625118 2557 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 07 14:26:04.276902 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.625120 2557 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 07 14:26:04.276902 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.625123 2557 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 07 14:26:04.276902 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.625126 2557 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 07 14:26:04.276902 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.625128 2557 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 07 14:26:04.278188 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.625131 2557 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 07 14:26:04.278188 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.625134 2557 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 07 14:26:04.278188 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.625137 2557 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 07 14:26:04.278188 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.625140 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 07 14:26:04.278188 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.625142 2557 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 07 14:26:04.278188 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.625145 2557 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 07 14:26:04.278188 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.625147 2557 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 07 14:26:04.278188 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.625150 2557 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 07 14:26:04.278188 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.625152 2557 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 07 14:26:04.278188 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.625155 2557 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 07 14:26:04.278188 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.625157 2557 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 07 14:26:04.278188 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.625159 2557 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 07 14:26:04.278188 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.625161 2557 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 07 14:26:04.278188 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.625164 2557 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 07 14:26:04.278188 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.625166 2557 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 07 14:26:04.278188 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.625169 2557 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 07 14:26:04.278188 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.625171 2557 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 07 14:26:04.278188 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.625173 2557 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 07 14:26:04.278188 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.625176 2557 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 07 14:26:04.278188 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.625179 2557 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 07 14:26:04.279044 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.625181 2557 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 07 14:26:04.279044 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.625184 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 07 14:26:04.279044 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.625186 2557 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 07 14:26:04.279044 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.625189 2557 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 07 14:26:04.279044 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.625191 2557 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 07 14:26:04.279044 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.625193 2557 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 07 14:26:04.279044 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.625197 2557 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 07 14:26:04.279044 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.625199 2557 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 07 14:26:04.279044 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.625202 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 07 14:26:04.279044 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.625204 2557 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 07 14:26:04.279044 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.625207 2557 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 07 14:26:04.279044 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.625209 2557 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 07 14:26:04.279044 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.625212 2557 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 07 14:26:04.279044 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.625216 2557 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 07 14:26:04.279044 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.625224 2557 feature_gate.go:328] unrecognized feature gate: Example2 Jul 07 14:26:04.279044 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.625227 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 07 14:26:04.279044 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.625230 2557 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 07 14:26:04.279044 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.625233 2557 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 07 14:26:04.279044 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.625236 2557 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 07 14:26:04.279044 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.625239 2557 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 07 14:26:04.279912 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.625242 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 07 14:26:04.279912 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.625244 2557 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 07 14:26:04.279912 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.625247 2557 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 07 14:26:04.279912 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.625249 2557 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 07 14:26:04.279912 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.625252 2557 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 07 14:26:04.279912 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.625254 2557 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 07 14:26:04.279912 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.625257 2557 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 07 14:26:04.279912 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.625259 2557 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 07 14:26:04.279912 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.625262 2557 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 07 14:26:04.279912 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.625264 2557 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 07 14:26:04.279912 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.625267 2557 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 07 14:26:04.279912 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.625269 2557 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 07 14:26:04.279912 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.625272 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 07 14:26:04.279912 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.625275 2557 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 07 14:26:04.279912 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.625277 2557 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 07 14:26:04.279912 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.625280 2557 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 07 14:26:04.279912 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.625282 2557 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 07 14:26:04.279912 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.625285 2557 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 07 14:26:04.279912 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.625288 2557 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 07 14:26:04.279912 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.625291 2557 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 07 14:26:04.280535 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.625294 2557 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 07 14:26:04.280535 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.625297 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 07 14:26:04.280535 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.625300 2557 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 07 14:26:04.280535 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.625303 2557 feature_gate.go:328] unrecognized feature gate: Example Jul 07 14:26:04.280535 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.625305 2557 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 07 14:26:04.280535 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.625308 2557 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 07 14:26:04.280535 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.625312 2557 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 07 14:26:04.280535 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.625315 2557 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 07 14:26:04.280535 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.625317 2557 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 07 14:26:04.280535 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.625320 2557 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 07 14:26:04.280535 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.625322 2557 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 07 14:26:04.280535 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.625325 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 07 14:26:04.280535 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.625327 2557 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 07 14:26:04.280535 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.625329 2557 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 07 14:26:04.280535 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.625333 2557 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 07 14:26:04.280535 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.625336 2557 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 07 14:26:04.280535 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:03.625339 2557 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 07 14:26:04.281414 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:03.625344 2557 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 07 14:26:04.281414 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:03.625440 2557 server.go:962] "Client rotation is on, will bootstrap in background" Jul 07 14:26:04.281414 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:03.627364 2557 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 07 14:26:04.281414 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:03.628965 2557 server.go:1019] "Starting client certificate rotation" Jul 07 14:26:04.281414 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:03.629061 2557 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 07 14:26:04.281414 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:03.629118 2557 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 07 14:26:04.281414 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:03.631286 2557 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 07 14:26:04.281414 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:03.632363 2557 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 07 14:26:04.281414 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:03.640845 2557 log.go:25] "Validated CRI v1 runtime API" Jul 07 14:26:04.281414 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:03.642792 2557 log.go:25] "Validated CRI v1 image API" Jul 07 14:26:04.281414 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:03.643535 2557 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 07 14:26:04.281414 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:03.647720 2557 fs.go:135] Filesystem UUIDs: map[00843c4e-7220-43f6-9188-85ccb2a8c65d:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 b6b4fbe3-ce3c-4497-ac5b-9a16e1f093f9:/dev/nvme0n1p4] Jul 07 14:26:04.282062 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:03.647737 2557 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 07 14:26:04.282273 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:03.654858 2557 manager.go:216] Machine: {Timestamp:2026-07-07 14:26:03.653481296 +0000 UTC m=+0.362619667 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3186348 MemoryCapacity:33164492800 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec299736e29a4258e9a0da64dc5c69e3 SystemUUID:ec299736-e29a-4258-e9a0-da64dc5c69e3 BootID:83ec6f7d-6300-481c-b279-3491ed065824 Filesystems:[{Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582246400 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:61:cc:53:d9:d7 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:61:cc:53:d9:d7 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:b6:c0:be:47:71:93 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164492800 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 07 14:26:04.282273 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:03.654960 2557 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 07 14:26:04.282273 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:03.655033 2557 manager.go:232] Version: {KernelVersion:5.14.0-570.125.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260630-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 07 14:26:04.282273 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:03.658016 2557 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 07 14:26:04.282273 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:03.658046 2557 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-134-42.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 07 14:26:04.282273 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:03.658449 2557 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 07 14:26:04.282273 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:03.658717 2557 topology_manager.go:138] "Creating topology manager with none policy" Jul 07 14:26:04.282811 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:03.658728 2557 container_manager_linux.go:309] "Creating device plugin manager" Jul 07 14:26:04.282811 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:03.658736 2557 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 07 14:26:04.282811 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:03.659921 2557 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 07 14:26:04.282811 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:03.661146 2557 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 07 14:26:04.282811 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:03.664234 2557 state_mem.go:36] "Initialized new in-memory state store" Jul 07 14:26:04.282811 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:03.664331 2557 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 07 14:26:04.282811 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:03.664357 2557 kubelet.go:486] "Attempting to sync node with API server" Jul 07 14:26:04.282811 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:03.664365 2557 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 07 14:26:04.282811 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:03.664380 2557 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 07 14:26:04.282811 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:03.664389 2557 kubelet.go:398] "Adding apiserver pod source" Jul 07 14:26:04.282811 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:03.664397 2557 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 07 14:26:04.282811 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:03.664778 2557 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 07 14:26:04.282811 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:03.664800 2557 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 07 14:26:04.282811 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:03.667072 2557 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.8-3.rhaos4.21.git91b4b0b.el9" apiVersion="v1" Jul 07 14:26:04.282811 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:03.668992 2557 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 07 14:26:04.282811 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:03.669019 2557 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 07 14:26:04.282811 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:03.670035 2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 07 14:26:04.282811 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:03.670049 2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 07 14:26:04.282811 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:03.670091 2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 07 14:26:04.282811 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:03.670098 2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 07 14:26:04.282811 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:03.670108 2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 07 14:26:04.282811 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:03.670113 2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 07 14:26:04.282811 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:03.670125 2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 07 14:26:04.282811 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:03.670130 2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 07 14:26:04.282811 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:03.670137 2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 07 14:26:04.282811 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:03.670142 2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 07 14:26:04.282811 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:03.670149 2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 07 14:26:04.282811 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:03.670155 2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 07 14:26:04.282811 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:03.671716 2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 07 14:26:04.282811 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:03.671727 2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 07 14:26:04.282811 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:03.674732 2557 server.go:1268] "Started kubelet" Jul 07 14:26:04.282811 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:03.674800 2557 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 07 14:26:04.282811 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:03.674835 2557 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 07 14:26:04.282811 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:03.674855 2557 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 07 14:26:04.282811 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:03.675944 2557 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 07 14:26:04.282811 ip-10-0-134-42 kubenswrapper[2557]: E0707 14:26:03.676120 2557 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 07 14:26:04.282811 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:03.676219 2557 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-134-42.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 07 14:26:04.282811 ip-10-0-134-42 kubenswrapper[2557]: E0707 14:26:03.676406 2557 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-134-42.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 07 14:26:04.282811 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:03.677114 2557 server.go:310] "Adding debug handlers to kubelet server" Jul 07 14:26:04.282811 ip-10-0-134-42 kubenswrapper[2557]: E0707 14:26:03.679863 2557 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 07 14:26:04.284048 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:03.682333 2557 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 07 14:26:04.284048 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:03.682358 2557 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 07 14:26:04.284048 ip-10-0-134-42 kubenswrapper[2557]: E0707 14:26:03.682423 2557 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-42.ec2.internal\" not found" Jul 07 14:26:04.284048 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:03.682909 2557 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 07 14:26:04.284048 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:03.683038 2557 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 07 14:26:04.284048 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:03.683060 2557 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 07 14:26:04.284048 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:03.683261 2557 reconstruct.go:97] "Volume reconstruction finished" Jul 07 14:26:04.284048 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:03.683271 2557 reconciler.go:29] "Reconciler: start to sync state" Jul 07 14:26:04.284048 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:03.683902 2557 factory.go:55] Registering systemd factory Jul 07 14:26:04.284048 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:03.683913 2557 factory.go:223] Registration of the systemd container factory successfully Jul 07 14:26:04.284048 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:03.684290 2557 factory.go:153] Registering CRI-O factory Jul 07 14:26:04.284048 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:03.684302 2557 factory.go:223] Registration of the crio container factory successfully Jul 07 14:26:04.284048 ip-10-0-134-42 kubenswrapper[2557]: E0707 14:26:03.683355 2557 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-134-42.ec2.internal.18c0081acf0e4c6a default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-134-42.ec2.internal,UID:ip-10-0-134-42.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-134-42.ec2.internal,},FirstTimestamp:2026-07-07 14:26:03.674709098 +0000 UTC m=+0.383847469,LastTimestamp:2026-07-07 14:26:03.674709098 +0000 UTC m=+0.383847469,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-134-42.ec2.internal,}" Jul 07 14:26:04.284048 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:03.684348 2557 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 07 14:26:04.284048 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:03.684372 2557 factory.go:103] Registering Raw factory Jul 07 14:26:04.284048 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:03.684385 2557 manager.go:1196] Started watching for new ooms in manager Jul 07 14:26:04.284048 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:03.684799 2557 manager.go:349] Starting recovery of all containers Jul 07 14:26:04.284048 ip-10-0-134-42 kubenswrapper[2557]: E0707 14:26:03.686964 2557 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 07 14:26:04.284048 ip-10-0-134-42 kubenswrapper[2557]: E0707 14:26:03.687128 2557 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-134-42.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 07 14:26:04.284048 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:03.695403 2557 manager.go:354] Recovery completed Jul 07 14:26:04.284048 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:03.701346 2557 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 14:26:04.284048 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:03.702946 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-42.ec2.internal" event="NodeHasSufficientMemory" Jul 07 14:26:04.284048 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:03.702970 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-42.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 14:26:04.284048 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:03.702985 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-42.ec2.internal" event="NodeHasSufficientPID" Jul 07 14:26:04.284048 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:03.703476 2557 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 07 14:26:04.284048 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:03.703488 2557 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 07 14:26:04.284048 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:03.703505 2557 state_mem.go:36] "Initialized new in-memory state store" Jul 07 14:26:04.284048 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:03.703478 2557 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 14:26:04.284048 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:03.704367 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-42.ec2.internal" event="NodeHasSufficientMemory" Jul 07 14:26:04.284048 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:03.704388 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-42.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 14:26:04.284048 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:03.704401 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-42.ec2.internal" event="NodeHasSufficientPID" Jul 07 14:26:04.284048 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:03.704773 2557 policy_none.go:49] "None policy: Start" Jul 07 14:26:04.284048 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:03.704784 2557 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 07 14:26:04.284048 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:03.704796 2557 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 07 14:26:04.285128 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:03.705882 2557 policy_none.go:47] "Start" Jul 07 14:26:04.285128 ip-10-0-134-42 kubenswrapper[2557]: E0707 14:26:03.706159 2557 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-134-42.ec2.internal.18c0081ad0bd5793 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-134-42.ec2.internal,UID:ip-10-0-134-42.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-134-42.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-134-42.ec2.internal,},FirstTimestamp:2026-07-07 14:26:03.702957971 +0000 UTC m=+0.412096345,LastTimestamp:2026-07-07 14:26:03.702957971 +0000 UTC m=+0.412096345,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-134-42.ec2.internal,}" Jul 07 14:26:04.285128 ip-10-0-134-42 kubenswrapper[2557]: E0707 14:26:03.714654 2557 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-134-42.ec2.internal.18c0081ad0bd963d default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-134-42.ec2.internal,UID:ip-10-0-134-42.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-134-42.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-134-42.ec2.internal,},FirstTimestamp:2026-07-07 14:26:03.702974013 +0000 UTC m=+0.412112384,LastTimestamp:2026-07-07 14:26:03.702974013 +0000 UTC m=+0.412112384,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-134-42.ec2.internal,}" Jul 07 14:26:04.285128 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:03.723566 2557 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-csbmt" Jul 07 14:26:04.285128 ip-10-0-134-42 kubenswrapper[2557]: E0707 14:26:03.725528 2557 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-134-42.ec2.internal.18c0081ad0bdd284 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-134-42.ec2.internal,UID:ip-10-0-134-42.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-134-42.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-134-42.ec2.internal,},FirstTimestamp:2026-07-07 14:26:03.702989444 +0000 UTC m=+0.412127815,LastTimestamp:2026-07-07 14:26:03.702989444 +0000 UTC m=+0.412127815,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-134-42.ec2.internal,}" Jul 07 14:26:04.285128 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:03.731629 2557 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-csbmt" Jul 07 14:26:04.285128 ip-10-0-134-42 kubenswrapper[2557]: E0707 14:26:03.782474 2557 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-42.ec2.internal\" not found" Jul 07 14:26:04.285128 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:03.793109 2557 manager.go:337] "Starting Device Plugin manager" Jul 07 14:26:04.285128 ip-10-0-134-42 kubenswrapper[2557]: E0707 14:26:03.793138 2557 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 07 14:26:04.285128 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:03.793146 2557 server.go:87] "Starting device plugin registration server" Jul 07 14:26:04.285128 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:03.793333 2557 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 07 14:26:04.285128 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:03.793345 2557 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 07 14:26:04.285128 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:03.793480 2557 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 07 14:26:04.285128 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:03.793573 2557 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 07 14:26:04.285128 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:03.793579 2557 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 07 14:26:04.285128 ip-10-0-134-42 kubenswrapper[2557]: E0707 14:26:03.793909 2557 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 07 14:26:04.285128 ip-10-0-134-42 kubenswrapper[2557]: E0707 14:26:03.793954 2557 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-134-42.ec2.internal\" not found" Jul 07 14:26:04.285128 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:03.827815 2557 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 07 14:26:04.285128 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:03.828983 2557 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 07 14:26:04.285128 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:03.829012 2557 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 07 14:26:04.285779 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:03.829030 2557 kubelet.go:2448] "Starting kubelet main sync loop" Jul 07 14:26:04.285779 ip-10-0-134-42 kubenswrapper[2557]: E0707 14:26:03.829058 2557 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 07 14:26:04.285779 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:03.831979 2557 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 07 14:26:04.285779 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:03.894285 2557 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 14:26:04.285779 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:03.895006 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-42.ec2.internal" event="NodeHasSufficientMemory" Jul 07 14:26:04.285779 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:03.895029 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-42.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 14:26:04.285779 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:03.895042 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-42.ec2.internal" event="NodeHasSufficientPID" Jul 07 14:26:04.285779 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:03.895067 2557 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-134-42.ec2.internal" Jul 07 14:26:04.285779 ip-10-0-134-42 kubenswrapper[2557]: E0707 14:26:03.895422 2557 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-134-42.ec2.internal\" not found" node="ip-10-0-134-42.ec2.internal" Jul 07 14:26:04.285779 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:03.905041 2557 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-134-42.ec2.internal" Jul 07 14:26:04.285779 ip-10-0-134-42 kubenswrapper[2557]: E0707 14:26:03.905057 2557 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-134-42.ec2.internal\": node \"ip-10-0-134-42.ec2.internal\" not found" Jul 07 14:26:04.285779 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:03.929862 2557 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["kube-system/kube-apiserver-proxy-ip-10-0-134-42.ec2.internal","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-42.ec2.internal"] Jul 07 14:26:04.285779 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:03.929929 2557 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 14:26:04.285779 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:03.931461 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-42.ec2.internal" event="NodeHasSufficientMemory" Jul 07 14:26:04.285779 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:03.931627 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-42.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 14:26:04.285779 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:03.931641 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-42.ec2.internal" event="NodeHasSufficientPID" Jul 07 14:26:04.285779 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:03.933185 2557 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 14:26:04.285779 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:03.933293 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-42.ec2.internal" Jul 07 14:26:04.285779 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:03.933335 2557 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 14:26:04.285779 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:03.933937 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-42.ec2.internal" event="NodeHasSufficientMemory" Jul 07 14:26:04.285779 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:03.933968 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-42.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 14:26:04.285779 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:03.933990 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-42.ec2.internal" event="NodeHasSufficientPID" Jul 07 14:26:04.285779 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:03.934035 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-42.ec2.internal" event="NodeHasSufficientMemory" Jul 07 14:26:04.285779 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:03.934060 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-42.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 14:26:04.285779 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:03.934069 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-42.ec2.internal" event="NodeHasSufficientPID" Jul 07 14:26:04.285779 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:03.935357 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-42.ec2.internal" Jul 07 14:26:04.285779 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:03.935380 2557 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 14:26:04.285779 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:03.936035 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-42.ec2.internal" event="NodeHasSufficientMemory" Jul 07 14:26:04.285779 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:03.936063 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-42.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 14:26:04.285779 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:03.936078 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-42.ec2.internal" event="NodeHasSufficientPID" Jul 07 14:26:04.285779 ip-10-0-134-42 kubenswrapper[2557]: E0707 14:26:03.950693 2557 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-42.ec2.internal\" not found" Jul 07 14:26:04.285779 ip-10-0-134-42 kubenswrapper[2557]: E0707 14:26:03.971270 2557 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-134-42.ec2.internal\" not found" node="ip-10-0-134-42.ec2.internal" Jul 07 14:26:04.286629 ip-10-0-134-42 kubenswrapper[2557]: E0707 14:26:03.975319 2557 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-134-42.ec2.internal\" not found" node="ip-10-0-134-42.ec2.internal" Jul 07 14:26:04.286629 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:03.984477 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6063cb059acc790b3bb3b3d99eea6099-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-134-42.ec2.internal\" (UID: \"6063cb059acc790b3bb3b3d99eea6099\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-42.ec2.internal" Jul 07 14:26:04.286629 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:03.984497 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/61dca777ea2260edb003f493cc5637ae-config\") pod \"kube-apiserver-proxy-ip-10-0-134-42.ec2.internal\" (UID: \"61dca777ea2260edb003f493cc5637ae\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-134-42.ec2.internal" Jul 07 14:26:04.286629 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:03.984513 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/6063cb059acc790b3bb3b3d99eea6099-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-134-42.ec2.internal\" (UID: \"6063cb059acc790b3bb3b3d99eea6099\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-42.ec2.internal" Jul 07 14:26:04.286629 ip-10-0-134-42 kubenswrapper[2557]: E0707 14:26:04.051671 2557 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-42.ec2.internal\" not found" Jul 07 14:26:04.286629 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.085046 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/6063cb059acc790b3bb3b3d99eea6099-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-134-42.ec2.internal\" (UID: \"6063cb059acc790b3bb3b3d99eea6099\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-42.ec2.internal" Jul 07 14:26:04.286629 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.085070 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6063cb059acc790b3bb3b3d99eea6099-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-134-42.ec2.internal\" (UID: \"6063cb059acc790b3bb3b3d99eea6099\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-42.ec2.internal" Jul 07 14:26:04.286629 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.085086 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/61dca777ea2260edb003f493cc5637ae-config\") pod \"kube-apiserver-proxy-ip-10-0-134-42.ec2.internal\" (UID: \"61dca777ea2260edb003f493cc5637ae\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-134-42.ec2.internal" Jul 07 14:26:04.286629 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.085110 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/61dca777ea2260edb003f493cc5637ae-config\") pod \"kube-apiserver-proxy-ip-10-0-134-42.ec2.internal\" (UID: \"61dca777ea2260edb003f493cc5637ae\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-134-42.ec2.internal" Jul 07 14:26:04.286629 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.085130 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/6063cb059acc790b3bb3b3d99eea6099-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-134-42.ec2.internal\" (UID: \"6063cb059acc790b3bb3b3d99eea6099\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-42.ec2.internal" Jul 07 14:26:04.286629 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.085144 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6063cb059acc790b3bb3b3d99eea6099-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-134-42.ec2.internal\" (UID: \"6063cb059acc790b3bb3b3d99eea6099\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-42.ec2.internal" Jul 07 14:26:04.286629 ip-10-0-134-42 kubenswrapper[2557]: E0707 14:26:04.152543 2557 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-42.ec2.internal\" not found" Jul 07 14:26:04.286629 ip-10-0-134-42 kubenswrapper[2557]: E0707 14:26:04.253425 2557 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-42.ec2.internal\" not found" Jul 07 14:26:04.286629 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.273602 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-42.ec2.internal" Jul 07 14:26:04.286629 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.278552 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-42.ec2.internal" Jul 07 14:26:04.393498 ip-10-0-134-42 kubenswrapper[2557]: E0707 14:26:04.353682 2557 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-42.ec2.internal\" not found" Jul 07 14:26:04.532653 ip-10-0-134-42 kubenswrapper[2557]: E0707 14:26:04.454209 2557 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-42.ec2.internal\" not found" Jul 07 14:26:04.532653 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.523693 2557 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 07 14:26:04.656928 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.583088 2557 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-42.ec2.internal" Jul 07 14:26:04.656928 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.595236 2557 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 07 14:26:04.656928 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.609353 2557 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 07 14:26:04.656928 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.610622 2557 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-42.ec2.internal" Jul 07 14:26:04.656928 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.627581 2557 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 07 14:26:04.656928 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.627713 2557 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection force closed via ClientConn.Close\") has prevented the request from succeeding" Jul 07 14:26:04.656928 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.627713 2557 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection force closed via ClientConn.Close\") has prevented the request from succeeding" Jul 07 14:26:04.656928 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.627754 2557 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection force closed via ClientConn.Close\") has prevented the request from succeeding" Jul 07 14:26:04.656928 ip-10-0-134-42 kubenswrapper[2557]: E0707 14:26:04.627776 2557 kubelet.go:3250] "Failed creating a mirror pod" err="Post \"https://a6ef97384509d4ad6aff53915b6cedbb-3fb2c87ed1a48548.elb.us-east-1.amazonaws.com:6443/api/v1/namespaces/openshift-machine-config-operator/pods\": http2: client connection force closed via ClientConn.Close" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-42.ec2.internal" Jul 07 14:26:04.761465 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.665634 2557 apiserver.go:52] "Watching apiserver" Jul 07 14:26:04.761465 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.682624 2557 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 07 14:26:04.761465 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.684984 2557 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 07 14:26:04.761465 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.685310 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/node-resolver-wmt48","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-42.ec2.internal","openshift-multus/multus-additional-cni-plugins-shb4c","openshift-multus/network-metrics-daemon-s4p8l","openshift-network-operator/iptables-alerter-rgvqn","kube-system/kube-apiserver-proxy-ip-10-0-134-42.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zh4rc","openshift-image-registry/node-ca-5g7bh","openshift-multus/multus-42n8m","openshift-network-diagnostics/network-check-target-bqthq","openshift-ovn-kubernetes/ovnkube-node-scfg4","kube-system/global-pull-secret-syncer-hvlzx","kube-system/konnectivity-agent-2m5td","openshift-cluster-node-tuning-operator/tuned-pwmxs"] Jul 07 14:26:04.761465 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.687070 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zh4rc" Jul 07 14:26:04.761465 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.688488 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-42n8m" Jul 07 14:26:04.761465 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.689900 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-shb4c" Jul 07 14:26:04.761465 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.691676 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s4p8l" Jul 07 14:26:04.761465 ip-10-0-134-42 kubenswrapper[2557]: E0707 14:26:04.691743 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-s4p8l" podUID="320ab15e-c868-4a37-a3e6-1cea72bb74a4" Jul 07 14:26:04.761465 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.693161 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-wmt48" Jul 07 14:26:04.761465 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.694704 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-scfg4" Jul 07 14:26:04.761465 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.696568 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-rgvqn" Jul 07 14:26:04.761465 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.697912 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-2m5td" Jul 07 14:26:04.761465 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.698976 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 07 14:26:04.761465 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.699001 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 07 14:26:04.761465 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.699038 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 07 14:26:04.761465 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.699038 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 07 14:26:04.761465 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.699268 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 07 14:26:04.761465 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.699278 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 07 14:26:04.761465 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.699288 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-vfm84\"" Jul 07 14:26:04.761465 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.699362 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-5q99j\"" Jul 07 14:26:04.761465 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.699292 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 07 14:26:04.761465 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.699437 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 07 14:26:04.761465 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.699576 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 07 14:26:04.761465 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.699621 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 07 14:26:04.761465 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.699635 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-gfjqt\"" Jul 07 14:26:04.761465 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.699675 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 07 14:26:04.761465 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.699624 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-sdw92\"" Jul 07 14:26:04.762509 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.699787 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 07 14:26:04.762509 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.699824 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 07 14:26:04.762509 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.699863 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-t4z46\"" Jul 07 14:26:04.762509 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.699897 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 07 14:26:04.762509 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.700033 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 07 14:26:04.762509 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.700059 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bqthq" Jul 07 14:26:04.762509 ip-10-0-134-42 kubenswrapper[2557]: E0707 14:26:04.700114 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bqthq" podUID="09a0edd9-f556-490d-a691-a0cb3dc3aff9" Jul 07 14:26:04.762509 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.700540 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 07 14:26:04.762509 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.700599 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 07 14:26:04.762509 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.700610 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 07 14:26:04.762509 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.700657 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 07 14:26:04.762509 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.700977 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-c4tjf\"" Jul 07 14:26:04.762509 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.701031 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 07 14:26:04.762509 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.701145 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 07 14:26:04.762509 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.701749 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-9gqcj\"" Jul 07 14:26:04.762509 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.701861 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 07 14:26:04.762509 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.703105 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hvlzx" Jul 07 14:26:04.762509 ip-10-0-134-42 kubenswrapper[2557]: E0707 14:26:04.703482 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hvlzx" podUID="ce774983-4cbb-4526-ad84-72478efce781" Jul 07 14:26:04.762509 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.704919 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-5g7bh" Jul 07 14:26:04.762509 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.715404 2557 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 07 14:26:04.762509 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.716781 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-4t5rd\"" Jul 07 14:26:04.762509 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.716800 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-pwmxs" Jul 07 14:26:04.762509 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.717082 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 07 14:26:04.762509 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.717146 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 07 14:26:04.762509 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.718155 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 07 14:26:04.762509 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.718604 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-xx5hm\"" Jul 07 14:26:04.762509 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.718629 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 07 14:26:04.762509 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.718983 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 07 14:26:04.763350 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.734417 2557 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-06 14:21:04 +0000 UTC" deadline="2027-12-03 08:33:37.211425205 +0000 UTC" Jul 07 14:26:04.763350 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.734459 2557 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12330h7m32.476968783s" Jul 07 14:26:04.867730 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.784866 2557 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 07 14:26:04.867730 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.790100 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/0b70b350-feef-419b-91c0-45a22ae7f30b-host\") pod \"node-ca-5g7bh\" (UID: \"0b70b350-feef-419b-91c0-45a22ae7f30b\") " pod="openshift-image-registry/node-ca-5g7bh" Jul 07 14:26:04.867730 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.790122 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hvmdz\" (UniqueName: \"kubernetes.io/projected/0b70b350-feef-419b-91c0-45a22ae7f30b-kube-api-access-hvmdz\") pod \"node-ca-5g7bh\" (UID: \"0b70b350-feef-419b-91c0-45a22ae7f30b\") " pod="openshift-image-registry/node-ca-5g7bh" Jul 07 14:26:04.867730 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.790177 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/01caeef5-2981-41b4-92c9-75a39f813595-etc-modprobe-d\") pod \"tuned-pwmxs\" (UID: \"01caeef5-2981-41b4-92c9-75a39f813595\") " pod="openshift-cluster-node-tuning-operator/tuned-pwmxs" Jul 07 14:26:04.867730 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.790194 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/90d2dce1-be42-4595-b9c4-74a024eef0e9-run-systemd\") pod \"ovnkube-node-scfg4\" (UID: \"90d2dce1-be42-4595-b9c4-74a024eef0e9\") " pod="openshift-ovn-kubernetes/ovnkube-node-scfg4" Jul 07 14:26:04.867730 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.790213 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/01caeef5-2981-41b4-92c9-75a39f813595-host\") pod \"tuned-pwmxs\" (UID: \"01caeef5-2981-41b4-92c9-75a39f813595\") " pod="openshift-cluster-node-tuning-operator/tuned-pwmxs" Jul 07 14:26:04.867730 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.790228 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2skqj\" (UniqueName: \"kubernetes.io/projected/9465f33d-a02b-4cd7-8c55-d30fe0648034-kube-api-access-2skqj\") pod \"multus-42n8m\" (UID: \"9465f33d-a02b-4cd7-8c55-d30fe0648034\") " pod="openshift-multus/multus-42n8m" Jul 07 14:26:04.867730 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.790266 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/d3b76148-a347-4b4e-9ba5-fb4c41ebc389-cni-binary-copy\") pod \"multus-additional-cni-plugins-shb4c\" (UID: \"d3b76148-a347-4b4e-9ba5-fb4c41ebc389\") " pod="openshift-multus/multus-additional-cni-plugins-shb4c" Jul 07 14:26:04.867730 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.790290 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/90d2dce1-be42-4595-b9c4-74a024eef0e9-host-run-ovn-kubernetes\") pod \"ovnkube-node-scfg4\" (UID: \"90d2dce1-be42-4595-b9c4-74a024eef0e9\") " pod="openshift-ovn-kubernetes/ovnkube-node-scfg4" Jul 07 14:26:04.867730 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.790313 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/90d2dce1-be42-4595-b9c4-74a024eef0e9-host-cni-netd\") pod \"ovnkube-node-scfg4\" (UID: \"90d2dce1-be42-4595-b9c4-74a024eef0e9\") " pod="openshift-ovn-kubernetes/ovnkube-node-scfg4" Jul 07 14:26:04.867730 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.790329 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/fa56eabc-b99d-4ce2-9f36-cb0242168cf4-host-slash\") pod \"iptables-alerter-rgvqn\" (UID: \"fa56eabc-b99d-4ce2-9f36-cb0242168cf4\") " pod="openshift-network-operator/iptables-alerter-rgvqn" Jul 07 14:26:04.867730 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.790344 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/d3b76148-a347-4b4e-9ba5-fb4c41ebc389-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-shb4c\" (UID: \"d3b76148-a347-4b4e-9ba5-fb4c41ebc389\") " pod="openshift-multus/multus-additional-cni-plugins-shb4c" Jul 07 14:26:04.867730 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.790415 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/d3b76148-a347-4b4e-9ba5-fb4c41ebc389-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-shb4c\" (UID: \"d3b76148-a347-4b4e-9ba5-fb4c41ebc389\") " pod="openshift-multus/multus-additional-cni-plugins-shb4c" Jul 07 14:26:04.867730 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.790431 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/90d2dce1-be42-4595-b9c4-74a024eef0e9-host-slash\") pod \"ovnkube-node-scfg4\" (UID: \"90d2dce1-be42-4595-b9c4-74a024eef0e9\") " pod="openshift-ovn-kubernetes/ovnkube-node-scfg4" Jul 07 14:26:04.869206 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.790445 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/90d2dce1-be42-4595-b9c4-74a024eef0e9-etc-openvswitch\") pod \"ovnkube-node-scfg4\" (UID: \"90d2dce1-be42-4595-b9c4-74a024eef0e9\") " pod="openshift-ovn-kubernetes/ovnkube-node-scfg4" Jul 07 14:26:04.869206 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.790460 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xf42z\" (UniqueName: \"kubernetes.io/projected/fa56eabc-b99d-4ce2-9f36-cb0242168cf4-kube-api-access-xf42z\") pod \"iptables-alerter-rgvqn\" (UID: \"fa56eabc-b99d-4ce2-9f36-cb0242168cf4\") " pod="openshift-network-operator/iptables-alerter-rgvqn" Jul 07 14:26:04.869206 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.790515 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/3f8eb048-9150-421e-8359-e64f3aec2211-etc-selinux\") pod \"aws-ebs-csi-driver-node-zh4rc\" (UID: \"3f8eb048-9150-421e-8359-e64f3aec2211\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zh4rc" Jul 07 14:26:04.869206 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.790575 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q5g9n\" (UniqueName: \"kubernetes.io/projected/09a0edd9-f556-490d-a691-a0cb3dc3aff9-kube-api-access-q5g9n\") pod \"network-check-target-bqthq\" (UID: \"09a0edd9-f556-490d-a691-a0cb3dc3aff9\") " pod="openshift-network-diagnostics/network-check-target-bqthq" Jul 07 14:26:04.869206 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.790651 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/9465f33d-a02b-4cd7-8c55-d30fe0648034-cnibin\") pod \"multus-42n8m\" (UID: \"9465f33d-a02b-4cd7-8c55-d30fe0648034\") " pod="openshift-multus/multus-42n8m" Jul 07 14:26:04.869206 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.790676 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/9465f33d-a02b-4cd7-8c55-d30fe0648034-cni-binary-copy\") pod \"multus-42n8m\" (UID: \"9465f33d-a02b-4cd7-8c55-d30fe0648034\") " pod="openshift-multus/multus-42n8m" Jul 07 14:26:04.869206 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.790699 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/9465f33d-a02b-4cd7-8c55-d30fe0648034-host-run-multus-certs\") pod \"multus-42n8m\" (UID: \"9465f33d-a02b-4cd7-8c55-d30fe0648034\") " pod="openshift-multus/multus-42n8m" Jul 07 14:26:04.869206 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.790723 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9465f33d-a02b-4cd7-8c55-d30fe0648034-etc-kubernetes\") pod \"multus-42n8m\" (UID: \"9465f33d-a02b-4cd7-8c55-d30fe0648034\") " pod="openshift-multus/multus-42n8m" Jul 07 14:26:04.869206 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.790747 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/90d2dce1-be42-4595-b9c4-74a024eef0e9-var-lib-openvswitch\") pod \"ovnkube-node-scfg4\" (UID: \"90d2dce1-be42-4595-b9c4-74a024eef0e9\") " pod="openshift-ovn-kubernetes/ovnkube-node-scfg4" Jul 07 14:26:04.869206 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.790776 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/90d2dce1-be42-4595-b9c4-74a024eef0e9-ovnkube-script-lib\") pod \"ovnkube-node-scfg4\" (UID: \"90d2dce1-be42-4595-b9c4-74a024eef0e9\") " pod="openshift-ovn-kubernetes/ovnkube-node-scfg4" Jul 07 14:26:04.869206 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.790799 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/01caeef5-2981-41b4-92c9-75a39f813595-etc-sysconfig\") pod \"tuned-pwmxs\" (UID: \"01caeef5-2981-41b4-92c9-75a39f813595\") " pod="openshift-cluster-node-tuning-operator/tuned-pwmxs" Jul 07 14:26:04.869206 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.790824 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tg5zt\" (UniqueName: \"kubernetes.io/projected/01caeef5-2981-41b4-92c9-75a39f813595-kube-api-access-tg5zt\") pod \"tuned-pwmxs\" (UID: \"01caeef5-2981-41b4-92c9-75a39f813595\") " pod="openshift-cluster-node-tuning-operator/tuned-pwmxs" Jul 07 14:26:04.869206 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.790866 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/9465f33d-a02b-4cd7-8c55-d30fe0648034-host-run-k8s-cni-cncf-io\") pod \"multus-42n8m\" (UID: \"9465f33d-a02b-4cd7-8c55-d30fe0648034\") " pod="openshift-multus/multus-42n8m" Jul 07 14:26:04.869206 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.790894 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p4z9t\" (UniqueName: \"kubernetes.io/projected/320ab15e-c868-4a37-a3e6-1cea72bb74a4-kube-api-access-p4z9t\") pod \"network-metrics-daemon-s4p8l\" (UID: \"320ab15e-c868-4a37-a3e6-1cea72bb74a4\") " pod="openshift-multus/network-metrics-daemon-s4p8l" Jul 07 14:26:04.871612 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.790919 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/6c88a149-4504-482b-90e7-87d3065096c2-konnectivity-ca\") pod \"konnectivity-agent-2m5td\" (UID: \"6c88a149-4504-482b-90e7-87d3065096c2\") " pod="kube-system/konnectivity-agent-2m5td" Jul 07 14:26:04.871612 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.790967 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ce774983-4cbb-4526-ad84-72478efce781-original-pull-secret\") pod \"global-pull-secret-syncer-hvlzx\" (UID: \"ce774983-4cbb-4526-ad84-72478efce781\") " pod="kube-system/global-pull-secret-syncer-hvlzx" Jul 07 14:26:04.871612 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.790995 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mhhjl\" (UniqueName: \"kubernetes.io/projected/d3b76148-a347-4b4e-9ba5-fb4c41ebc389-kube-api-access-mhhjl\") pod \"multus-additional-cni-plugins-shb4c\" (UID: \"d3b76148-a347-4b4e-9ba5-fb4c41ebc389\") " pod="openshift-multus/multus-additional-cni-plugins-shb4c" Jul 07 14:26:04.871612 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.791031 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-csjl2\" (UniqueName: \"kubernetes.io/projected/aa141f90-5856-4779-90d7-c6377b1e32d6-kube-api-access-csjl2\") pod \"node-resolver-wmt48\" (UID: \"aa141f90-5856-4779-90d7-c6377b1e32d6\") " pod="openshift-dns/node-resolver-wmt48" Jul 07 14:26:04.871612 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.791061 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/9465f33d-a02b-4cd7-8c55-d30fe0648034-host-run-netns\") pod \"multus-42n8m\" (UID: \"9465f33d-a02b-4cd7-8c55-d30fe0648034\") " pod="openshift-multus/multus-42n8m" Jul 07 14:26:04.871612 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.791086 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/ce774983-4cbb-4526-ad84-72478efce781-dbus\") pod \"global-pull-secret-syncer-hvlzx\" (UID: \"ce774983-4cbb-4526-ad84-72478efce781\") " pod="kube-system/global-pull-secret-syncer-hvlzx" Jul 07 14:26:04.871612 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.791116 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/aa141f90-5856-4779-90d7-c6377b1e32d6-tmp-dir\") pod \"node-resolver-wmt48\" (UID: \"aa141f90-5856-4779-90d7-c6377b1e32d6\") " pod="openshift-dns/node-resolver-wmt48" Jul 07 14:26:04.871612 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.791142 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/90d2dce1-be42-4595-b9c4-74a024eef0e9-host-kubelet\") pod \"ovnkube-node-scfg4\" (UID: \"90d2dce1-be42-4595-b9c4-74a024eef0e9\") " pod="openshift-ovn-kubernetes/ovnkube-node-scfg4" Jul 07 14:26:04.871612 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.791173 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/90d2dce1-be42-4595-b9c4-74a024eef0e9-ovnkube-config\") pod \"ovnkube-node-scfg4\" (UID: \"90d2dce1-be42-4595-b9c4-74a024eef0e9\") " pod="openshift-ovn-kubernetes/ovnkube-node-scfg4" Jul 07 14:26:04.871612 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.791214 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/0b70b350-feef-419b-91c0-45a22ae7f30b-serviceca\") pod \"node-ca-5g7bh\" (UID: \"0b70b350-feef-419b-91c0-45a22ae7f30b\") " pod="openshift-image-registry/node-ca-5g7bh" Jul 07 14:26:04.871612 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.791251 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/01caeef5-2981-41b4-92c9-75a39f813595-run\") pod \"tuned-pwmxs\" (UID: \"01caeef5-2981-41b4-92c9-75a39f813595\") " pod="openshift-cluster-node-tuning-operator/tuned-pwmxs" Jul 07 14:26:04.871612 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.791270 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/9465f33d-a02b-4cd7-8c55-d30fe0648034-host-var-lib-cni-multus\") pod \"multus-42n8m\" (UID: \"9465f33d-a02b-4cd7-8c55-d30fe0648034\") " pod="openshift-multus/multus-42n8m" Jul 07 14:26:04.871612 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.791311 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/6c88a149-4504-482b-90e7-87d3065096c2-agent-certs\") pod \"konnectivity-agent-2m5td\" (UID: \"6c88a149-4504-482b-90e7-87d3065096c2\") " pod="kube-system/konnectivity-agent-2m5td" Jul 07 14:26:04.871612 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.791346 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/90d2dce1-be42-4595-b9c4-74a024eef0e9-node-log\") pod \"ovnkube-node-scfg4\" (UID: \"90d2dce1-be42-4595-b9c4-74a024eef0e9\") " pod="openshift-ovn-kubernetes/ovnkube-node-scfg4" Jul 07 14:26:04.872744 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.791367 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9465f33d-a02b-4cd7-8c55-d30fe0648034-system-cni-dir\") pod \"multus-42n8m\" (UID: \"9465f33d-a02b-4cd7-8c55-d30fe0648034\") " pod="openshift-multus/multus-42n8m" Jul 07 14:26:04.872744 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.791401 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/90d2dce1-be42-4595-b9c4-74a024eef0e9-host-run-netns\") pod \"ovnkube-node-scfg4\" (UID: \"90d2dce1-be42-4595-b9c4-74a024eef0e9\") " pod="openshift-ovn-kubernetes/ovnkube-node-scfg4" Jul 07 14:26:04.872744 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.791422 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/01caeef5-2981-41b4-92c9-75a39f813595-etc-tuned\") pod \"tuned-pwmxs\" (UID: \"01caeef5-2981-41b4-92c9-75a39f813595\") " pod="openshift-cluster-node-tuning-operator/tuned-pwmxs" Jul 07 14:26:04.872744 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.791445 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9465f33d-a02b-4cd7-8c55-d30fe0648034-multus-cni-dir\") pod \"multus-42n8m\" (UID: \"9465f33d-a02b-4cd7-8c55-d30fe0648034\") " pod="openshift-multus/multus-42n8m" Jul 07 14:26:04.872744 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.791464 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g9z6j\" (UniqueName: \"kubernetes.io/projected/3f8eb048-9150-421e-8359-e64f3aec2211-kube-api-access-g9z6j\") pod \"aws-ebs-csi-driver-node-zh4rc\" (UID: \"3f8eb048-9150-421e-8359-e64f3aec2211\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zh4rc" Jul 07 14:26:04.872744 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.791481 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/90d2dce1-be42-4595-b9c4-74a024eef0e9-run-ovn\") pod \"ovnkube-node-scfg4\" (UID: \"90d2dce1-be42-4595-b9c4-74a024eef0e9\") " pod="openshift-ovn-kubernetes/ovnkube-node-scfg4" Jul 07 14:26:04.872744 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.791494 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/90d2dce1-be42-4595-b9c4-74a024eef0e9-log-socket\") pod \"ovnkube-node-scfg4\" (UID: \"90d2dce1-be42-4595-b9c4-74a024eef0e9\") " pod="openshift-ovn-kubernetes/ovnkube-node-scfg4" Jul 07 14:26:04.872744 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.791515 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/fa56eabc-b99d-4ce2-9f36-cb0242168cf4-iptables-alerter-script\") pod \"iptables-alerter-rgvqn\" (UID: \"fa56eabc-b99d-4ce2-9f36-cb0242168cf4\") " pod="openshift-network-operator/iptables-alerter-rgvqn" Jul 07 14:26:04.872744 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.791554 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/01caeef5-2981-41b4-92c9-75a39f813595-etc-sysctl-conf\") pod \"tuned-pwmxs\" (UID: \"01caeef5-2981-41b4-92c9-75a39f813595\") " pod="openshift-cluster-node-tuning-operator/tuned-pwmxs" Jul 07 14:26:04.872744 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.791587 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/01caeef5-2981-41b4-92c9-75a39f813595-sys\") pod \"tuned-pwmxs\" (UID: \"01caeef5-2981-41b4-92c9-75a39f813595\") " pod="openshift-cluster-node-tuning-operator/tuned-pwmxs" Jul 07 14:26:04.872744 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.791602 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/01caeef5-2981-41b4-92c9-75a39f813595-lib-modules\") pod \"tuned-pwmxs\" (UID: \"01caeef5-2981-41b4-92c9-75a39f813595\") " pod="openshift-cluster-node-tuning-operator/tuned-pwmxs" Jul 07 14:26:04.872744 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.791630 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/9465f33d-a02b-4cd7-8c55-d30fe0648034-multus-socket-dir-parent\") pod \"multus-42n8m\" (UID: \"9465f33d-a02b-4cd7-8c55-d30fe0648034\") " pod="openshift-multus/multus-42n8m" Jul 07 14:26:04.872744 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.791651 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/ce774983-4cbb-4526-ad84-72478efce781-kubelet-config\") pod \"global-pull-secret-syncer-hvlzx\" (UID: \"ce774983-4cbb-4526-ad84-72478efce781\") " pod="kube-system/global-pull-secret-syncer-hvlzx" Jul 07 14:26:04.872744 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.791669 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/3f8eb048-9150-421e-8359-e64f3aec2211-registration-dir\") pod \"aws-ebs-csi-driver-node-zh4rc\" (UID: \"3f8eb048-9150-421e-8359-e64f3aec2211\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zh4rc" Jul 07 14:26:04.873284 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.791683 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d3b76148-a347-4b4e-9ba5-fb4c41ebc389-system-cni-dir\") pod \"multus-additional-cni-plugins-shb4c\" (UID: \"d3b76148-a347-4b4e-9ba5-fb4c41ebc389\") " pod="openshift-multus/multus-additional-cni-plugins-shb4c" Jul 07 14:26:04.873284 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.791697 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/90d2dce1-be42-4595-b9c4-74a024eef0e9-systemd-units\") pod \"ovnkube-node-scfg4\" (UID: \"90d2dce1-be42-4595-b9c4-74a024eef0e9\") " pod="openshift-ovn-kubernetes/ovnkube-node-scfg4" Jul 07 14:26:04.873284 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.791721 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/9465f33d-a02b-4cd7-8c55-d30fe0648034-hostroot\") pod \"multus-42n8m\" (UID: \"9465f33d-a02b-4cd7-8c55-d30fe0648034\") " pod="openshift-multus/multus-42n8m" Jul 07 14:26:04.873284 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.791745 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/320ab15e-c868-4a37-a3e6-1cea72bb74a4-metrics-certs\") pod \"network-metrics-daemon-s4p8l\" (UID: \"320ab15e-c868-4a37-a3e6-1cea72bb74a4\") " pod="openshift-multus/network-metrics-daemon-s4p8l" Jul 07 14:26:04.873284 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.791768 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/3f8eb048-9150-421e-8359-e64f3aec2211-device-dir\") pod \"aws-ebs-csi-driver-node-zh4rc\" (UID: \"3f8eb048-9150-421e-8359-e64f3aec2211\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zh4rc" Jul 07 14:26:04.873284 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.791783 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/90d2dce1-be42-4595-b9c4-74a024eef0e9-host-cni-bin\") pod \"ovnkube-node-scfg4\" (UID: \"90d2dce1-be42-4595-b9c4-74a024eef0e9\") " pod="openshift-ovn-kubernetes/ovnkube-node-scfg4" Jul 07 14:26:04.873284 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.791797 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/90d2dce1-be42-4595-b9c4-74a024eef0e9-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-scfg4\" (UID: \"90d2dce1-be42-4595-b9c4-74a024eef0e9\") " pod="openshift-ovn-kubernetes/ovnkube-node-scfg4" Jul 07 14:26:04.873284 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.791813 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/3f8eb048-9150-421e-8359-e64f3aec2211-kubelet-dir\") pod \"aws-ebs-csi-driver-node-zh4rc\" (UID: \"3f8eb048-9150-421e-8359-e64f3aec2211\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zh4rc" Jul 07 14:26:04.873284 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.791869 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/01caeef5-2981-41b4-92c9-75a39f813595-etc-sysctl-d\") pod \"tuned-pwmxs\" (UID: \"01caeef5-2981-41b4-92c9-75a39f813595\") " pod="openshift-cluster-node-tuning-operator/tuned-pwmxs" Jul 07 14:26:04.873284 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.791894 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/01caeef5-2981-41b4-92c9-75a39f813595-etc-systemd\") pod \"tuned-pwmxs\" (UID: \"01caeef5-2981-41b4-92c9-75a39f813595\") " pod="openshift-cluster-node-tuning-operator/tuned-pwmxs" Jul 07 14:26:04.873284 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.791916 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/9465f33d-a02b-4cd7-8c55-d30fe0648034-os-release\") pod \"multus-42n8m\" (UID: \"9465f33d-a02b-4cd7-8c55-d30fe0648034\") " pod="openshift-multus/multus-42n8m" Jul 07 14:26:04.873284 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.791952 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/9465f33d-a02b-4cd7-8c55-d30fe0648034-multus-conf-dir\") pod \"multus-42n8m\" (UID: \"9465f33d-a02b-4cd7-8c55-d30fe0648034\") " pod="openshift-multus/multus-42n8m" Jul 07 14:26:04.873284 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.791977 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/3f8eb048-9150-421e-8359-e64f3aec2211-sys-fs\") pod \"aws-ebs-csi-driver-node-zh4rc\" (UID: \"3f8eb048-9150-421e-8359-e64f3aec2211\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zh4rc" Jul 07 14:26:04.873284 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.792002 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/90d2dce1-be42-4595-b9c4-74a024eef0e9-env-overrides\") pod \"ovnkube-node-scfg4\" (UID: \"90d2dce1-be42-4595-b9c4-74a024eef0e9\") " pod="openshift-ovn-kubernetes/ovnkube-node-scfg4" Jul 07 14:26:04.873970 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.792034 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/d3b76148-a347-4b4e-9ba5-fb4c41ebc389-os-release\") pod \"multus-additional-cni-plugins-shb4c\" (UID: \"d3b76148-a347-4b4e-9ba5-fb4c41ebc389\") " pod="openshift-multus/multus-additional-cni-plugins-shb4c" Jul 07 14:26:04.873970 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.792063 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/9465f33d-a02b-4cd7-8c55-d30fe0648034-host-var-lib-cni-bin\") pod \"multus-42n8m\" (UID: \"9465f33d-a02b-4cd7-8c55-d30fe0648034\") " pod="openshift-multus/multus-42n8m" Jul 07 14:26:04.873970 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.792080 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/3f8eb048-9150-421e-8359-e64f3aec2211-socket-dir\") pod \"aws-ebs-csi-driver-node-zh4rc\" (UID: \"3f8eb048-9150-421e-8359-e64f3aec2211\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zh4rc" Jul 07 14:26:04.873970 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.792095 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/90d2dce1-be42-4595-b9c4-74a024eef0e9-run-openvswitch\") pod \"ovnkube-node-scfg4\" (UID: \"90d2dce1-be42-4595-b9c4-74a024eef0e9\") " pod="openshift-ovn-kubernetes/ovnkube-node-scfg4" Jul 07 14:26:04.873970 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.792109 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/01caeef5-2981-41b4-92c9-75a39f813595-etc-kubernetes\") pod \"tuned-pwmxs\" (UID: \"01caeef5-2981-41b4-92c9-75a39f813595\") " pod="openshift-cluster-node-tuning-operator/tuned-pwmxs" Jul 07 14:26:04.873970 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.792141 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/9465f33d-a02b-4cd7-8c55-d30fe0648034-multus-daemon-config\") pod \"multus-42n8m\" (UID: \"9465f33d-a02b-4cd7-8c55-d30fe0648034\") " pod="openshift-multus/multus-42n8m" Jul 07 14:26:04.873970 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.792157 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/d3b76148-a347-4b4e-9ba5-fb4c41ebc389-tuning-conf-dir\") pod \"multus-additional-cni-plugins-shb4c\" (UID: \"d3b76148-a347-4b4e-9ba5-fb4c41ebc389\") " pod="openshift-multus/multus-additional-cni-plugins-shb4c" Jul 07 14:26:04.873970 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.792173 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/aa141f90-5856-4779-90d7-c6377b1e32d6-hosts-file\") pod \"node-resolver-wmt48\" (UID: \"aa141f90-5856-4779-90d7-c6377b1e32d6\") " pod="openshift-dns/node-resolver-wmt48" Jul 07 14:26:04.873970 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.792188 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/d3b76148-a347-4b4e-9ba5-fb4c41ebc389-cnibin\") pod \"multus-additional-cni-plugins-shb4c\" (UID: \"d3b76148-a347-4b4e-9ba5-fb4c41ebc389\") " pod="openshift-multus/multus-additional-cni-plugins-shb4c" Jul 07 14:26:04.873970 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.792205 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/01caeef5-2981-41b4-92c9-75a39f813595-var-lib-kubelet\") pod \"tuned-pwmxs\" (UID: \"01caeef5-2981-41b4-92c9-75a39f813595\") " pod="openshift-cluster-node-tuning-operator/tuned-pwmxs" Jul 07 14:26:04.873970 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.792220 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/01caeef5-2981-41b4-92c9-75a39f813595-tmp\") pod \"tuned-pwmxs\" (UID: \"01caeef5-2981-41b4-92c9-75a39f813595\") " pod="openshift-cluster-node-tuning-operator/tuned-pwmxs" Jul 07 14:26:04.873970 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.792235 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9465f33d-a02b-4cd7-8c55-d30fe0648034-host-var-lib-kubelet\") pod \"multus-42n8m\" (UID: \"9465f33d-a02b-4cd7-8c55-d30fe0648034\") " pod="openshift-multus/multus-42n8m" Jul 07 14:26:04.873970 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.792254 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/90d2dce1-be42-4595-b9c4-74a024eef0e9-ovn-node-metrics-cert\") pod \"ovnkube-node-scfg4\" (UID: \"90d2dce1-be42-4595-b9c4-74a024eef0e9\") " pod="openshift-ovn-kubernetes/ovnkube-node-scfg4" Jul 07 14:26:04.873970 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.792269 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gxlbb\" (UniqueName: \"kubernetes.io/projected/90d2dce1-be42-4595-b9c4-74a024eef0e9-kube-api-access-gxlbb\") pod \"ovnkube-node-scfg4\" (UID: \"90d2dce1-be42-4595-b9c4-74a024eef0e9\") " pod="openshift-ovn-kubernetes/ovnkube-node-scfg4" Jul 07 14:26:04.874416 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.820356 2557 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-9vn6j" Jul 07 14:26:04.874416 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.826894 2557 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-9vn6j" Jul 07 14:26:04.992284 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.892721 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xf42z\" (UniqueName: \"kubernetes.io/projected/fa56eabc-b99d-4ce2-9f36-cb0242168cf4-kube-api-access-xf42z\") pod \"iptables-alerter-rgvqn\" (UID: \"fa56eabc-b99d-4ce2-9f36-cb0242168cf4\") " pod="openshift-network-operator/iptables-alerter-rgvqn" Jul 07 14:26:04.992284 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.892748 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/3f8eb048-9150-421e-8359-e64f3aec2211-etc-selinux\") pod \"aws-ebs-csi-driver-node-zh4rc\" (UID: \"3f8eb048-9150-421e-8359-e64f3aec2211\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zh4rc" Jul 07 14:26:04.992284 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.892774 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q5g9n\" (UniqueName: \"kubernetes.io/projected/09a0edd9-f556-490d-a691-a0cb3dc3aff9-kube-api-access-q5g9n\") pod \"network-check-target-bqthq\" (UID: \"09a0edd9-f556-490d-a691-a0cb3dc3aff9\") " pod="openshift-network-diagnostics/network-check-target-bqthq" Jul 07 14:26:04.992284 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.892790 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/9465f33d-a02b-4cd7-8c55-d30fe0648034-cnibin\") pod \"multus-42n8m\" (UID: \"9465f33d-a02b-4cd7-8c55-d30fe0648034\") " pod="openshift-multus/multus-42n8m" Jul 07 14:26:04.992284 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.892846 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/3f8eb048-9150-421e-8359-e64f3aec2211-etc-selinux\") pod \"aws-ebs-csi-driver-node-zh4rc\" (UID: \"3f8eb048-9150-421e-8359-e64f3aec2211\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zh4rc" Jul 07 14:26:04.992284 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.892857 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/9465f33d-a02b-4cd7-8c55-d30fe0648034-cnibin\") pod \"multus-42n8m\" (UID: \"9465f33d-a02b-4cd7-8c55-d30fe0648034\") " pod="openshift-multus/multus-42n8m" Jul 07 14:26:04.992284 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.892849 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/9465f33d-a02b-4cd7-8c55-d30fe0648034-cni-binary-copy\") pod \"multus-42n8m\" (UID: \"9465f33d-a02b-4cd7-8c55-d30fe0648034\") " pod="openshift-multus/multus-42n8m" Jul 07 14:26:04.992284 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.892909 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/9465f33d-a02b-4cd7-8c55-d30fe0648034-host-run-multus-certs\") pod \"multus-42n8m\" (UID: \"9465f33d-a02b-4cd7-8c55-d30fe0648034\") " pod="openshift-multus/multus-42n8m" Jul 07 14:26:04.992284 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.892954 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9465f33d-a02b-4cd7-8c55-d30fe0648034-etc-kubernetes\") pod \"multus-42n8m\" (UID: \"9465f33d-a02b-4cd7-8c55-d30fe0648034\") " pod="openshift-multus/multus-42n8m" Jul 07 14:26:04.992284 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.892978 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/90d2dce1-be42-4595-b9c4-74a024eef0e9-var-lib-openvswitch\") pod \"ovnkube-node-scfg4\" (UID: \"90d2dce1-be42-4595-b9c4-74a024eef0e9\") " pod="openshift-ovn-kubernetes/ovnkube-node-scfg4" Jul 07 14:26:04.992284 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.893004 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/90d2dce1-be42-4595-b9c4-74a024eef0e9-ovnkube-script-lib\") pod \"ovnkube-node-scfg4\" (UID: \"90d2dce1-be42-4595-b9c4-74a024eef0e9\") " pod="openshift-ovn-kubernetes/ovnkube-node-scfg4" Jul 07 14:26:04.992284 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.893006 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/9465f33d-a02b-4cd7-8c55-d30fe0648034-host-run-multus-certs\") pod \"multus-42n8m\" (UID: \"9465f33d-a02b-4cd7-8c55-d30fe0648034\") " pod="openshift-multus/multus-42n8m" Jul 07 14:26:04.992284 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.893006 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9465f33d-a02b-4cd7-8c55-d30fe0648034-etc-kubernetes\") pod \"multus-42n8m\" (UID: \"9465f33d-a02b-4cd7-8c55-d30fe0648034\") " pod="openshift-multus/multus-42n8m" Jul 07 14:26:04.992284 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.893031 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/90d2dce1-be42-4595-b9c4-74a024eef0e9-var-lib-openvswitch\") pod \"ovnkube-node-scfg4\" (UID: \"90d2dce1-be42-4595-b9c4-74a024eef0e9\") " pod="openshift-ovn-kubernetes/ovnkube-node-scfg4" Jul 07 14:26:04.992284 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.893026 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/01caeef5-2981-41b4-92c9-75a39f813595-etc-sysconfig\") pod \"tuned-pwmxs\" (UID: \"01caeef5-2981-41b4-92c9-75a39f813595\") " pod="openshift-cluster-node-tuning-operator/tuned-pwmxs" Jul 07 14:26:04.993155 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.893073 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/01caeef5-2981-41b4-92c9-75a39f813595-etc-sysconfig\") pod \"tuned-pwmxs\" (UID: \"01caeef5-2981-41b4-92c9-75a39f813595\") " pod="openshift-cluster-node-tuning-operator/tuned-pwmxs" Jul 07 14:26:04.993155 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.893090 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tg5zt\" (UniqueName: \"kubernetes.io/projected/01caeef5-2981-41b4-92c9-75a39f813595-kube-api-access-tg5zt\") pod \"tuned-pwmxs\" (UID: \"01caeef5-2981-41b4-92c9-75a39f813595\") " pod="openshift-cluster-node-tuning-operator/tuned-pwmxs" Jul 07 14:26:04.993155 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.893117 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/9465f33d-a02b-4cd7-8c55-d30fe0648034-host-run-k8s-cni-cncf-io\") pod \"multus-42n8m\" (UID: \"9465f33d-a02b-4cd7-8c55-d30fe0648034\") " pod="openshift-multus/multus-42n8m" Jul 07 14:26:04.993155 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.893141 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p4z9t\" (UniqueName: \"kubernetes.io/projected/320ab15e-c868-4a37-a3e6-1cea72bb74a4-kube-api-access-p4z9t\") pod \"network-metrics-daemon-s4p8l\" (UID: \"320ab15e-c868-4a37-a3e6-1cea72bb74a4\") " pod="openshift-multus/network-metrics-daemon-s4p8l" Jul 07 14:26:04.993155 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.893164 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/6c88a149-4504-482b-90e7-87d3065096c2-konnectivity-ca\") pod \"konnectivity-agent-2m5td\" (UID: \"6c88a149-4504-482b-90e7-87d3065096c2\") " pod="kube-system/konnectivity-agent-2m5td" Jul 07 14:26:04.993155 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.893191 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ce774983-4cbb-4526-ad84-72478efce781-original-pull-secret\") pod \"global-pull-secret-syncer-hvlzx\" (UID: \"ce774983-4cbb-4526-ad84-72478efce781\") " pod="kube-system/global-pull-secret-syncer-hvlzx" Jul 07 14:26:04.993155 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.893220 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mhhjl\" (UniqueName: \"kubernetes.io/projected/d3b76148-a347-4b4e-9ba5-fb4c41ebc389-kube-api-access-mhhjl\") pod \"multus-additional-cni-plugins-shb4c\" (UID: \"d3b76148-a347-4b4e-9ba5-fb4c41ebc389\") " pod="openshift-multus/multus-additional-cni-plugins-shb4c" Jul 07 14:26:04.993155 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.893261 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-csjl2\" (UniqueName: \"kubernetes.io/projected/aa141f90-5856-4779-90d7-c6377b1e32d6-kube-api-access-csjl2\") pod \"node-resolver-wmt48\" (UID: \"aa141f90-5856-4779-90d7-c6377b1e32d6\") " pod="openshift-dns/node-resolver-wmt48" Jul 07 14:26:04.993155 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.893250 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/9465f33d-a02b-4cd7-8c55-d30fe0648034-host-run-k8s-cni-cncf-io\") pod \"multus-42n8m\" (UID: \"9465f33d-a02b-4cd7-8c55-d30fe0648034\") " pod="openshift-multus/multus-42n8m" Jul 07 14:26:04.993155 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.893297 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/9465f33d-a02b-4cd7-8c55-d30fe0648034-host-run-netns\") pod \"multus-42n8m\" (UID: \"9465f33d-a02b-4cd7-8c55-d30fe0648034\") " pod="openshift-multus/multus-42n8m" Jul 07 14:26:04.993155 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.893331 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/ce774983-4cbb-4526-ad84-72478efce781-dbus\") pod \"global-pull-secret-syncer-hvlzx\" (UID: \"ce774983-4cbb-4526-ad84-72478efce781\") " pod="kube-system/global-pull-secret-syncer-hvlzx" Jul 07 14:26:04.993155 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.893371 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/aa141f90-5856-4779-90d7-c6377b1e32d6-tmp-dir\") pod \"node-resolver-wmt48\" (UID: \"aa141f90-5856-4779-90d7-c6377b1e32d6\") " pod="openshift-dns/node-resolver-wmt48" Jul 07 14:26:04.993155 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.893433 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/90d2dce1-be42-4595-b9c4-74a024eef0e9-host-kubelet\") pod \"ovnkube-node-scfg4\" (UID: \"90d2dce1-be42-4595-b9c4-74a024eef0e9\") " pod="openshift-ovn-kubernetes/ovnkube-node-scfg4" Jul 07 14:26:04.993155 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.893450 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/9465f33d-a02b-4cd7-8c55-d30fe0648034-host-run-netns\") pod \"multus-42n8m\" (UID: \"9465f33d-a02b-4cd7-8c55-d30fe0648034\") " pod="openshift-multus/multus-42n8m" Jul 07 14:26:04.997822 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.893466 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/90d2dce1-be42-4595-b9c4-74a024eef0e9-ovnkube-config\") pod \"ovnkube-node-scfg4\" (UID: \"90d2dce1-be42-4595-b9c4-74a024eef0e9\") " pod="openshift-ovn-kubernetes/ovnkube-node-scfg4" Jul 07 14:26:04.997822 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.893499 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/0b70b350-feef-419b-91c0-45a22ae7f30b-serviceca\") pod \"node-ca-5g7bh\" (UID: \"0b70b350-feef-419b-91c0-45a22ae7f30b\") " pod="openshift-image-registry/node-ca-5g7bh" Jul 07 14:26:04.997822 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.893531 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/01caeef5-2981-41b4-92c9-75a39f813595-run\") pod \"tuned-pwmxs\" (UID: \"01caeef5-2981-41b4-92c9-75a39f813595\") " pod="openshift-cluster-node-tuning-operator/tuned-pwmxs" Jul 07 14:26:04.997822 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.893578 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/9465f33d-a02b-4cd7-8c55-d30fe0648034-host-var-lib-cni-multus\") pod \"multus-42n8m\" (UID: \"9465f33d-a02b-4cd7-8c55-d30fe0648034\") " pod="openshift-multus/multus-42n8m" Jul 07 14:26:04.997822 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.893610 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/6c88a149-4504-482b-90e7-87d3065096c2-agent-certs\") pod \"konnectivity-agent-2m5td\" (UID: \"6c88a149-4504-482b-90e7-87d3065096c2\") " pod="kube-system/konnectivity-agent-2m5td" Jul 07 14:26:04.997822 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.893592 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/ce774983-4cbb-4526-ad84-72478efce781-dbus\") pod \"global-pull-secret-syncer-hvlzx\" (UID: \"ce774983-4cbb-4526-ad84-72478efce781\") " pod="kube-system/global-pull-secret-syncer-hvlzx" Jul 07 14:26:04.997822 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.893637 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/90d2dce1-be42-4595-b9c4-74a024eef0e9-node-log\") pod \"ovnkube-node-scfg4\" (UID: \"90d2dce1-be42-4595-b9c4-74a024eef0e9\") " pod="openshift-ovn-kubernetes/ovnkube-node-scfg4" Jul 07 14:26:04.997822 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.893668 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9465f33d-a02b-4cd7-8c55-d30fe0648034-system-cni-dir\") pod \"multus-42n8m\" (UID: \"9465f33d-a02b-4cd7-8c55-d30fe0648034\") " pod="openshift-multus/multus-42n8m" Jul 07 14:26:04.997822 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.893699 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/90d2dce1-be42-4595-b9c4-74a024eef0e9-host-run-netns\") pod \"ovnkube-node-scfg4\" (UID: \"90d2dce1-be42-4595-b9c4-74a024eef0e9\") " pod="openshift-ovn-kubernetes/ovnkube-node-scfg4" Jul 07 14:26:04.997822 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.893725 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/01caeef5-2981-41b4-92c9-75a39f813595-etc-tuned\") pod \"tuned-pwmxs\" (UID: \"01caeef5-2981-41b4-92c9-75a39f813595\") " pod="openshift-cluster-node-tuning-operator/tuned-pwmxs" Jul 07 14:26:04.997822 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.893753 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9465f33d-a02b-4cd7-8c55-d30fe0648034-multus-cni-dir\") pod \"multus-42n8m\" (UID: \"9465f33d-a02b-4cd7-8c55-d30fe0648034\") " pod="openshift-multus/multus-42n8m" Jul 07 14:26:04.997822 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.893779 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g9z6j\" (UniqueName: \"kubernetes.io/projected/3f8eb048-9150-421e-8359-e64f3aec2211-kube-api-access-g9z6j\") pod \"aws-ebs-csi-driver-node-zh4rc\" (UID: \"3f8eb048-9150-421e-8359-e64f3aec2211\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zh4rc" Jul 07 14:26:04.997822 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.893805 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/90d2dce1-be42-4595-b9c4-74a024eef0e9-run-ovn\") pod \"ovnkube-node-scfg4\" (UID: \"90d2dce1-be42-4595-b9c4-74a024eef0e9\") " pod="openshift-ovn-kubernetes/ovnkube-node-scfg4" Jul 07 14:26:04.997822 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.893830 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/90d2dce1-be42-4595-b9c4-74a024eef0e9-log-socket\") pod \"ovnkube-node-scfg4\" (UID: \"90d2dce1-be42-4595-b9c4-74a024eef0e9\") " pod="openshift-ovn-kubernetes/ovnkube-node-scfg4" Jul 07 14:26:04.997822 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.893862 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/fa56eabc-b99d-4ce2-9f36-cb0242168cf4-iptables-alerter-script\") pod \"iptables-alerter-rgvqn\" (UID: \"fa56eabc-b99d-4ce2-9f36-cb0242168cf4\") " pod="openshift-network-operator/iptables-alerter-rgvqn" Jul 07 14:26:04.998763 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.893897 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/01caeef5-2981-41b4-92c9-75a39f813595-etc-sysctl-conf\") pod \"tuned-pwmxs\" (UID: \"01caeef5-2981-41b4-92c9-75a39f813595\") " pod="openshift-cluster-node-tuning-operator/tuned-pwmxs" Jul 07 14:26:04.998763 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.893926 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/01caeef5-2981-41b4-92c9-75a39f813595-sys\") pod \"tuned-pwmxs\" (UID: \"01caeef5-2981-41b4-92c9-75a39f813595\") " pod="openshift-cluster-node-tuning-operator/tuned-pwmxs" Jul 07 14:26:04.998763 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.893959 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/01caeef5-2981-41b4-92c9-75a39f813595-lib-modules\") pod \"tuned-pwmxs\" (UID: \"01caeef5-2981-41b4-92c9-75a39f813595\") " pod="openshift-cluster-node-tuning-operator/tuned-pwmxs" Jul 07 14:26:04.998763 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.893994 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/9465f33d-a02b-4cd7-8c55-d30fe0648034-multus-socket-dir-parent\") pod \"multus-42n8m\" (UID: \"9465f33d-a02b-4cd7-8c55-d30fe0648034\") " pod="openshift-multus/multus-42n8m" Jul 07 14:26:04.998763 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.894015 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/90d2dce1-be42-4595-b9c4-74a024eef0e9-host-kubelet\") pod \"ovnkube-node-scfg4\" (UID: \"90d2dce1-be42-4595-b9c4-74a024eef0e9\") " pod="openshift-ovn-kubernetes/ovnkube-node-scfg4" Jul 07 14:26:04.998763 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.894023 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/ce774983-4cbb-4526-ad84-72478efce781-kubelet-config\") pod \"global-pull-secret-syncer-hvlzx\" (UID: \"ce774983-4cbb-4526-ad84-72478efce781\") " pod="kube-system/global-pull-secret-syncer-hvlzx" Jul 07 14:26:04.998763 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.894049 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/3f8eb048-9150-421e-8359-e64f3aec2211-registration-dir\") pod \"aws-ebs-csi-driver-node-zh4rc\" (UID: \"3f8eb048-9150-421e-8359-e64f3aec2211\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zh4rc" Jul 07 14:26:04.998763 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.894066 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/6c88a149-4504-482b-90e7-87d3065096c2-konnectivity-ca\") pod \"konnectivity-agent-2m5td\" (UID: \"6c88a149-4504-482b-90e7-87d3065096c2\") " pod="kube-system/konnectivity-agent-2m5td" Jul 07 14:26:04.998763 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.894078 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d3b76148-a347-4b4e-9ba5-fb4c41ebc389-system-cni-dir\") pod \"multus-additional-cni-plugins-shb4c\" (UID: \"d3b76148-a347-4b4e-9ba5-fb4c41ebc389\") " pod="openshift-multus/multus-additional-cni-plugins-shb4c" Jul 07 14:26:04.998763 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.894105 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/90d2dce1-be42-4595-b9c4-74a024eef0e9-systemd-units\") pod \"ovnkube-node-scfg4\" (UID: \"90d2dce1-be42-4595-b9c4-74a024eef0e9\") " pod="openshift-ovn-kubernetes/ovnkube-node-scfg4" Jul 07 14:26:04.998763 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.894113 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/0b70b350-feef-419b-91c0-45a22ae7f30b-serviceca\") pod \"node-ca-5g7bh\" (UID: \"0b70b350-feef-419b-91c0-45a22ae7f30b\") " pod="openshift-image-registry/node-ca-5g7bh" Jul 07 14:26:04.998763 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.894139 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/9465f33d-a02b-4cd7-8c55-d30fe0648034-hostroot\") pod \"multus-42n8m\" (UID: \"9465f33d-a02b-4cd7-8c55-d30fe0648034\") " pod="openshift-multus/multus-42n8m" Jul 07 14:26:04.998763 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.894168 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/320ab15e-c868-4a37-a3e6-1cea72bb74a4-metrics-certs\") pod \"network-metrics-daemon-s4p8l\" (UID: \"320ab15e-c868-4a37-a3e6-1cea72bb74a4\") " pod="openshift-multus/network-metrics-daemon-s4p8l" Jul 07 14:26:04.998763 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.894171 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/90d2dce1-be42-4595-b9c4-74a024eef0e9-run-ovn\") pod \"ovnkube-node-scfg4\" (UID: \"90d2dce1-be42-4595-b9c4-74a024eef0e9\") " pod="openshift-ovn-kubernetes/ovnkube-node-scfg4" Jul 07 14:26:04.998763 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.894180 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/90d2dce1-be42-4595-b9c4-74a024eef0e9-ovnkube-script-lib\") pod \"ovnkube-node-scfg4\" (UID: \"90d2dce1-be42-4595-b9c4-74a024eef0e9\") " pod="openshift-ovn-kubernetes/ovnkube-node-scfg4" Jul 07 14:26:05.000393 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.894171 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/01caeef5-2981-41b4-92c9-75a39f813595-run\") pod \"tuned-pwmxs\" (UID: \"01caeef5-2981-41b4-92c9-75a39f813595\") " pod="openshift-cluster-node-tuning-operator/tuned-pwmxs" Jul 07 14:26:05.000393 ip-10-0-134-42 kubenswrapper[2557]: E0707 14:26:04.894193 2557 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 07 14:26:05.000393 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.894213 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/3f8eb048-9150-421e-8359-e64f3aec2211-device-dir\") pod \"aws-ebs-csi-driver-node-zh4rc\" (UID: \"3f8eb048-9150-421e-8359-e64f3aec2211\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zh4rc" Jul 07 14:26:05.000393 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.894242 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/90d2dce1-be42-4595-b9c4-74a024eef0e9-host-cni-bin\") pod \"ovnkube-node-scfg4\" (UID: \"90d2dce1-be42-4595-b9c4-74a024eef0e9\") " pod="openshift-ovn-kubernetes/ovnkube-node-scfg4" Jul 07 14:26:05.000393 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.894245 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/9465f33d-a02b-4cd7-8c55-d30fe0648034-host-var-lib-cni-multus\") pod \"multus-42n8m\" (UID: \"9465f33d-a02b-4cd7-8c55-d30fe0648034\") " pod="openshift-multus/multus-42n8m" Jul 07 14:26:05.000393 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.894273 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/90d2dce1-be42-4595-b9c4-74a024eef0e9-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-scfg4\" (UID: \"90d2dce1-be42-4595-b9c4-74a024eef0e9\") " pod="openshift-ovn-kubernetes/ovnkube-node-scfg4" Jul 07 14:26:05.000393 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.894296 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/90d2dce1-be42-4595-b9c4-74a024eef0e9-host-run-netns\") pod \"ovnkube-node-scfg4\" (UID: \"90d2dce1-be42-4595-b9c4-74a024eef0e9\") " pod="openshift-ovn-kubernetes/ovnkube-node-scfg4" Jul 07 14:26:05.000393 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.894307 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/3f8eb048-9150-421e-8359-e64f3aec2211-kubelet-dir\") pod \"aws-ebs-csi-driver-node-zh4rc\" (UID: \"3f8eb048-9150-421e-8359-e64f3aec2211\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zh4rc" Jul 07 14:26:05.000393 ip-10-0-134-42 kubenswrapper[2557]: E0707 14:26:04.894342 2557 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 14:26:05.000393 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.894388 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/90d2dce1-be42-4595-b9c4-74a024eef0e9-host-cni-bin\") pod \"ovnkube-node-scfg4\" (UID: \"90d2dce1-be42-4595-b9c4-74a024eef0e9\") " pod="openshift-ovn-kubernetes/ovnkube-node-scfg4" Jul 07 14:26:05.000393 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.894395 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/3f8eb048-9150-421e-8359-e64f3aec2211-kubelet-dir\") pod \"aws-ebs-csi-driver-node-zh4rc\" (UID: \"3f8eb048-9150-421e-8359-e64f3aec2211\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zh4rc" Jul 07 14:26:05.000393 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.894338 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/aa141f90-5856-4779-90d7-c6377b1e32d6-tmp-dir\") pod \"node-resolver-wmt48\" (UID: \"aa141f90-5856-4779-90d7-c6377b1e32d6\") " pod="openshift-dns/node-resolver-wmt48" Jul 07 14:26:05.000393 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.894438 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/90d2dce1-be42-4595-b9c4-74a024eef0e9-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-scfg4\" (UID: \"90d2dce1-be42-4595-b9c4-74a024eef0e9\") " pod="openshift-ovn-kubernetes/ovnkube-node-scfg4" Jul 07 14:26:05.000393 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.894548 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/01caeef5-2981-41b4-92c9-75a39f813595-etc-sysctl-d\") pod \"tuned-pwmxs\" (UID: \"01caeef5-2981-41b4-92c9-75a39f813595\") " pod="openshift-cluster-node-tuning-operator/tuned-pwmxs" Jul 07 14:26:05.000393 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.894582 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/3f8eb048-9150-421e-8359-e64f3aec2211-device-dir\") pod \"aws-ebs-csi-driver-node-zh4rc\" (UID: \"3f8eb048-9150-421e-8359-e64f3aec2211\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zh4rc" Jul 07 14:26:05.001019 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.894620 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/90d2dce1-be42-4595-b9c4-74a024eef0e9-node-log\") pod \"ovnkube-node-scfg4\" (UID: \"90d2dce1-be42-4595-b9c4-74a024eef0e9\") " pod="openshift-ovn-kubernetes/ovnkube-node-scfg4" Jul 07 14:26:05.001019 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.894338 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/01caeef5-2981-41b4-92c9-75a39f813595-etc-sysctl-d\") pod \"tuned-pwmxs\" (UID: \"01caeef5-2981-41b4-92c9-75a39f813595\") " pod="openshift-cluster-node-tuning-operator/tuned-pwmxs" Jul 07 14:26:05.001019 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.894619 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/90d2dce1-be42-4595-b9c4-74a024eef0e9-ovnkube-config\") pod \"ovnkube-node-scfg4\" (UID: \"90d2dce1-be42-4595-b9c4-74a024eef0e9\") " pod="openshift-ovn-kubernetes/ovnkube-node-scfg4" Jul 07 14:26:05.001019 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.894640 2557 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 07 14:26:05.001019 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.894696 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9465f33d-a02b-4cd7-8c55-d30fe0648034-system-cni-dir\") pod \"multus-42n8m\" (UID: \"9465f33d-a02b-4cd7-8c55-d30fe0648034\") " pod="openshift-multus/multus-42n8m" Jul 07 14:26:05.001019 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.894750 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/90d2dce1-be42-4595-b9c4-74a024eef0e9-log-socket\") pod \"ovnkube-node-scfg4\" (UID: \"90d2dce1-be42-4595-b9c4-74a024eef0e9\") " pod="openshift-ovn-kubernetes/ovnkube-node-scfg4" Jul 07 14:26:05.001019 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.894765 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/01caeef5-2981-41b4-92c9-75a39f813595-etc-sysctl-conf\") pod \"tuned-pwmxs\" (UID: \"01caeef5-2981-41b4-92c9-75a39f813595\") " pod="openshift-cluster-node-tuning-operator/tuned-pwmxs" Jul 07 14:26:05.001019 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.895101 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/90d2dce1-be42-4595-b9c4-74a024eef0e9-systemd-units\") pod \"ovnkube-node-scfg4\" (UID: \"90d2dce1-be42-4595-b9c4-74a024eef0e9\") " pod="openshift-ovn-kubernetes/ovnkube-node-scfg4" Jul 07 14:26:05.001019 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.894794 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d3b76148-a347-4b4e-9ba5-fb4c41ebc389-system-cni-dir\") pod \"multus-additional-cni-plugins-shb4c\" (UID: \"d3b76148-a347-4b4e-9ba5-fb4c41ebc389\") " pod="openshift-multus/multus-additional-cni-plugins-shb4c" Jul 07 14:26:05.001019 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.894780 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/3f8eb048-9150-421e-8359-e64f3aec2211-registration-dir\") pod \"aws-ebs-csi-driver-node-zh4rc\" (UID: \"3f8eb048-9150-421e-8359-e64f3aec2211\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zh4rc" Jul 07 14:26:05.001019 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.894802 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/fa56eabc-b99d-4ce2-9f36-cb0242168cf4-iptables-alerter-script\") pod \"iptables-alerter-rgvqn\" (UID: \"fa56eabc-b99d-4ce2-9f36-cb0242168cf4\") " pod="openshift-network-operator/iptables-alerter-rgvqn" Jul 07 14:26:05.001019 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.894814 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/ce774983-4cbb-4526-ad84-72478efce781-kubelet-config\") pod \"global-pull-secret-syncer-hvlzx\" (UID: \"ce774983-4cbb-4526-ad84-72478efce781\") " pod="kube-system/global-pull-secret-syncer-hvlzx" Jul 07 14:26:05.001019 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.895288 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/01caeef5-2981-41b4-92c9-75a39f813595-lib-modules\") pod \"tuned-pwmxs\" (UID: \"01caeef5-2981-41b4-92c9-75a39f813595\") " pod="openshift-cluster-node-tuning-operator/tuned-pwmxs" Jul 07 14:26:05.001019 ip-10-0-134-42 kubenswrapper[2557]: E0707 14:26:04.894574 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ce774983-4cbb-4526-ad84-72478efce781-original-pull-secret podName:ce774983-4cbb-4526-ad84-72478efce781 nodeName:}" failed. No retries permitted until 2026-07-07 14:26:05.394507379 +0000 UTC m=+2.103645761 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ce774983-4cbb-4526-ad84-72478efce781-original-pull-secret") pod "global-pull-secret-syncer-hvlzx" (UID: "ce774983-4cbb-4526-ad84-72478efce781") : object "kube-system"/"original-pull-secret" not registered Jul 07 14:26:05.002138 ip-10-0-134-42 kubenswrapper[2557]: E0707 14:26:04.895411 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/320ab15e-c868-4a37-a3e6-1cea72bb74a4-metrics-certs podName:320ab15e-c868-4a37-a3e6-1cea72bb74a4 nodeName:}" failed. No retries permitted until 2026-07-07 14:26:05.395386462 +0000 UTC m=+2.104524826 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/320ab15e-c868-4a37-a3e6-1cea72bb74a4-metrics-certs") pod "network-metrics-daemon-s4p8l" (UID: "320ab15e-c868-4a37-a3e6-1cea72bb74a4") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 14:26:05.002138 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.895441 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/9465f33d-a02b-4cd7-8c55-d30fe0648034-cni-binary-copy\") pod \"multus-42n8m\" (UID: \"9465f33d-a02b-4cd7-8c55-d30fe0648034\") " pod="openshift-multus/multus-42n8m" Jul 07 14:26:05.002138 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.895445 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/01caeef5-2981-41b4-92c9-75a39f813595-etc-systemd\") pod \"tuned-pwmxs\" (UID: \"01caeef5-2981-41b4-92c9-75a39f813595\") " pod="openshift-cluster-node-tuning-operator/tuned-pwmxs" Jul 07 14:26:05.002138 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.895508 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/01caeef5-2981-41b4-92c9-75a39f813595-etc-systemd\") pod \"tuned-pwmxs\" (UID: \"01caeef5-2981-41b4-92c9-75a39f813595\") " pod="openshift-cluster-node-tuning-operator/tuned-pwmxs" Jul 07 14:26:05.002138 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.895520 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/01caeef5-2981-41b4-92c9-75a39f813595-sys\") pod \"tuned-pwmxs\" (UID: \"01caeef5-2981-41b4-92c9-75a39f813595\") " pod="openshift-cluster-node-tuning-operator/tuned-pwmxs" Jul 07 14:26:05.002138 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.895559 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/9465f33d-a02b-4cd7-8c55-d30fe0648034-hostroot\") pod \"multus-42n8m\" (UID: \"9465f33d-a02b-4cd7-8c55-d30fe0648034\") " pod="openshift-multus/multus-42n8m" Jul 07 14:26:05.002138 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.895607 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/9465f33d-a02b-4cd7-8c55-d30fe0648034-os-release\") pod \"multus-42n8m\" (UID: \"9465f33d-a02b-4cd7-8c55-d30fe0648034\") " pod="openshift-multus/multus-42n8m" Jul 07 14:26:05.002138 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.895637 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/9465f33d-a02b-4cd7-8c55-d30fe0648034-multus-conf-dir\") pod \"multus-42n8m\" (UID: \"9465f33d-a02b-4cd7-8c55-d30fe0648034\") " pod="openshift-multus/multus-42n8m" Jul 07 14:26:05.002138 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.895664 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/3f8eb048-9150-421e-8359-e64f3aec2211-sys-fs\") pod \"aws-ebs-csi-driver-node-zh4rc\" (UID: \"3f8eb048-9150-421e-8359-e64f3aec2211\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zh4rc" Jul 07 14:26:05.002138 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.895669 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9465f33d-a02b-4cd7-8c55-d30fe0648034-multus-cni-dir\") pod \"multus-42n8m\" (UID: \"9465f33d-a02b-4cd7-8c55-d30fe0648034\") " pod="openshift-multus/multus-42n8m" Jul 07 14:26:05.002138 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.895690 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/90d2dce1-be42-4595-b9c4-74a024eef0e9-env-overrides\") pod \"ovnkube-node-scfg4\" (UID: \"90d2dce1-be42-4595-b9c4-74a024eef0e9\") " pod="openshift-ovn-kubernetes/ovnkube-node-scfg4" Jul 07 14:26:05.002138 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.895748 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/d3b76148-a347-4b4e-9ba5-fb4c41ebc389-os-release\") pod \"multus-additional-cni-plugins-shb4c\" (UID: \"d3b76148-a347-4b4e-9ba5-fb4c41ebc389\") " pod="openshift-multus/multus-additional-cni-plugins-shb4c" Jul 07 14:26:05.002138 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.895781 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/9465f33d-a02b-4cd7-8c55-d30fe0648034-host-var-lib-cni-bin\") pod \"multus-42n8m\" (UID: \"9465f33d-a02b-4cd7-8c55-d30fe0648034\") " pod="openshift-multus/multus-42n8m" Jul 07 14:26:05.002138 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.895810 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/3f8eb048-9150-421e-8359-e64f3aec2211-socket-dir\") pod \"aws-ebs-csi-driver-node-zh4rc\" (UID: \"3f8eb048-9150-421e-8359-e64f3aec2211\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zh4rc" Jul 07 14:26:05.006279 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.895836 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/90d2dce1-be42-4595-b9c4-74a024eef0e9-run-openvswitch\") pod \"ovnkube-node-scfg4\" (UID: \"90d2dce1-be42-4595-b9c4-74a024eef0e9\") " pod="openshift-ovn-kubernetes/ovnkube-node-scfg4" Jul 07 14:26:05.006279 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.895871 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/90d2dce1-be42-4595-b9c4-74a024eef0e9-run-openvswitch\") pod \"ovnkube-node-scfg4\" (UID: \"90d2dce1-be42-4595-b9c4-74a024eef0e9\") " pod="openshift-ovn-kubernetes/ovnkube-node-scfg4" Jul 07 14:26:05.006279 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.895899 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/01caeef5-2981-41b4-92c9-75a39f813595-etc-kubernetes\") pod \"tuned-pwmxs\" (UID: \"01caeef5-2981-41b4-92c9-75a39f813595\") " pod="openshift-cluster-node-tuning-operator/tuned-pwmxs" Jul 07 14:26:05.006279 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.895927 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/9465f33d-a02b-4cd7-8c55-d30fe0648034-multus-socket-dir-parent\") pod \"multus-42n8m\" (UID: \"9465f33d-a02b-4cd7-8c55-d30fe0648034\") " pod="openshift-multus/multus-42n8m" Jul 07 14:26:05.006279 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.895931 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/9465f33d-a02b-4cd7-8c55-d30fe0648034-multus-daemon-config\") pod \"multus-42n8m\" (UID: \"9465f33d-a02b-4cd7-8c55-d30fe0648034\") " pod="openshift-multus/multus-42n8m" Jul 07 14:26:05.006279 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.895967 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/d3b76148-a347-4b4e-9ba5-fb4c41ebc389-tuning-conf-dir\") pod \"multus-additional-cni-plugins-shb4c\" (UID: \"d3b76148-a347-4b4e-9ba5-fb4c41ebc389\") " pod="openshift-multus/multus-additional-cni-plugins-shb4c" Jul 07 14:26:05.006279 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.895991 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/01caeef5-2981-41b4-92c9-75a39f813595-etc-kubernetes\") pod \"tuned-pwmxs\" (UID: \"01caeef5-2981-41b4-92c9-75a39f813595\") " pod="openshift-cluster-node-tuning-operator/tuned-pwmxs" Jul 07 14:26:05.006279 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.895996 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/aa141f90-5856-4779-90d7-c6377b1e32d6-hosts-file\") pod \"node-resolver-wmt48\" (UID: \"aa141f90-5856-4779-90d7-c6377b1e32d6\") " pod="openshift-dns/node-resolver-wmt48" Jul 07 14:26:05.006279 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.896025 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/d3b76148-a347-4b4e-9ba5-fb4c41ebc389-cnibin\") pod \"multus-additional-cni-plugins-shb4c\" (UID: \"d3b76148-a347-4b4e-9ba5-fb4c41ebc389\") " pod="openshift-multus/multus-additional-cni-plugins-shb4c" Jul 07 14:26:05.006279 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.896053 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/01caeef5-2981-41b4-92c9-75a39f813595-var-lib-kubelet\") pod \"tuned-pwmxs\" (UID: \"01caeef5-2981-41b4-92c9-75a39f813595\") " pod="openshift-cluster-node-tuning-operator/tuned-pwmxs" Jul 07 14:26:05.006279 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.896054 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/aa141f90-5856-4779-90d7-c6377b1e32d6-hosts-file\") pod \"node-resolver-wmt48\" (UID: \"aa141f90-5856-4779-90d7-c6377b1e32d6\") " pod="openshift-dns/node-resolver-wmt48" Jul 07 14:26:05.006279 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.896112 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/d3b76148-a347-4b4e-9ba5-fb4c41ebc389-os-release\") pod \"multus-additional-cni-plugins-shb4c\" (UID: \"d3b76148-a347-4b4e-9ba5-fb4c41ebc389\") " pod="openshift-multus/multus-additional-cni-plugins-shb4c" Jul 07 14:26:05.006279 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.896150 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/01caeef5-2981-41b4-92c9-75a39f813595-tmp\") pod \"tuned-pwmxs\" (UID: \"01caeef5-2981-41b4-92c9-75a39f813595\") " pod="openshift-cluster-node-tuning-operator/tuned-pwmxs" Jul 07 14:26:05.006279 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.896187 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/d3b76148-a347-4b4e-9ba5-fb4c41ebc389-cnibin\") pod \"multus-additional-cni-plugins-shb4c\" (UID: \"d3b76148-a347-4b4e-9ba5-fb4c41ebc389\") " pod="openshift-multus/multus-additional-cni-plugins-shb4c" Jul 07 14:26:05.006279 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.896220 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9465f33d-a02b-4cd7-8c55-d30fe0648034-host-var-lib-kubelet\") pod \"multus-42n8m\" (UID: \"9465f33d-a02b-4cd7-8c55-d30fe0648034\") " pod="openshift-multus/multus-42n8m" Jul 07 14:26:05.006945 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.896301 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/9465f33d-a02b-4cd7-8c55-d30fe0648034-host-var-lib-cni-bin\") pod \"multus-42n8m\" (UID: \"9465f33d-a02b-4cd7-8c55-d30fe0648034\") " pod="openshift-multus/multus-42n8m" Jul 07 14:26:05.006945 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.896334 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/90d2dce1-be42-4595-b9c4-74a024eef0e9-ovn-node-metrics-cert\") pod \"ovnkube-node-scfg4\" (UID: \"90d2dce1-be42-4595-b9c4-74a024eef0e9\") " pod="openshift-ovn-kubernetes/ovnkube-node-scfg4" Jul 07 14:26:05.006945 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.896363 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gxlbb\" (UniqueName: \"kubernetes.io/projected/90d2dce1-be42-4595-b9c4-74a024eef0e9-kube-api-access-gxlbb\") pod \"ovnkube-node-scfg4\" (UID: \"90d2dce1-be42-4595-b9c4-74a024eef0e9\") " pod="openshift-ovn-kubernetes/ovnkube-node-scfg4" Jul 07 14:26:05.006945 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.896384 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/0b70b350-feef-419b-91c0-45a22ae7f30b-host\") pod \"node-ca-5g7bh\" (UID: \"0b70b350-feef-419b-91c0-45a22ae7f30b\") " pod="openshift-image-registry/node-ca-5g7bh" Jul 07 14:26:05.006945 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.896410 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hvmdz\" (UniqueName: \"kubernetes.io/projected/0b70b350-feef-419b-91c0-45a22ae7f30b-kube-api-access-hvmdz\") pod \"node-ca-5g7bh\" (UID: \"0b70b350-feef-419b-91c0-45a22ae7f30b\") " pod="openshift-image-registry/node-ca-5g7bh" Jul 07 14:26:05.006945 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.896486 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/3f8eb048-9150-421e-8359-e64f3aec2211-socket-dir\") pod \"aws-ebs-csi-driver-node-zh4rc\" (UID: \"3f8eb048-9150-421e-8359-e64f3aec2211\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zh4rc" Jul 07 14:26:05.006945 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.896436 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/01caeef5-2981-41b4-92c9-75a39f813595-etc-modprobe-d\") pod \"tuned-pwmxs\" (UID: \"01caeef5-2981-41b4-92c9-75a39f813595\") " pod="openshift-cluster-node-tuning-operator/tuned-pwmxs" Jul 07 14:26:05.006945 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.896545 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/90d2dce1-be42-4595-b9c4-74a024eef0e9-run-systemd\") pod \"ovnkube-node-scfg4\" (UID: \"90d2dce1-be42-4595-b9c4-74a024eef0e9\") " pod="openshift-ovn-kubernetes/ovnkube-node-scfg4" Jul 07 14:26:05.006945 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.896570 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/01caeef5-2981-41b4-92c9-75a39f813595-host\") pod \"tuned-pwmxs\" (UID: \"01caeef5-2981-41b4-92c9-75a39f813595\") " pod="openshift-cluster-node-tuning-operator/tuned-pwmxs" Jul 07 14:26:05.006945 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.896612 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2skqj\" (UniqueName: \"kubernetes.io/projected/9465f33d-a02b-4cd7-8c55-d30fe0648034-kube-api-access-2skqj\") pod \"multus-42n8m\" (UID: \"9465f33d-a02b-4cd7-8c55-d30fe0648034\") " pod="openshift-multus/multus-42n8m" Jul 07 14:26:05.006945 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.896639 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/d3b76148-a347-4b4e-9ba5-fb4c41ebc389-cni-binary-copy\") pod \"multus-additional-cni-plugins-shb4c\" (UID: \"d3b76148-a347-4b4e-9ba5-fb4c41ebc389\") " pod="openshift-multus/multus-additional-cni-plugins-shb4c" Jul 07 14:26:05.006945 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.896665 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/90d2dce1-be42-4595-b9c4-74a024eef0e9-host-run-ovn-kubernetes\") pod \"ovnkube-node-scfg4\" (UID: \"90d2dce1-be42-4595-b9c4-74a024eef0e9\") " pod="openshift-ovn-kubernetes/ovnkube-node-scfg4" Jul 07 14:26:05.006945 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.896690 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/90d2dce1-be42-4595-b9c4-74a024eef0e9-host-cni-netd\") pod \"ovnkube-node-scfg4\" (UID: \"90d2dce1-be42-4595-b9c4-74a024eef0e9\") " pod="openshift-ovn-kubernetes/ovnkube-node-scfg4" Jul 07 14:26:05.006945 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.896721 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/fa56eabc-b99d-4ce2-9f36-cb0242168cf4-host-slash\") pod \"iptables-alerter-rgvqn\" (UID: \"fa56eabc-b99d-4ce2-9f36-cb0242168cf4\") " pod="openshift-network-operator/iptables-alerter-rgvqn" Jul 07 14:26:05.007636 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.896789 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/d3b76148-a347-4b4e-9ba5-fb4c41ebc389-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-shb4c\" (UID: \"d3b76148-a347-4b4e-9ba5-fb4c41ebc389\") " pod="openshift-multus/multus-additional-cni-plugins-shb4c" Jul 07 14:26:05.007636 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.896816 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/d3b76148-a347-4b4e-9ba5-fb4c41ebc389-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-shb4c\" (UID: \"d3b76148-a347-4b4e-9ba5-fb4c41ebc389\") " pod="openshift-multus/multus-additional-cni-plugins-shb4c" Jul 07 14:26:05.007636 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.896842 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/90d2dce1-be42-4595-b9c4-74a024eef0e9-host-slash\") pod \"ovnkube-node-scfg4\" (UID: \"90d2dce1-be42-4595-b9c4-74a024eef0e9\") " pod="openshift-ovn-kubernetes/ovnkube-node-scfg4" Jul 07 14:26:05.007636 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.896868 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/90d2dce1-be42-4595-b9c4-74a024eef0e9-etc-openvswitch\") pod \"ovnkube-node-scfg4\" (UID: \"90d2dce1-be42-4595-b9c4-74a024eef0e9\") " pod="openshift-ovn-kubernetes/ovnkube-node-scfg4" Jul 07 14:26:05.007636 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.897108 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/90d2dce1-be42-4595-b9c4-74a024eef0e9-etc-openvswitch\") pod \"ovnkube-node-scfg4\" (UID: \"90d2dce1-be42-4595-b9c4-74a024eef0e9\") " pod="openshift-ovn-kubernetes/ovnkube-node-scfg4" Jul 07 14:26:05.007636 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.897022 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/9465f33d-a02b-4cd7-8c55-d30fe0648034-multus-daemon-config\") pod \"multus-42n8m\" (UID: \"9465f33d-a02b-4cd7-8c55-d30fe0648034\") " pod="openshift-multus/multus-42n8m" Jul 07 14:26:05.007636 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.897185 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/01caeef5-2981-41b4-92c9-75a39f813595-var-lib-kubelet\") pod \"tuned-pwmxs\" (UID: \"01caeef5-2981-41b4-92c9-75a39f813595\") " pod="openshift-cluster-node-tuning-operator/tuned-pwmxs" Jul 07 14:26:05.007636 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.897732 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/0b70b350-feef-419b-91c0-45a22ae7f30b-host\") pod \"node-ca-5g7bh\" (UID: \"0b70b350-feef-419b-91c0-45a22ae7f30b\") " pod="openshift-image-registry/node-ca-5g7bh" Jul 07 14:26:05.007636 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.898176 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/01caeef5-2981-41b4-92c9-75a39f813595-etc-modprobe-d\") pod \"tuned-pwmxs\" (UID: \"01caeef5-2981-41b4-92c9-75a39f813595\") " pod="openshift-cluster-node-tuning-operator/tuned-pwmxs" Jul 07 14:26:05.007636 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.898226 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/90d2dce1-be42-4595-b9c4-74a024eef0e9-run-systemd\") pod \"ovnkube-node-scfg4\" (UID: \"90d2dce1-be42-4595-b9c4-74a024eef0e9\") " pod="openshift-ovn-kubernetes/ovnkube-node-scfg4" Jul 07 14:26:05.007636 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.898277 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/01caeef5-2981-41b4-92c9-75a39f813595-host\") pod \"tuned-pwmxs\" (UID: \"01caeef5-2981-41b4-92c9-75a39f813595\") " pod="openshift-cluster-node-tuning-operator/tuned-pwmxs" Jul 07 14:26:05.007636 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.898360 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/90d2dce1-be42-4595-b9c4-74a024eef0e9-env-overrides\") pod \"ovnkube-node-scfg4\" (UID: \"90d2dce1-be42-4595-b9c4-74a024eef0e9\") " pod="openshift-ovn-kubernetes/ovnkube-node-scfg4" Jul 07 14:26:05.007636 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.898543 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/9465f33d-a02b-4cd7-8c55-d30fe0648034-os-release\") pod \"multus-42n8m\" (UID: \"9465f33d-a02b-4cd7-8c55-d30fe0648034\") " pod="openshift-multus/multus-42n8m" Jul 07 14:26:05.007636 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.898562 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/01caeef5-2981-41b4-92c9-75a39f813595-etc-tuned\") pod \"tuned-pwmxs\" (UID: \"01caeef5-2981-41b4-92c9-75a39f813595\") " pod="openshift-cluster-node-tuning-operator/tuned-pwmxs" Jul 07 14:26:05.007636 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.898609 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/9465f33d-a02b-4cd7-8c55-d30fe0648034-multus-conf-dir\") pod \"multus-42n8m\" (UID: \"9465f33d-a02b-4cd7-8c55-d30fe0648034\") " pod="openshift-multus/multus-42n8m" Jul 07 14:26:05.008692 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.898654 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/fa56eabc-b99d-4ce2-9f36-cb0242168cf4-host-slash\") pod \"iptables-alerter-rgvqn\" (UID: \"fa56eabc-b99d-4ce2-9f36-cb0242168cf4\") " pod="openshift-network-operator/iptables-alerter-rgvqn" Jul 07 14:26:05.008692 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.898895 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/3f8eb048-9150-421e-8359-e64f3aec2211-sys-fs\") pod \"aws-ebs-csi-driver-node-zh4rc\" (UID: \"3f8eb048-9150-421e-8359-e64f3aec2211\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zh4rc" Jul 07 14:26:05.008692 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.898939 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/90d2dce1-be42-4595-b9c4-74a024eef0e9-host-run-ovn-kubernetes\") pod \"ovnkube-node-scfg4\" (UID: \"90d2dce1-be42-4595-b9c4-74a024eef0e9\") " pod="openshift-ovn-kubernetes/ovnkube-node-scfg4" Jul 07 14:26:05.008692 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.899056 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/d3b76148-a347-4b4e-9ba5-fb4c41ebc389-cni-binary-copy\") pod \"multus-additional-cni-plugins-shb4c\" (UID: \"d3b76148-a347-4b4e-9ba5-fb4c41ebc389\") " pod="openshift-multus/multus-additional-cni-plugins-shb4c" Jul 07 14:26:05.008692 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.899091 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/90d2dce1-be42-4595-b9c4-74a024eef0e9-host-cni-netd\") pod \"ovnkube-node-scfg4\" (UID: \"90d2dce1-be42-4595-b9c4-74a024eef0e9\") " pod="openshift-ovn-kubernetes/ovnkube-node-scfg4" Jul 07 14:26:05.008692 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.899273 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/90d2dce1-be42-4595-b9c4-74a024eef0e9-host-slash\") pod \"ovnkube-node-scfg4\" (UID: \"90d2dce1-be42-4595-b9c4-74a024eef0e9\") " pod="openshift-ovn-kubernetes/ovnkube-node-scfg4" Jul 07 14:26:05.008692 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.899326 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9465f33d-a02b-4cd7-8c55-d30fe0648034-host-var-lib-kubelet\") pod \"multus-42n8m\" (UID: \"9465f33d-a02b-4cd7-8c55-d30fe0648034\") " pod="openshift-multus/multus-42n8m" Jul 07 14:26:05.008692 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.899405 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/01caeef5-2981-41b4-92c9-75a39f813595-tmp\") pod \"tuned-pwmxs\" (UID: \"01caeef5-2981-41b4-92c9-75a39f813595\") " pod="openshift-cluster-node-tuning-operator/tuned-pwmxs" Jul 07 14:26:05.008692 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.899460 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/d3b76148-a347-4b4e-9ba5-fb4c41ebc389-tuning-conf-dir\") pod \"multus-additional-cni-plugins-shb4c\" (UID: \"d3b76148-a347-4b4e-9ba5-fb4c41ebc389\") " pod="openshift-multus/multus-additional-cni-plugins-shb4c" Jul 07 14:26:05.008692 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.899685 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/d3b76148-a347-4b4e-9ba5-fb4c41ebc389-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-shb4c\" (UID: \"d3b76148-a347-4b4e-9ba5-fb4c41ebc389\") " pod="openshift-multus/multus-additional-cni-plugins-shb4c" Jul 07 14:26:05.008692 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.899719 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/6c88a149-4504-482b-90e7-87d3065096c2-agent-certs\") pod \"konnectivity-agent-2m5td\" (UID: \"6c88a149-4504-482b-90e7-87d3065096c2\") " pod="kube-system/konnectivity-agent-2m5td" Jul 07 14:26:05.008692 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.899838 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/d3b76148-a347-4b4e-9ba5-fb4c41ebc389-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-shb4c\" (UID: \"d3b76148-a347-4b4e-9ba5-fb4c41ebc389\") " pod="openshift-multus/multus-additional-cni-plugins-shb4c" Jul 07 14:26:05.008692 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.905116 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/90d2dce1-be42-4595-b9c4-74a024eef0e9-ovn-node-metrics-cert\") pod \"ovnkube-node-scfg4\" (UID: \"90d2dce1-be42-4595-b9c4-74a024eef0e9\") " pod="openshift-ovn-kubernetes/ovnkube-node-scfg4" Jul 07 14:26:05.008692 ip-10-0-134-42 kubenswrapper[2557]: E0707 14:26:04.905999 2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 07 14:26:05.008692 ip-10-0-134-42 kubenswrapper[2557]: E0707 14:26:04.906017 2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 07 14:26:05.009560 ip-10-0-134-42 kubenswrapper[2557]: E0707 14:26:04.906028 2557 projected.go:196] Error preparing data for projected volume kube-api-access-q5g9n for pod openshift-network-diagnostics/network-check-target-bqthq: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 14:26:05.009560 ip-10-0-134-42 kubenswrapper[2557]: E0707 14:26:04.906070 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/09a0edd9-f556-490d-a691-a0cb3dc3aff9-kube-api-access-q5g9n podName:09a0edd9-f556-490d-a691-a0cb3dc3aff9 nodeName:}" failed. No retries permitted until 2026-07-07 14:26:05.406058942 +0000 UTC m=+2.115197305 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-q5g9n" (UniqueName: "kubernetes.io/projected/09a0edd9-f556-490d-a691-a0cb3dc3aff9-kube-api-access-q5g9n") pod "network-check-target-bqthq" (UID: "09a0edd9-f556-490d-a691-a0cb3dc3aff9") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 14:26:05.009560 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.908766 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p4z9t\" (UniqueName: \"kubernetes.io/projected/320ab15e-c868-4a37-a3e6-1cea72bb74a4-kube-api-access-p4z9t\") pod \"network-metrics-daemon-s4p8l\" (UID: \"320ab15e-c868-4a37-a3e6-1cea72bb74a4\") " pod="openshift-multus/network-metrics-daemon-s4p8l" Jul 07 14:26:05.009560 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.909405 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xf42z\" (UniqueName: \"kubernetes.io/projected/fa56eabc-b99d-4ce2-9f36-cb0242168cf4-kube-api-access-xf42z\") pod \"iptables-alerter-rgvqn\" (UID: \"fa56eabc-b99d-4ce2-9f36-cb0242168cf4\") " pod="openshift-network-operator/iptables-alerter-rgvqn" Jul 07 14:26:05.009560 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.909475 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tg5zt\" (UniqueName: \"kubernetes.io/projected/01caeef5-2981-41b4-92c9-75a39f813595-kube-api-access-tg5zt\") pod \"tuned-pwmxs\" (UID: \"01caeef5-2981-41b4-92c9-75a39f813595\") " pod="openshift-cluster-node-tuning-operator/tuned-pwmxs" Jul 07 14:26:05.009560 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.913934 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gxlbb\" (UniqueName: \"kubernetes.io/projected/90d2dce1-be42-4595-b9c4-74a024eef0e9-kube-api-access-gxlbb\") pod \"ovnkube-node-scfg4\" (UID: \"90d2dce1-be42-4595-b9c4-74a024eef0e9\") " pod="openshift-ovn-kubernetes/ovnkube-node-scfg4" Jul 07 14:26:05.009560 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.915399 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hvmdz\" (UniqueName: \"kubernetes.io/projected/0b70b350-feef-419b-91c0-45a22ae7f30b-kube-api-access-hvmdz\") pod \"node-ca-5g7bh\" (UID: \"0b70b350-feef-419b-91c0-45a22ae7f30b\") " pod="openshift-image-registry/node-ca-5g7bh" Jul 07 14:26:05.009560 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.916402 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2skqj\" (UniqueName: \"kubernetes.io/projected/9465f33d-a02b-4cd7-8c55-d30fe0648034-kube-api-access-2skqj\") pod \"multus-42n8m\" (UID: \"9465f33d-a02b-4cd7-8c55-d30fe0648034\") " pod="openshift-multus/multus-42n8m" Jul 07 14:26:05.009560 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.917485 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-csjl2\" (UniqueName: \"kubernetes.io/projected/aa141f90-5856-4779-90d7-c6377b1e32d6-kube-api-access-csjl2\") pod \"node-resolver-wmt48\" (UID: \"aa141f90-5856-4779-90d7-c6377b1e32d6\") " pod="openshift-dns/node-resolver-wmt48" Jul 07 14:26:05.009560 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.918690 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mhhjl\" (UniqueName: \"kubernetes.io/projected/d3b76148-a347-4b4e-9ba5-fb4c41ebc389-kube-api-access-mhhjl\") pod \"multus-additional-cni-plugins-shb4c\" (UID: \"d3b76148-a347-4b4e-9ba5-fb4c41ebc389\") " pod="openshift-multus/multus-additional-cni-plugins-shb4c" Jul 07 14:26:05.009560 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.920940 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g9z6j\" (UniqueName: \"kubernetes.io/projected/3f8eb048-9150-421e-8359-e64f3aec2211-kube-api-access-g9z6j\") pod \"aws-ebs-csi-driver-node-zh4rc\" (UID: \"3f8eb048-9150-421e-8359-e64f3aec2211\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zh4rc" Jul 07 14:26:05.009560 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:04.949781 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-2m5td" Jul 07 14:26:05.009560 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:05.009275 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod61dca777ea2260edb003f493cc5637ae.slice/crio-7ce3bb33aa7c530e514c6e213d31f7f4d22b6d35752190fe2231bb1c27baddd7 WatchSource:0}: Error finding container 7ce3bb33aa7c530e514c6e213d31f7f4d22b6d35752190fe2231bb1c27baddd7: Status 404 returned error can't find the container with id 7ce3bb33aa7c530e514c6e213d31f7f4d22b6d35752190fe2231bb1c27baddd7 Jul 07 14:26:05.010374 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:05.009780 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-5g7bh" Jul 07 14:26:05.010374 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:05.009895 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6063cb059acc790b3bb3b3d99eea6099.slice/crio-408230ea114c883787f8a91ad7fc8a44f7753ff449989b88b64157e14586fe0a WatchSource:0}: Error finding container 408230ea114c883787f8a91ad7fc8a44f7753ff449989b88b64157e14586fe0a: Status 404 returned error can't find the container with id 408230ea114c883787f8a91ad7fc8a44f7753ff449989b88b64157e14586fe0a Jul 07 14:26:05.165306 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:05.014276 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-pwmxs" Jul 07 14:26:05.165306 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:05.015726 2557 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 07 14:26:05.165306 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:05.020958 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0b70b350_feef_419b_91c0_45a22ae7f30b.slice/crio-4835a0d375b056231932d0bdddab3ca59901e361f78201bdf7c1df40e048f162 WatchSource:0}: Error finding container 4835a0d375b056231932d0bdddab3ca59901e361f78201bdf7c1df40e048f162: Status 404 returned error can't find the container with id 4835a0d375b056231932d0bdddab3ca59901e361f78201bdf7c1df40e048f162 Jul 07 14:26:05.165306 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:05.021595 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod01caeef5_2981_41b4_92c9_75a39f813595.slice/crio-0f01effd072d0590f015dadbcd0f197f0cf0fe2d61e0e2b837d2398ce94717cf WatchSource:0}: Error finding container 0f01effd072d0590f015dadbcd0f197f0cf0fe2d61e0e2b837d2398ce94717cf: Status 404 returned error can't find the container with id 0f01effd072d0590f015dadbcd0f197f0cf0fe2d61e0e2b837d2398ce94717cf Jul 07 14:26:05.165306 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:05.037615 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zh4rc" Jul 07 14:26:05.165306 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:05.045747 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3f8eb048_9150_421e_8359_e64f3aec2211.slice/crio-8a3eca3502c7d8e815f44d680d24395c93fac12b93e5cff62d31a82efe8d3466 WatchSource:0}: Error finding container 8a3eca3502c7d8e815f44d680d24395c93fac12b93e5cff62d31a82efe8d3466: Status 404 returned error can't find the container with id 8a3eca3502c7d8e815f44d680d24395c93fac12b93e5cff62d31a82efe8d3466 Jul 07 14:26:05.165306 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:05.071922 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-42n8m" Jul 07 14:26:05.165306 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:05.078863 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9465f33d_a02b_4cd7_8c55_d30fe0648034.slice/crio-b318319fdfbd10d85c5e59e1c0cf3e36a497e8e7586b7dbec7c1fb6176e90bd5 WatchSource:0}: Error finding container b318319fdfbd10d85c5e59e1c0cf3e36a497e8e7586b7dbec7c1fb6176e90bd5: Status 404 returned error can't find the container with id b318319fdfbd10d85c5e59e1c0cf3e36a497e8e7586b7dbec7c1fb6176e90bd5 Jul 07 14:26:05.165306 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:05.095656 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-shb4c" Jul 07 14:26:05.165306 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:05.103292 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd3b76148_a347_4b4e_9ba5_fb4c41ebc389.slice/crio-a277de7fae0e4eb131553fd814e3f46e7fec9865c78c934c17e4617c5dc11a02 WatchSource:0}: Error finding container a277de7fae0e4eb131553fd814e3f46e7fec9865c78c934c17e4617c5dc11a02: Status 404 returned error can't find the container with id a277de7fae0e4eb131553fd814e3f46e7fec9865c78c934c17e4617c5dc11a02 Jul 07 14:26:05.165306 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:05.108380 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-wmt48" Jul 07 14:26:05.165306 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:05.115619 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaa141f90_5856_4779_90d7_c6377b1e32d6.slice/crio-65ec2e26dd0cbbfec35db9a21a050980464724a306954fc6d55ff487a85f3192 WatchSource:0}: Error finding container 65ec2e26dd0cbbfec35db9a21a050980464724a306954fc6d55ff487a85f3192: Status 404 returned error can't find the container with id 65ec2e26dd0cbbfec35db9a21a050980464724a306954fc6d55ff487a85f3192 Jul 07 14:26:05.165306 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:05.132982 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-scfg4" Jul 07 14:26:05.165306 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:05.138046 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-rgvqn" Jul 07 14:26:05.165306 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:05.139888 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod90d2dce1_be42_4595_b9c4_74a024eef0e9.slice/crio-d540ec9c0a02809eb5f7ea38bad76446a45ee0f7bcb90bd99795f88b7295b255 WatchSource:0}: Error finding container d540ec9c0a02809eb5f7ea38bad76446a45ee0f7bcb90bd99795f88b7295b255: Status 404 returned error can't find the container with id d540ec9c0a02809eb5f7ea38bad76446a45ee0f7bcb90bd99795f88b7295b255 Jul 07 14:26:05.167346 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:05.149834 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfa56eabc_b99d_4ce2_9f36_cb0242168cf4.slice/crio-6708274a0cda188c1a347715a4bd277515cbea0c886faa5141334ad6777dd530 WatchSource:0}: Error finding container 6708274a0cda188c1a347715a4bd277515cbea0c886faa5141334ad6777dd530: Status 404 returned error can't find the container with id 6708274a0cda188c1a347715a4bd277515cbea0c886faa5141334ad6777dd530 Jul 07 14:26:05.459043 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:05.385069 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6c88a149_4504_482b_90e7_87d3065096c2.slice/crio-7d432b15a2ea684da43dd262f74eab5f56ad977ee04ed559aa86f18cc47d4281 WatchSource:0}: Error finding container 7d432b15a2ea684da43dd262f74eab5f56ad977ee04ed559aa86f18cc47d4281: Status 404 returned error can't find the container with id 7d432b15a2ea684da43dd262f74eab5f56ad977ee04ed559aa86f18cc47d4281 Jul 07 14:26:05.459043 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:05.412253 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q5g9n\" (UniqueName: \"kubernetes.io/projected/09a0edd9-f556-490d-a691-a0cb3dc3aff9-kube-api-access-q5g9n\") pod \"network-check-target-bqthq\" (UID: \"09a0edd9-f556-490d-a691-a0cb3dc3aff9\") " pod="openshift-network-diagnostics/network-check-target-bqthq" Jul 07 14:26:05.459043 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:05.412284 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ce774983-4cbb-4526-ad84-72478efce781-original-pull-secret\") pod \"global-pull-secret-syncer-hvlzx\" (UID: \"ce774983-4cbb-4526-ad84-72478efce781\") " pod="kube-system/global-pull-secret-syncer-hvlzx" Jul 07 14:26:05.459043 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:05.412318 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/320ab15e-c868-4a37-a3e6-1cea72bb74a4-metrics-certs\") pod \"network-metrics-daemon-s4p8l\" (UID: \"320ab15e-c868-4a37-a3e6-1cea72bb74a4\") " pod="openshift-multus/network-metrics-daemon-s4p8l" Jul 07 14:26:05.459043 ip-10-0-134-42 kubenswrapper[2557]: E0707 14:26:05.412404 2557 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 14:26:05.459043 ip-10-0-134-42 kubenswrapper[2557]: E0707 14:26:05.412445 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/320ab15e-c868-4a37-a3e6-1cea72bb74a4-metrics-certs podName:320ab15e-c868-4a37-a3e6-1cea72bb74a4 nodeName:}" failed. No retries permitted until 2026-07-07 14:26:06.412432747 +0000 UTC m=+3.121571108 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/320ab15e-c868-4a37-a3e6-1cea72bb74a4-metrics-certs") pod "network-metrics-daemon-s4p8l" (UID: "320ab15e-c868-4a37-a3e6-1cea72bb74a4") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 14:26:05.459043 ip-10-0-134-42 kubenswrapper[2557]: E0707 14:26:05.412402 2557 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 07 14:26:05.459043 ip-10-0-134-42 kubenswrapper[2557]: E0707 14:26:05.412534 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ce774983-4cbb-4526-ad84-72478efce781-original-pull-secret podName:ce774983-4cbb-4526-ad84-72478efce781 nodeName:}" failed. No retries permitted until 2026-07-07 14:26:06.412507174 +0000 UTC m=+3.121645534 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ce774983-4cbb-4526-ad84-72478efce781-original-pull-secret") pod "global-pull-secret-syncer-hvlzx" (UID: "ce774983-4cbb-4526-ad84-72478efce781") : object "kube-system"/"original-pull-secret" not registered Jul 07 14:26:05.459043 ip-10-0-134-42 kubenswrapper[2557]: E0707 14:26:05.412404 2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 07 14:26:05.459043 ip-10-0-134-42 kubenswrapper[2557]: E0707 14:26:05.412553 2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 07 14:26:05.459043 ip-10-0-134-42 kubenswrapper[2557]: E0707 14:26:05.412564 2557 projected.go:196] Error preparing data for projected volume kube-api-access-q5g9n for pod openshift-network-diagnostics/network-check-target-bqthq: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 14:26:05.459043 ip-10-0-134-42 kubenswrapper[2557]: E0707 14:26:05.412587 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/09a0edd9-f556-490d-a691-a0cb3dc3aff9-kube-api-access-q5g9n podName:09a0edd9-f556-490d-a691-a0cb3dc3aff9 nodeName:}" failed. No retries permitted until 2026-07-07 14:26:06.412580629 +0000 UTC m=+3.121718989 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-q5g9n" (UniqueName: "kubernetes.io/projected/09a0edd9-f556-490d-a691-a0cb3dc3aff9-kube-api-access-q5g9n") pod "network-check-target-bqthq" (UID: "09a0edd9-f556-490d-a691-a0cb3dc3aff9") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 14:26:05.631216 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:05.630366 2557 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 07 14:26:05.849975 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:05.829585 2557 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-06 14:21:05 +0000 UTC" deadline="2028-03-15 18:22:00.2848485 +0000 UTC" Jul 07 14:26:05.849975 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:05.829617 2557 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14811h55m54.455235262s" Jul 07 14:26:05.920083 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:05.859623 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-scfg4" event={"ID":"90d2dce1-be42-4595-b9c4-74a024eef0e9","Type":"ContainerStarted","Data":"d540ec9c0a02809eb5f7ea38bad76446a45ee0f7bcb90bd99795f88b7295b255"} Jul 07 14:26:05.920083 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:05.864243 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-wmt48" event={"ID":"aa141f90-5856-4779-90d7-c6377b1e32d6","Type":"ContainerStarted","Data":"65ec2e26dd0cbbfec35db9a21a050980464724a306954fc6d55ff487a85f3192"} Jul 07 14:26:05.920083 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:05.867887 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-shb4c" event={"ID":"d3b76148-a347-4b4e-9ba5-fb4c41ebc389","Type":"ContainerStarted","Data":"a277de7fae0e4eb131553fd814e3f46e7fec9865c78c934c17e4617c5dc11a02"} Jul 07 14:26:05.920083 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:05.869881 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-42n8m" event={"ID":"9465f33d-a02b-4cd7-8c55-d30fe0648034","Type":"ContainerStarted","Data":"b318319fdfbd10d85c5e59e1c0cf3e36a497e8e7586b7dbec7c1fb6176e90bd5"} Jul 07 14:26:05.920083 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:05.871418 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zh4rc" event={"ID":"3f8eb048-9150-421e-8359-e64f3aec2211","Type":"ContainerStarted","Data":"8a3eca3502c7d8e815f44d680d24395c93fac12b93e5cff62d31a82efe8d3466"} Jul 07 14:26:05.920083 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:05.873892 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-pwmxs" event={"ID":"01caeef5-2981-41b4-92c9-75a39f813595","Type":"ContainerStarted","Data":"0f01effd072d0590f015dadbcd0f197f0cf0fe2d61e0e2b837d2398ce94717cf"} Jul 07 14:26:05.920083 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:05.877684 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-42.ec2.internal" event={"ID":"6063cb059acc790b3bb3b3d99eea6099","Type":"ContainerStarted","Data":"408230ea114c883787f8a91ad7fc8a44f7753ff449989b88b64157e14586fe0a"} Jul 07 14:26:05.920083 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:05.888220 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-2m5td" event={"ID":"6c88a149-4504-482b-90e7-87d3065096c2","Type":"ContainerStarted","Data":"7d432b15a2ea684da43dd262f74eab5f56ad977ee04ed559aa86f18cc47d4281"} Jul 07 14:26:05.920083 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:05.890053 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-rgvqn" event={"ID":"fa56eabc-b99d-4ce2-9f36-cb0242168cf4","Type":"ContainerStarted","Data":"6708274a0cda188c1a347715a4bd277515cbea0c886faa5141334ad6777dd530"} Jul 07 14:26:05.935169 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:05.933576 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-5g7bh" event={"ID":"0b70b350-feef-419b-91c0-45a22ae7f30b","Type":"ContainerStarted","Data":"4835a0d375b056231932d0bdddab3ca59901e361f78201bdf7c1df40e048f162"} Jul 07 14:26:05.942139 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:05.942054 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-42.ec2.internal" event={"ID":"61dca777ea2260edb003f493cc5637ae","Type":"ContainerStarted","Data":"7ce3bb33aa7c530e514c6e213d31f7f4d22b6d35752190fe2231bb1c27baddd7"} Jul 07 14:26:06.127262 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:06.127216 2557 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 07 14:26:06.423959 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:06.423849 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q5g9n\" (UniqueName: \"kubernetes.io/projected/09a0edd9-f556-490d-a691-a0cb3dc3aff9-kube-api-access-q5g9n\") pod \"network-check-target-bqthq\" (UID: \"09a0edd9-f556-490d-a691-a0cb3dc3aff9\") " pod="openshift-network-diagnostics/network-check-target-bqthq" Jul 07 14:26:06.423959 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:06.423906 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ce774983-4cbb-4526-ad84-72478efce781-original-pull-secret\") pod \"global-pull-secret-syncer-hvlzx\" (UID: \"ce774983-4cbb-4526-ad84-72478efce781\") " pod="kube-system/global-pull-secret-syncer-hvlzx" Jul 07 14:26:06.424173 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:06.423993 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/320ab15e-c868-4a37-a3e6-1cea72bb74a4-metrics-certs\") pod \"network-metrics-daemon-s4p8l\" (UID: \"320ab15e-c868-4a37-a3e6-1cea72bb74a4\") " pod="openshift-multus/network-metrics-daemon-s4p8l" Jul 07 14:26:06.424173 ip-10-0-134-42 kubenswrapper[2557]: E0707 14:26:06.424126 2557 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 14:26:06.424272 ip-10-0-134-42 kubenswrapper[2557]: E0707 14:26:06.424184 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/320ab15e-c868-4a37-a3e6-1cea72bb74a4-metrics-certs podName:320ab15e-c868-4a37-a3e6-1cea72bb74a4 nodeName:}" failed. No retries permitted until 2026-07-07 14:26:08.424164587 +0000 UTC m=+5.133302962 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/320ab15e-c868-4a37-a3e6-1cea72bb74a4-metrics-certs") pod "network-metrics-daemon-s4p8l" (UID: "320ab15e-c868-4a37-a3e6-1cea72bb74a4") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 14:26:06.424613 ip-10-0-134-42 kubenswrapper[2557]: E0707 14:26:06.424431 2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 07 14:26:06.424613 ip-10-0-134-42 kubenswrapper[2557]: E0707 14:26:06.424453 2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 07 14:26:06.424613 ip-10-0-134-42 kubenswrapper[2557]: E0707 14:26:06.424465 2557 projected.go:196] Error preparing data for projected volume kube-api-access-q5g9n for pod openshift-network-diagnostics/network-check-target-bqthq: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 14:26:06.424613 ip-10-0-134-42 kubenswrapper[2557]: E0707 14:26:06.424507 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/09a0edd9-f556-490d-a691-a0cb3dc3aff9-kube-api-access-q5g9n podName:09a0edd9-f556-490d-a691-a0cb3dc3aff9 nodeName:}" failed. No retries permitted until 2026-07-07 14:26:08.424493634 +0000 UTC m=+5.133632003 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-q5g9n" (UniqueName: "kubernetes.io/projected/09a0edd9-f556-490d-a691-a0cb3dc3aff9-kube-api-access-q5g9n") pod "network-check-target-bqthq" (UID: "09a0edd9-f556-490d-a691-a0cb3dc3aff9") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 14:26:06.424613 ip-10-0-134-42 kubenswrapper[2557]: E0707 14:26:06.424567 2557 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 07 14:26:06.424613 ip-10-0-134-42 kubenswrapper[2557]: E0707 14:26:06.424616 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ce774983-4cbb-4526-ad84-72478efce781-original-pull-secret podName:ce774983-4cbb-4526-ad84-72478efce781 nodeName:}" failed. No retries permitted until 2026-07-07 14:26:08.424587502 +0000 UTC m=+5.133725864 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ce774983-4cbb-4526-ad84-72478efce781-original-pull-secret") pod "global-pull-secret-syncer-hvlzx" (UID: "ce774983-4cbb-4526-ad84-72478efce781") : object "kube-system"/"original-pull-secret" not registered Jul 07 14:26:06.597446 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:06.597413 2557 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 07 14:26:06.829287 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:06.829255 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bqthq" Jul 07 14:26:06.829474 ip-10-0-134-42 kubenswrapper[2557]: E0707 14:26:06.829389 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bqthq" podUID="09a0edd9-f556-490d-a691-a0cb3dc3aff9" Jul 07 14:26:06.830210 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:06.829823 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s4p8l" Jul 07 14:26:06.830210 ip-10-0-134-42 kubenswrapper[2557]: E0707 14:26:06.829935 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-s4p8l" podUID="320ab15e-c868-4a37-a3e6-1cea72bb74a4" Jul 07 14:26:06.830210 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:06.830004 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hvlzx" Jul 07 14:26:06.830210 ip-10-0-134-42 kubenswrapper[2557]: E0707 14:26:06.830091 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hvlzx" podUID="ce774983-4cbb-4526-ad84-72478efce781" Jul 07 14:26:06.830490 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:06.830186 2557 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-06 14:21:05 +0000 UTC" deadline="2027-12-04 23:49:00.341677206 +0000 UTC" Jul 07 14:26:06.830490 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:06.830237 2557 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12369h22m53.511444627s" Jul 07 14:26:07.241695 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:07.241618 2557 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 07 14:26:08.442289 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:08.442250 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/320ab15e-c868-4a37-a3e6-1cea72bb74a4-metrics-certs\") pod \"network-metrics-daemon-s4p8l\" (UID: \"320ab15e-c868-4a37-a3e6-1cea72bb74a4\") " pod="openshift-multus/network-metrics-daemon-s4p8l" Jul 07 14:26:08.442916 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:08.442331 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q5g9n\" (UniqueName: \"kubernetes.io/projected/09a0edd9-f556-490d-a691-a0cb3dc3aff9-kube-api-access-q5g9n\") pod \"network-check-target-bqthq\" (UID: \"09a0edd9-f556-490d-a691-a0cb3dc3aff9\") " pod="openshift-network-diagnostics/network-check-target-bqthq" Jul 07 14:26:08.442916 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:08.442405 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ce774983-4cbb-4526-ad84-72478efce781-original-pull-secret\") pod \"global-pull-secret-syncer-hvlzx\" (UID: \"ce774983-4cbb-4526-ad84-72478efce781\") " pod="kube-system/global-pull-secret-syncer-hvlzx" Jul 07 14:26:08.442916 ip-10-0-134-42 kubenswrapper[2557]: E0707 14:26:08.442543 2557 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 07 14:26:08.442916 ip-10-0-134-42 kubenswrapper[2557]: E0707 14:26:08.442629 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ce774983-4cbb-4526-ad84-72478efce781-original-pull-secret podName:ce774983-4cbb-4526-ad84-72478efce781 nodeName:}" failed. No retries permitted until 2026-07-07 14:26:12.442608168 +0000 UTC m=+9.151746550 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ce774983-4cbb-4526-ad84-72478efce781-original-pull-secret") pod "global-pull-secret-syncer-hvlzx" (UID: "ce774983-4cbb-4526-ad84-72478efce781") : object "kube-system"/"original-pull-secret" not registered Jul 07 14:26:08.443157 ip-10-0-134-42 kubenswrapper[2557]: E0707 14:26:08.443043 2557 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 14:26:08.443157 ip-10-0-134-42 kubenswrapper[2557]: E0707 14:26:08.443095 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/320ab15e-c868-4a37-a3e6-1cea72bb74a4-metrics-certs podName:320ab15e-c868-4a37-a3e6-1cea72bb74a4 nodeName:}" failed. No retries permitted until 2026-07-07 14:26:12.443080198 +0000 UTC m=+9.152218574 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/320ab15e-c868-4a37-a3e6-1cea72bb74a4-metrics-certs") pod "network-metrics-daemon-s4p8l" (UID: "320ab15e-c868-4a37-a3e6-1cea72bb74a4") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 14:26:08.443266 ip-10-0-134-42 kubenswrapper[2557]: E0707 14:26:08.443177 2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 07 14:26:08.443266 ip-10-0-134-42 kubenswrapper[2557]: E0707 14:26:08.443195 2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 07 14:26:08.443266 ip-10-0-134-42 kubenswrapper[2557]: E0707 14:26:08.443208 2557 projected.go:196] Error preparing data for projected volume kube-api-access-q5g9n for pod openshift-network-diagnostics/network-check-target-bqthq: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 14:26:08.443266 ip-10-0-134-42 kubenswrapper[2557]: E0707 14:26:08.443242 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/09a0edd9-f556-490d-a691-a0cb3dc3aff9-kube-api-access-q5g9n podName:09a0edd9-f556-490d-a691-a0cb3dc3aff9 nodeName:}" failed. No retries permitted until 2026-07-07 14:26:12.44323203 +0000 UTC m=+9.152370392 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-q5g9n" (UniqueName: "kubernetes.io/projected/09a0edd9-f556-490d-a691-a0cb3dc3aff9-kube-api-access-q5g9n") pod "network-check-target-bqthq" (UID: "09a0edd9-f556-490d-a691-a0cb3dc3aff9") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 14:26:08.830171 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:08.830132 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hvlzx" Jul 07 14:26:08.830357 ip-10-0-134-42 kubenswrapper[2557]: E0707 14:26:08.830290 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hvlzx" podUID="ce774983-4cbb-4526-ad84-72478efce781" Jul 07 14:26:08.830634 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:08.830615 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s4p8l" Jul 07 14:26:08.830753 ip-10-0-134-42 kubenswrapper[2557]: E0707 14:26:08.830731 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-s4p8l" podUID="320ab15e-c868-4a37-a3e6-1cea72bb74a4" Jul 07 14:26:08.830821 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:08.830790 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bqthq" Jul 07 14:26:08.830872 ip-10-0-134-42 kubenswrapper[2557]: E0707 14:26:08.830858 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bqthq" podUID="09a0edd9-f556-490d-a691-a0cb3dc3aff9" Jul 07 14:26:10.829483 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:10.829441 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s4p8l" Jul 07 14:26:10.829990 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:10.829495 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bqthq" Jul 07 14:26:10.829990 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:10.829455 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hvlzx" Jul 07 14:26:10.829990 ip-10-0-134-42 kubenswrapper[2557]: E0707 14:26:10.829621 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-s4p8l" podUID="320ab15e-c868-4a37-a3e6-1cea72bb74a4" Jul 07 14:26:10.829990 ip-10-0-134-42 kubenswrapper[2557]: E0707 14:26:10.829680 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bqthq" podUID="09a0edd9-f556-490d-a691-a0cb3dc3aff9" Jul 07 14:26:10.829990 ip-10-0-134-42 kubenswrapper[2557]: E0707 14:26:10.829767 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hvlzx" podUID="ce774983-4cbb-4526-ad84-72478efce781" Jul 07 14:26:12.477328 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:12.477244 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/320ab15e-c868-4a37-a3e6-1cea72bb74a4-metrics-certs\") pod \"network-metrics-daemon-s4p8l\" (UID: \"320ab15e-c868-4a37-a3e6-1cea72bb74a4\") " pod="openshift-multus/network-metrics-daemon-s4p8l" Jul 07 14:26:12.477799 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:12.477332 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q5g9n\" (UniqueName: \"kubernetes.io/projected/09a0edd9-f556-490d-a691-a0cb3dc3aff9-kube-api-access-q5g9n\") pod \"network-check-target-bqthq\" (UID: \"09a0edd9-f556-490d-a691-a0cb3dc3aff9\") " pod="openshift-network-diagnostics/network-check-target-bqthq" Jul 07 14:26:12.477799 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:12.477369 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ce774983-4cbb-4526-ad84-72478efce781-original-pull-secret\") pod \"global-pull-secret-syncer-hvlzx\" (UID: \"ce774983-4cbb-4526-ad84-72478efce781\") " pod="kube-system/global-pull-secret-syncer-hvlzx" Jul 07 14:26:12.477799 ip-10-0-134-42 kubenswrapper[2557]: E0707 14:26:12.477405 2557 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 14:26:12.477799 ip-10-0-134-42 kubenswrapper[2557]: E0707 14:26:12.477467 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/320ab15e-c868-4a37-a3e6-1cea72bb74a4-metrics-certs podName:320ab15e-c868-4a37-a3e6-1cea72bb74a4 nodeName:}" failed. No retries permitted until 2026-07-07 14:26:20.477448362 +0000 UTC m=+17.186586743 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/320ab15e-c868-4a37-a3e6-1cea72bb74a4-metrics-certs") pod "network-metrics-daemon-s4p8l" (UID: "320ab15e-c868-4a37-a3e6-1cea72bb74a4") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 14:26:12.477799 ip-10-0-134-42 kubenswrapper[2557]: E0707 14:26:12.477467 2557 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 07 14:26:12.477799 ip-10-0-134-42 kubenswrapper[2557]: E0707 14:26:12.477504 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ce774983-4cbb-4526-ad84-72478efce781-original-pull-secret podName:ce774983-4cbb-4526-ad84-72478efce781 nodeName:}" failed. No retries permitted until 2026-07-07 14:26:20.477494216 +0000 UTC m=+17.186632578 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ce774983-4cbb-4526-ad84-72478efce781-original-pull-secret") pod "global-pull-secret-syncer-hvlzx" (UID: "ce774983-4cbb-4526-ad84-72478efce781") : object "kube-system"/"original-pull-secret" not registered Jul 07 14:26:12.477799 ip-10-0-134-42 kubenswrapper[2557]: E0707 14:26:12.477552 2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 07 14:26:12.477799 ip-10-0-134-42 kubenswrapper[2557]: E0707 14:26:12.477576 2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 07 14:26:12.477799 ip-10-0-134-42 kubenswrapper[2557]: E0707 14:26:12.477590 2557 projected.go:196] Error preparing data for projected volume kube-api-access-q5g9n for pod openshift-network-diagnostics/network-check-target-bqthq: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 14:26:12.477799 ip-10-0-134-42 kubenswrapper[2557]: E0707 14:26:12.477643 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/09a0edd9-f556-490d-a691-a0cb3dc3aff9-kube-api-access-q5g9n podName:09a0edd9-f556-490d-a691-a0cb3dc3aff9 nodeName:}" failed. No retries permitted until 2026-07-07 14:26:20.47762609 +0000 UTC m=+17.186764453 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-q5g9n" (UniqueName: "kubernetes.io/projected/09a0edd9-f556-490d-a691-a0cb3dc3aff9-kube-api-access-q5g9n") pod "network-check-target-bqthq" (UID: "09a0edd9-f556-490d-a691-a0cb3dc3aff9") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 14:26:12.829677 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:12.829580 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s4p8l" Jul 07 14:26:12.829677 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:12.829636 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hvlzx" Jul 07 14:26:12.829934 ip-10-0-134-42 kubenswrapper[2557]: E0707 14:26:12.829729 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-s4p8l" podUID="320ab15e-c868-4a37-a3e6-1cea72bb74a4" Jul 07 14:26:12.829934 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:12.829587 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bqthq" Jul 07 14:26:12.829934 ip-10-0-134-42 kubenswrapper[2557]: E0707 14:26:12.829843 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hvlzx" podUID="ce774983-4cbb-4526-ad84-72478efce781" Jul 07 14:26:12.830078 ip-10-0-134-42 kubenswrapper[2557]: E0707 14:26:12.829949 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bqthq" podUID="09a0edd9-f556-490d-a691-a0cb3dc3aff9" Jul 07 14:26:14.829595 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:14.829561 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hvlzx" Jul 07 14:26:14.829595 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:14.829586 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bqthq" Jul 07 14:26:14.829595 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:14.829589 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s4p8l" Jul 07 14:26:14.830246 ip-10-0-134-42 kubenswrapper[2557]: E0707 14:26:14.829707 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hvlzx" podUID="ce774983-4cbb-4526-ad84-72478efce781" Jul 07 14:26:14.830246 ip-10-0-134-42 kubenswrapper[2557]: E0707 14:26:14.829831 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bqthq" podUID="09a0edd9-f556-490d-a691-a0cb3dc3aff9" Jul 07 14:26:14.830246 ip-10-0-134-42 kubenswrapper[2557]: E0707 14:26:14.829936 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-s4p8l" podUID="320ab15e-c868-4a37-a3e6-1cea72bb74a4" Jul 07 14:26:16.830206 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:16.830177 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bqthq" Jul 07 14:26:16.830639 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:16.830232 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s4p8l" Jul 07 14:26:16.830639 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:16.830242 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hvlzx" Jul 07 14:26:16.830639 ip-10-0-134-42 kubenswrapper[2557]: E0707 14:26:16.830350 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-s4p8l" podUID="320ab15e-c868-4a37-a3e6-1cea72bb74a4" Jul 07 14:26:16.830639 ip-10-0-134-42 kubenswrapper[2557]: E0707 14:26:16.830491 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hvlzx" podUID="ce774983-4cbb-4526-ad84-72478efce781" Jul 07 14:26:16.830639 ip-10-0-134-42 kubenswrapper[2557]: E0707 14:26:16.830593 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bqthq" podUID="09a0edd9-f556-490d-a691-a0cb3dc3aff9" Jul 07 14:26:18.829915 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:18.829871 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hvlzx" Jul 07 14:26:18.829915 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:18.829913 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bqthq" Jul 07 14:26:18.830385 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:18.829981 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s4p8l" Jul 07 14:26:18.830385 ip-10-0-134-42 kubenswrapper[2557]: E0707 14:26:18.829998 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hvlzx" podUID="ce774983-4cbb-4526-ad84-72478efce781" Jul 07 14:26:18.830385 ip-10-0-134-42 kubenswrapper[2557]: E0707 14:26:18.830066 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bqthq" podUID="09a0edd9-f556-490d-a691-a0cb3dc3aff9" Jul 07 14:26:18.830385 ip-10-0-134-42 kubenswrapper[2557]: E0707 14:26:18.830166 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-s4p8l" podUID="320ab15e-c868-4a37-a3e6-1cea72bb74a4" Jul 07 14:26:20.544578 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:20.544529 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q5g9n\" (UniqueName: \"kubernetes.io/projected/09a0edd9-f556-490d-a691-a0cb3dc3aff9-kube-api-access-q5g9n\") pod \"network-check-target-bqthq\" (UID: \"09a0edd9-f556-490d-a691-a0cb3dc3aff9\") " pod="openshift-network-diagnostics/network-check-target-bqthq" Jul 07 14:26:20.545062 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:20.544602 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ce774983-4cbb-4526-ad84-72478efce781-original-pull-secret\") pod \"global-pull-secret-syncer-hvlzx\" (UID: \"ce774983-4cbb-4526-ad84-72478efce781\") " pod="kube-system/global-pull-secret-syncer-hvlzx" Jul 07 14:26:20.545062 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:20.544640 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/320ab15e-c868-4a37-a3e6-1cea72bb74a4-metrics-certs\") pod \"network-metrics-daemon-s4p8l\" (UID: \"320ab15e-c868-4a37-a3e6-1cea72bb74a4\") " pod="openshift-multus/network-metrics-daemon-s4p8l" Jul 07 14:26:20.545062 ip-10-0-134-42 kubenswrapper[2557]: E0707 14:26:20.544654 2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 07 14:26:20.545062 ip-10-0-134-42 kubenswrapper[2557]: E0707 14:26:20.544681 2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 07 14:26:20.545062 ip-10-0-134-42 kubenswrapper[2557]: E0707 14:26:20.544694 2557 projected.go:196] Error preparing data for projected volume kube-api-access-q5g9n for pod openshift-network-diagnostics/network-check-target-bqthq: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 14:26:20.545062 ip-10-0-134-42 kubenswrapper[2557]: E0707 14:26:20.544744 2557 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 07 14:26:20.545062 ip-10-0-134-42 kubenswrapper[2557]: E0707 14:26:20.544760 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/09a0edd9-f556-490d-a691-a0cb3dc3aff9-kube-api-access-q5g9n podName:09a0edd9-f556-490d-a691-a0cb3dc3aff9 nodeName:}" failed. No retries permitted until 2026-07-07 14:26:36.544742855 +0000 UTC m=+33.253881224 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-q5g9n" (UniqueName: "kubernetes.io/projected/09a0edd9-f556-490d-a691-a0cb3dc3aff9-kube-api-access-q5g9n") pod "network-check-target-bqthq" (UID: "09a0edd9-f556-490d-a691-a0cb3dc3aff9") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 14:26:20.545062 ip-10-0-134-42 kubenswrapper[2557]: E0707 14:26:20.544794 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ce774983-4cbb-4526-ad84-72478efce781-original-pull-secret podName:ce774983-4cbb-4526-ad84-72478efce781 nodeName:}" failed. No retries permitted until 2026-07-07 14:26:36.54478146 +0000 UTC m=+33.253919822 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ce774983-4cbb-4526-ad84-72478efce781-original-pull-secret") pod "global-pull-secret-syncer-hvlzx" (UID: "ce774983-4cbb-4526-ad84-72478efce781") : object "kube-system"/"original-pull-secret" not registered Jul 07 14:26:20.545062 ip-10-0-134-42 kubenswrapper[2557]: E0707 14:26:20.544744 2557 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 14:26:20.545062 ip-10-0-134-42 kubenswrapper[2557]: E0707 14:26:20.544834 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/320ab15e-c868-4a37-a3e6-1cea72bb74a4-metrics-certs podName:320ab15e-c868-4a37-a3e6-1cea72bb74a4 nodeName:}" failed. No retries permitted until 2026-07-07 14:26:36.544827097 +0000 UTC m=+33.253965455 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/320ab15e-c868-4a37-a3e6-1cea72bb74a4-metrics-certs") pod "network-metrics-daemon-s4p8l" (UID: "320ab15e-c868-4a37-a3e6-1cea72bb74a4") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 14:26:20.830113 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:20.830092 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bqthq" Jul 07 14:26:20.830243 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:20.830184 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s4p8l" Jul 07 14:26:20.830243 ip-10-0-134-42 kubenswrapper[2557]: E0707 14:26:20.830189 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bqthq" podUID="09a0edd9-f556-490d-a691-a0cb3dc3aff9" Jul 07 14:26:20.830307 ip-10-0-134-42 kubenswrapper[2557]: E0707 14:26:20.830273 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-s4p8l" podUID="320ab15e-c868-4a37-a3e6-1cea72bb74a4" Jul 07 14:26:20.830307 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:20.830188 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hvlzx" Jul 07 14:26:20.830377 ip-10-0-134-42 kubenswrapper[2557]: E0707 14:26:20.830350 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hvlzx" podUID="ce774983-4cbb-4526-ad84-72478efce781" Jul 07 14:26:22.829471 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:22.829438 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hvlzx" Jul 07 14:26:22.829471 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:22.829454 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bqthq" Jul 07 14:26:22.829471 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:22.829475 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s4p8l" Jul 07 14:26:22.829989 ip-10-0-134-42 kubenswrapper[2557]: E0707 14:26:22.829580 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hvlzx" podUID="ce774983-4cbb-4526-ad84-72478efce781" Jul 07 14:26:22.829989 ip-10-0-134-42 kubenswrapper[2557]: E0707 14:26:22.829655 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bqthq" podUID="09a0edd9-f556-490d-a691-a0cb3dc3aff9" Jul 07 14:26:22.829989 ip-10-0-134-42 kubenswrapper[2557]: E0707 14:26:22.829716 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-s4p8l" podUID="320ab15e-c868-4a37-a3e6-1cea72bb74a4" Jul 07 14:26:24.830226 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:24.830191 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bqthq" Jul 07 14:26:24.830226 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:24.830224 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s4p8l" Jul 07 14:26:24.830725 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:24.830233 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hvlzx" Jul 07 14:26:24.830725 ip-10-0-134-42 kubenswrapper[2557]: E0707 14:26:24.830304 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bqthq" podUID="09a0edd9-f556-490d-a691-a0cb3dc3aff9" Jul 07 14:26:24.830725 ip-10-0-134-42 kubenswrapper[2557]: E0707 14:26:24.830408 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-s4p8l" podUID="320ab15e-c868-4a37-a3e6-1cea72bb74a4" Jul 07 14:26:24.830725 ip-10-0-134-42 kubenswrapper[2557]: E0707 14:26:24.830495 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hvlzx" podUID="ce774983-4cbb-4526-ad84-72478efce781" Jul 07 14:26:26.002983 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:26.002640 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-pwmxs" event={"ID":"01caeef5-2981-41b4-92c9-75a39f813595","Type":"ContainerStarted","Data":"c9747ed9fc9cb52f9c3042f64b4584c8a07ea2eddc888a683dd94271d6e0735e"} Jul 07 14:26:26.004119 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:26.004081 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-42.ec2.internal" event={"ID":"61dca777ea2260edb003f493cc5637ae","Type":"ContainerStarted","Data":"bd1f4c28a131756ce04624142f43ebc55e60eb0cdca1d3a3ce5bd910485e2b06"} Jul 07 14:26:26.007632 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:26.007616 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-scfg4_90d2dce1-be42-4595-b9c4-74a024eef0e9/ovn-acl-logging/0.log" Jul 07 14:26:26.007912 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:26.007895 2557 generic.go:356] "Generic (PLEG): container finished" podID="90d2dce1-be42-4595-b9c4-74a024eef0e9" containerID="db1f721a8919d2a9403d58ba8f3478ba134f38190e80258f30683df2f2efd7b7" exitCode=1 Jul 07 14:26:26.007962 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:26.007951 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-scfg4" event={"ID":"90d2dce1-be42-4595-b9c4-74a024eef0e9","Type":"ContainerStarted","Data":"ff2a1eb9374025ad1951e2e5272fb4d1fe1d54cc43680da9b39a4b7598f7d02c"} Jul 07 14:26:26.008002 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:26.007993 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-scfg4" event={"ID":"90d2dce1-be42-4595-b9c4-74a024eef0e9","Type":"ContainerStarted","Data":"57cb02f603f4f039f12b6b539a0d8170d7072da9ef7ec2fd632a55bb09ba93e2"} Jul 07 14:26:26.008039 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:26.008024 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-scfg4" event={"ID":"90d2dce1-be42-4595-b9c4-74a024eef0e9","Type":"ContainerStarted","Data":"bfa158184d5ed216de3f5045241a05631d6e88c0cf84f4e322db3eba302fe5ca"} Jul 07 14:26:26.008075 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:26.008051 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-scfg4" event={"ID":"90d2dce1-be42-4595-b9c4-74a024eef0e9","Type":"ContainerDied","Data":"db1f721a8919d2a9403d58ba8f3478ba134f38190e80258f30683df2f2efd7b7"} Jul 07 14:26:26.008114 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:26.008080 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-scfg4" event={"ID":"90d2dce1-be42-4595-b9c4-74a024eef0e9","Type":"ContainerStarted","Data":"5749c7b4c654a6a038a1b887b7117decead0ce65512bf375822e178cd9caf1c0"} Jul 07 14:26:26.009173 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:26.009155 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-42n8m" event={"ID":"9465f33d-a02b-4cd7-8c55-d30fe0648034","Type":"ContainerStarted","Data":"0f2a5c25d02c454e0dc555c9b808e8f6bd84722afa43790ad483a1e6be24240d"} Jul 07 14:26:26.020814 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:26.020503 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-pwmxs" podStartSLOduration=0.712304589 podStartE2EDuration="21.020489481s" totalImagesPullingTime="20.308184892s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 14:26:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:26:26.020336902 +0000 UTC m=+22.729475282" watchObservedRunningTime="2026-07-07 14:26:26.020489481 +0000 UTC m=+22.729627862" Jul 07 14:26:26.033604 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:26.033574 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-42.ec2.internal" podStartSLOduration=21.033563204 podStartE2EDuration="21.033563204s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 14:26:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:26:26.033372611 +0000 UTC m=+22.742510989" watchObservedRunningTime="2026-07-07 14:26:26.033563204 +0000 UTC m=+22.742701584" Jul 07 14:26:26.830174 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:26.830150 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hvlzx" Jul 07 14:26:26.830294 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:26.830155 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s4p8l" Jul 07 14:26:26.830294 ip-10-0-134-42 kubenswrapper[2557]: E0707 14:26:26.830258 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hvlzx" podUID="ce774983-4cbb-4526-ad84-72478efce781" Jul 07 14:26:26.830294 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:26.830268 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bqthq" Jul 07 14:26:26.830422 ip-10-0-134-42 kubenswrapper[2557]: E0707 14:26:26.830357 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-s4p8l" podUID="320ab15e-c868-4a37-a3e6-1cea72bb74a4" Jul 07 14:26:26.830461 ip-10-0-134-42 kubenswrapper[2557]: E0707 14:26:26.830426 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bqthq" podUID="09a0edd9-f556-490d-a691-a0cb3dc3aff9" Jul 07 14:26:27.012074 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:27.012049 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zh4rc" event={"ID":"3f8eb048-9150-421e-8359-e64f3aec2211","Type":"ContainerStarted","Data":"bdd11d963f083a2e35139e8052eaec9b2073f242e345757849ad86c698756610"} Jul 07 14:26:27.013439 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:27.013420 2557 generic.go:356] "Generic (PLEG): container finished" podID="6063cb059acc790b3bb3b3d99eea6099" containerID="42965f4df50144080f1ef7fce674661d124e59bbf86fa33d90e251e8b5d4ca3c" exitCode=0 Jul 07 14:26:27.013555 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:27.013475 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-42.ec2.internal" event={"ID":"6063cb059acc790b3bb3b3d99eea6099","Type":"ContainerDied","Data":"42965f4df50144080f1ef7fce674661d124e59bbf86fa33d90e251e8b5d4ca3c"} Jul 07 14:26:27.014857 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:27.014839 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-2m5td" event={"ID":"6c88a149-4504-482b-90e7-87d3065096c2","Type":"ContainerStarted","Data":"22bfc118e11c7ca3921699fb99fd314910810485338264f7afa1b1aa61bb81f4"} Jul 07 14:26:27.016269 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:27.016241 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-rgvqn" event={"ID":"fa56eabc-b99d-4ce2-9f36-cb0242168cf4","Type":"ContainerStarted","Data":"b0da4e0c076d61cc43799e8eef071af1bd57ba419da98673958eb8e1a5550ea3"} Jul 07 14:26:27.017627 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:27.017606 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-5g7bh" event={"ID":"0b70b350-feef-419b-91c0-45a22ae7f30b","Type":"ContainerStarted","Data":"456e7c35bfc16ecfcd12c3f7bb4a230af14809a50ebe6e9b6a91f00d87fca82d"} Jul 07 14:26:27.020078 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:27.020048 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-scfg4_90d2dce1-be42-4595-b9c4-74a024eef0e9/ovn-acl-logging/0.log" Jul 07 14:26:27.020375 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:27.020356 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-scfg4" event={"ID":"90d2dce1-be42-4595-b9c4-74a024eef0e9","Type":"ContainerStarted","Data":"ecdc569d0a7e068beda5eff9bf697dc1a7633d05a8e8f8233008d1c3abbf605b"} Jul 07 14:26:27.023161 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:27.023137 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-wmt48" event={"ID":"aa141f90-5856-4779-90d7-c6377b1e32d6","Type":"ContainerStarted","Data":"91db7055861263c16ef19043413fb863a88e392a793f97c2deda8ce5bc2aaa8f"} Jul 07 14:26:27.025021 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:27.024874 2557 generic.go:356] "Generic (PLEG): container finished" podID="d3b76148-a347-4b4e-9ba5-fb4c41ebc389" containerID="3ac58a1c12beca44b46f607656fdcf72eb09a2e26f1d57b9143ebd05f58d3969" exitCode=0 Jul 07 14:26:27.025093 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:27.024951 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-shb4c" event={"ID":"d3b76148-a347-4b4e-9ba5-fb4c41ebc389","Type":"ContainerDied","Data":"3ac58a1c12beca44b46f607656fdcf72eb09a2e26f1d57b9143ebd05f58d3969"} Jul 07 14:26:27.032729 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:27.032679 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-42n8m" podStartSLOduration=2.516497248 podStartE2EDuration="23.032663724s" totalImagesPullingTime="20.516166476s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 14:26:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:26:26.053433655 +0000 UTC m=+22.762572035" watchObservedRunningTime="2026-07-07 14:26:27.032663724 +0000 UTC m=+23.741802105" Jul 07 14:26:27.069873 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:27.069830 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-2m5td" podStartSLOduration=2.152538691 podStartE2EDuration="22.069814132s" totalImagesPullingTime="19.917275441s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 14:26:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:26:27.069167493 +0000 UTC m=+23.778305872" watchObservedRunningTime="2026-07-07 14:26:27.069814132 +0000 UTC m=+23.778952512" Jul 07 14:26:27.101332 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:27.101301 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-5g7bh" podStartSLOduration=9.945164699 podStartE2EDuration="22.101293528s" totalImagesPullingTime="12.156128829s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 14:26:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:26:27.087274444 +0000 UTC m=+23.796412823" watchObservedRunningTime="2026-07-07 14:26:27.101293528 +0000 UTC m=+23.810431902" Jul 07 14:26:27.101666 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:27.101644 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-wmt48" podStartSLOduration=2.88868629 podStartE2EDuration="23.10163961s" totalImagesPullingTime="20.21295332s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 14:26:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:26:27.10113417 +0000 UTC m=+23.810272549" watchObservedRunningTime="2026-07-07 14:26:27.10163961 +0000 UTC m=+23.810777989" Jul 07 14:26:27.840156 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:27.840131 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-2m5td" Jul 07 14:26:27.840685 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:27.840666 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-2m5td" Jul 07 14:26:27.855960 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:27.855915 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-rgvqn" podStartSLOduration=2.703334205 podStartE2EDuration="22.855903596s" totalImagesPullingTime="20.152569391s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 14:26:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:26:27.125446414 +0000 UTC m=+23.834584796" watchObservedRunningTime="2026-07-07 14:26:27.855903596 +0000 UTC m=+24.565042031" Jul 07 14:26:28.029075 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:28.028864 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-42.ec2.internal" event={"ID":"6063cb059acc790b3bb3b3d99eea6099","Type":"ContainerStarted","Data":"697572031dc6ddd9717b306e08205029c4ec0ecc656c78a868bc9c5688b07335"} Jul 07 14:26:28.030925 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:28.030651 2557 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 07 14:26:28.032051 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:28.031802 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zh4rc" event={"ID":"3f8eb048-9150-421e-8359-e64f3aec2211","Type":"ContainerStarted","Data":"308cf75aefff0a1a2b48ee4509acd71ade29ba4181fb2f7d9dd5e5bea9149d8e"} Jul 07 14:26:28.032507 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:28.032489 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-2m5td" Jul 07 14:26:28.032621 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:28.032580 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-2m5td" Jul 07 14:26:28.061185 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:28.061148 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-42.ec2.internal" podStartSLOduration=23.06113762 podStartE2EDuration="23.06113762s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 14:26:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:26:28.047776947 +0000 UTC m=+24.756915326" watchObservedRunningTime="2026-07-07 14:26:28.06113762 +0000 UTC m=+24.770275999" Jul 07 14:26:28.805529 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:28.805404 2557 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-07T14:26:28.030670156Z","UUID":"a30e45a9-c9f0-45d7-96be-bf7fdadc7adb","Handler":null,"Name":"","Endpoint":""} Jul 07 14:26:28.806996 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:28.806969 2557 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 07 14:26:28.807131 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:28.807015 2557 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 07 14:26:28.829564 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:28.829535 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s4p8l" Jul 07 14:26:28.829691 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:28.829575 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hvlzx" Jul 07 14:26:28.829691 ip-10-0-134-42 kubenswrapper[2557]: E0707 14:26:28.829662 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-s4p8l" podUID="320ab15e-c868-4a37-a3e6-1cea72bb74a4" Jul 07 14:26:28.829782 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:28.829728 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bqthq" Jul 07 14:26:28.829815 ip-10-0-134-42 kubenswrapper[2557]: E0707 14:26:28.829799 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hvlzx" podUID="ce774983-4cbb-4526-ad84-72478efce781" Jul 07 14:26:28.829872 ip-10-0-134-42 kubenswrapper[2557]: E0707 14:26:28.829858 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bqthq" podUID="09a0edd9-f556-490d-a691-a0cb3dc3aff9" Jul 07 14:26:29.037253 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:29.037210 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-scfg4_90d2dce1-be42-4595-b9c4-74a024eef0e9/ovn-acl-logging/0.log" Jul 07 14:26:29.037789 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:29.037620 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-scfg4" event={"ID":"90d2dce1-be42-4595-b9c4-74a024eef0e9","Type":"ContainerStarted","Data":"6c79ee548723e36260c682f5cd844aa5b6b069d04710ccfd012169a2123f1722"} Jul 07 14:26:30.041415 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:30.041372 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zh4rc" event={"ID":"3f8eb048-9150-421e-8359-e64f3aec2211","Type":"ContainerStarted","Data":"3d0370108147b1f6c6158b42b888e81e041719cffe437895ae42af39495898ac"} Jul 07 14:26:30.830066 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:30.829839 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hvlzx" Jul 07 14:26:30.830262 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:30.829839 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bqthq" Jul 07 14:26:30.830262 ip-10-0-134-42 kubenswrapper[2557]: E0707 14:26:30.830178 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hvlzx" podUID="ce774983-4cbb-4526-ad84-72478efce781" Jul 07 14:26:30.830359 ip-10-0-134-42 kubenswrapper[2557]: E0707 14:26:30.830269 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bqthq" podUID="09a0edd9-f556-490d-a691-a0cb3dc3aff9" Jul 07 14:26:30.830359 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:30.829867 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s4p8l" Jul 07 14:26:30.830448 ip-10-0-134-42 kubenswrapper[2557]: E0707 14:26:30.830376 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-s4p8l" podUID="320ab15e-c868-4a37-a3e6-1cea72bb74a4" Jul 07 14:26:31.047173 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:31.047144 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-scfg4_90d2dce1-be42-4595-b9c4-74a024eef0e9/ovn-acl-logging/0.log" Jul 07 14:26:31.047724 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:31.047553 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-scfg4" event={"ID":"90d2dce1-be42-4595-b9c4-74a024eef0e9","Type":"ContainerStarted","Data":"eefe3d994a09e7cd03a9d091e01d311c93b3b1844496f8b26da56d21601af4be"} Jul 07 14:26:31.048151 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:31.048129 2557 scope.go:117] "RemoveContainer" containerID="db1f721a8919d2a9403d58ba8f3478ba134f38190e80258f30683df2f2efd7b7" Jul 07 14:26:31.074670 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:31.074623 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zh4rc" podStartSLOduration=4.48850714 podStartE2EDuration="27.074605433s" totalImagesPullingTime="22.586098293s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 14:26:04 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:26:30.057836946 +0000 UTC m=+26.766975327" watchObservedRunningTime="2026-07-07 14:26:31.074605433 +0000 UTC m=+27.783743815" Jul 07 14:26:32.463067 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:32.462825 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-hvlzx"] Jul 07 14:26:32.463905 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:32.463171 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hvlzx" Jul 07 14:26:32.463905 ip-10-0-134-42 kubenswrapper[2557]: E0707 14:26:32.463264 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hvlzx" podUID="ce774983-4cbb-4526-ad84-72478efce781" Jul 07 14:26:32.466211 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:32.465949 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-s4p8l"] Jul 07 14:26:32.466211 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:32.466070 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s4p8l" Jul 07 14:26:32.466211 ip-10-0-134-42 kubenswrapper[2557]: E0707 14:26:32.466172 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-s4p8l" podUID="320ab15e-c868-4a37-a3e6-1cea72bb74a4" Jul 07 14:26:32.466900 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:32.466684 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-bqthq"] Jul 07 14:26:32.466900 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:32.466784 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bqthq" Jul 07 14:26:32.466900 ip-10-0-134-42 kubenswrapper[2557]: E0707 14:26:32.466868 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bqthq" podUID="09a0edd9-f556-490d-a691-a0cb3dc3aff9" Jul 07 14:26:33.055306 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:33.055278 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-scfg4_90d2dce1-be42-4595-b9c4-74a024eef0e9/ovn-acl-logging/0.log" Jul 07 14:26:33.055678 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:33.055642 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-scfg4" event={"ID":"90d2dce1-be42-4595-b9c4-74a024eef0e9","Type":"ContainerStarted","Data":"2cebaed6daa205502eea3757e0c284c0f5b8ee992b84e0fa1d0e00b3e51c63c9"} Jul 07 14:26:33.056044 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:33.056024 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-scfg4" Jul 07 14:26:33.056140 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:33.056096 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-scfg4" Jul 07 14:26:33.056140 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:33.056135 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-scfg4" Jul 07 14:26:33.057700 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:33.057673 2557 generic.go:356] "Generic (PLEG): container finished" podID="d3b76148-a347-4b4e-9ba5-fb4c41ebc389" containerID="9e5ab6bb26be3bc73ac13fa6a203a0a3d09b0fba9e17717c582cfcc159e8d9fe" exitCode=0 Jul 07 14:26:33.057792 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:33.057728 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-shb4c" event={"ID":"d3b76148-a347-4b4e-9ba5-fb4c41ebc389","Type":"ContainerDied","Data":"9e5ab6bb26be3bc73ac13fa6a203a0a3d09b0fba9e17717c582cfcc159e8d9fe"} Jul 07 14:26:33.070941 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:33.070912 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-scfg4" Jul 07 14:26:33.071374 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:33.071359 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-scfg4" Jul 07 14:26:33.104893 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:33.104852 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-scfg4" podStartSLOduration=8.87422724 podStartE2EDuration="29.104840532s" totalImagesPullingTime="20.230613292s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 14:26:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:26:33.104815921 +0000 UTC m=+29.813954292" watchObservedRunningTime="2026-07-07 14:26:33.104840532 +0000 UTC m=+29.813978890" Jul 07 14:26:33.830480 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:33.830450 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s4p8l" Jul 07 14:26:33.830993 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:33.830510 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hvlzx" Jul 07 14:26:33.831101 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:33.830588 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bqthq" Jul 07 14:26:33.831165 ip-10-0-134-42 kubenswrapper[2557]: E0707 14:26:33.831072 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-s4p8l" podUID="320ab15e-c868-4a37-a3e6-1cea72bb74a4" Jul 07 14:26:33.831225 ip-10-0-134-42 kubenswrapper[2557]: E0707 14:26:33.831171 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hvlzx" podUID="ce774983-4cbb-4526-ad84-72478efce781" Jul 07 14:26:33.831281 ip-10-0-134-42 kubenswrapper[2557]: E0707 14:26:33.831254 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bqthq" podUID="09a0edd9-f556-490d-a691-a0cb3dc3aff9" Jul 07 14:26:35.064474 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:35.064442 2557 generic.go:356] "Generic (PLEG): container finished" podID="d3b76148-a347-4b4e-9ba5-fb4c41ebc389" containerID="37ce28bd112267ece9e03d9e045a17d536bc5da66bfbdc52be4f8e4aad37868f" exitCode=0 Jul 07 14:26:35.064835 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:35.064500 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-shb4c" event={"ID":"d3b76148-a347-4b4e-9ba5-fb4c41ebc389","Type":"ContainerDied","Data":"37ce28bd112267ece9e03d9e045a17d536bc5da66bfbdc52be4f8e4aad37868f"} Jul 07 14:26:35.077816 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:35.077774 2557 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ovn-kubernetes/ovnkube-node-scfg4" podUID="90d2dce1-be42-4595-b9c4-74a024eef0e9" containerName="ovnkube-controller" probeResult="failure" output="" Jul 07 14:26:35.829651 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:35.829624 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s4p8l" Jul 07 14:26:35.829822 ip-10-0-134-42 kubenswrapper[2557]: E0707 14:26:35.829800 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-s4p8l" podUID="320ab15e-c868-4a37-a3e6-1cea72bb74a4" Jul 07 14:26:35.830241 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:35.830219 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hvlzx" Jul 07 14:26:35.830358 ip-10-0-134-42 kubenswrapper[2557]: E0707 14:26:35.830320 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hvlzx" podUID="ce774983-4cbb-4526-ad84-72478efce781" Jul 07 14:26:35.830358 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:35.830348 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bqthq" Jul 07 14:26:35.830469 ip-10-0-134-42 kubenswrapper[2557]: E0707 14:26:35.830423 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bqthq" podUID="09a0edd9-f556-490d-a691-a0cb3dc3aff9" Jul 07 14:26:36.563898 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:36.563823 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q5g9n\" (UniqueName: \"kubernetes.io/projected/09a0edd9-f556-490d-a691-a0cb3dc3aff9-kube-api-access-q5g9n\") pod \"network-check-target-bqthq\" (UID: \"09a0edd9-f556-490d-a691-a0cb3dc3aff9\") " pod="openshift-network-diagnostics/network-check-target-bqthq" Jul 07 14:26:36.563898 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:36.563859 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ce774983-4cbb-4526-ad84-72478efce781-original-pull-secret\") pod \"global-pull-secret-syncer-hvlzx\" (UID: \"ce774983-4cbb-4526-ad84-72478efce781\") " pod="kube-system/global-pull-secret-syncer-hvlzx" Jul 07 14:26:36.563898 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:36.563888 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/320ab15e-c868-4a37-a3e6-1cea72bb74a4-metrics-certs\") pod \"network-metrics-daemon-s4p8l\" (UID: \"320ab15e-c868-4a37-a3e6-1cea72bb74a4\") " pod="openshift-multus/network-metrics-daemon-s4p8l" Jul 07 14:26:36.564287 ip-10-0-134-42 kubenswrapper[2557]: E0707 14:26:36.563967 2557 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 14:26:36.564287 ip-10-0-134-42 kubenswrapper[2557]: E0707 14:26:36.563968 2557 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 07 14:26:36.564287 ip-10-0-134-42 kubenswrapper[2557]: E0707 14:26:36.563979 2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 07 14:26:36.564287 ip-10-0-134-42 kubenswrapper[2557]: E0707 14:26:36.564024 2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 07 14:26:36.564287 ip-10-0-134-42 kubenswrapper[2557]: E0707 14:26:36.564034 2557 projected.go:196] Error preparing data for projected volume kube-api-access-q5g9n for pod openshift-network-diagnostics/network-check-target-bqthq: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 14:26:36.564287 ip-10-0-134-42 kubenswrapper[2557]: E0707 14:26:36.564010 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/320ab15e-c868-4a37-a3e6-1cea72bb74a4-metrics-certs podName:320ab15e-c868-4a37-a3e6-1cea72bb74a4 nodeName:}" failed. No retries permitted until 2026-07-07 14:27:08.563998201 +0000 UTC m=+65.273136559 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/320ab15e-c868-4a37-a3e6-1cea72bb74a4-metrics-certs") pod "network-metrics-daemon-s4p8l" (UID: "320ab15e-c868-4a37-a3e6-1cea72bb74a4") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 14:26:36.564287 ip-10-0-134-42 kubenswrapper[2557]: E0707 14:26:36.564075 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ce774983-4cbb-4526-ad84-72478efce781-original-pull-secret podName:ce774983-4cbb-4526-ad84-72478efce781 nodeName:}" failed. No retries permitted until 2026-07-07 14:27:08.564062465 +0000 UTC m=+65.273200826 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ce774983-4cbb-4526-ad84-72478efce781-original-pull-secret") pod "global-pull-secret-syncer-hvlzx" (UID: "ce774983-4cbb-4526-ad84-72478efce781") : object "kube-system"/"original-pull-secret" not registered Jul 07 14:26:36.564287 ip-10-0-134-42 kubenswrapper[2557]: E0707 14:26:36.564092 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/09a0edd9-f556-490d-a691-a0cb3dc3aff9-kube-api-access-q5g9n podName:09a0edd9-f556-490d-a691-a0cb3dc3aff9 nodeName:}" failed. No retries permitted until 2026-07-07 14:27:08.564083987 +0000 UTC m=+65.273222345 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-q5g9n" (UniqueName: "kubernetes.io/projected/09a0edd9-f556-490d-a691-a0cb3dc3aff9-kube-api-access-q5g9n") pod "network-check-target-bqthq" (UID: "09a0edd9-f556-490d-a691-a0cb3dc3aff9") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 14:26:37.070935 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:37.070901 2557 generic.go:356] "Generic (PLEG): container finished" podID="d3b76148-a347-4b4e-9ba5-fb4c41ebc389" containerID="5ecd373ee517497437038912453099b2d952110e5b0cd99b6105831d680492bf" exitCode=0 Jul 07 14:26:37.071122 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:37.070970 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-shb4c" event={"ID":"d3b76148-a347-4b4e-9ba5-fb4c41ebc389","Type":"ContainerDied","Data":"5ecd373ee517497437038912453099b2d952110e5b0cd99b6105831d680492bf"} Jul 07 14:26:37.830164 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:37.830053 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bqthq" Jul 07 14:26:37.830164 ip-10-0-134-42 kubenswrapper[2557]: E0707 14:26:37.830163 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bqthq" podUID="09a0edd9-f556-490d-a691-a0cb3dc3aff9" Jul 07 14:26:37.830688 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:37.830061 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s4p8l" Jul 07 14:26:37.830688 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:37.830188 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hvlzx" Jul 07 14:26:37.830688 ip-10-0-134-42 kubenswrapper[2557]: E0707 14:26:37.830273 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-s4p8l" podUID="320ab15e-c868-4a37-a3e6-1cea72bb74a4" Jul 07 14:26:37.830688 ip-10-0-134-42 kubenswrapper[2557]: E0707 14:26:37.830342 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hvlzx" podUID="ce774983-4cbb-4526-ad84-72478efce781" Jul 07 14:26:38.508376 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:38.508221 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-42.ec2.internal" event="NodeReady" Jul 07 14:26:38.508496 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:38.508451 2557 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 07 14:26:38.548747 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:38.548705 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-687956bbfb-m9bjn"] Jul 07 14:26:38.550645 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:38.550621 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-687956bbfb-m9bjn" Jul 07 14:26:38.552305 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:38.552286 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 07 14:26:38.552718 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:38.552698 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-6pj2m\"" Jul 07 14:26:38.553043 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:38.553023 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 07 14:26:38.553188 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:38.553102 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 07 14:26:38.558476 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:38.558281 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 07 14:26:38.567423 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:38.567391 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-687956bbfb-m9bjn"] Jul 07 14:26:38.570393 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:38.569851 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-8dtp8"] Jul 07 14:26:38.572229 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:38.572210 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-5ngmq"] Jul 07 14:26:38.572355 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:38.572339 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-8dtp8" Jul 07 14:26:38.573974 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:38.573958 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-5ngmq" Jul 07 14:26:38.577584 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:38.577561 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 07 14:26:38.577675 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:38.577605 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 07 14:26:38.577675 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:38.577611 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 07 14:26:38.577782 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:38.577570 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-ksq8k\"" Jul 07 14:26:38.578172 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:38.577900 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-ck6xv\"" Jul 07 14:26:38.578172 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:38.577966 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 07 14:26:38.578172 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:38.577897 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 07 14:26:38.579973 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:38.579953 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/3d597bc9-74bf-4809-a33d-0a28dec34cb1-config-volume\") pod \"dns-default-5ngmq\" (UID: \"3d597bc9-74bf-4809-a33d-0a28dec34cb1\") " pod="openshift-dns/dns-default-5ngmq" Jul 07 14:26:38.580060 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:38.579997 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rl7qw\" (UniqueName: \"kubernetes.io/projected/3d597bc9-74bf-4809-a33d-0a28dec34cb1-kube-api-access-rl7qw\") pod \"dns-default-5ngmq\" (UID: \"3d597bc9-74bf-4809-a33d-0a28dec34cb1\") " pod="openshift-dns/dns-default-5ngmq" Jul 07 14:26:38.580060 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:38.580025 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/648e2b2f-08a5-4a83-b9aa-319ab2527ae4-image-registry-private-configuration\") pod \"image-registry-687956bbfb-m9bjn\" (UID: \"648e2b2f-08a5-4a83-b9aa-319ab2527ae4\") " pod="openshift-image-registry/image-registry-687956bbfb-m9bjn" Jul 07 14:26:38.580060 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:38.580052 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tr24x\" (UniqueName: \"kubernetes.io/projected/8c3755dc-b805-4f6a-8c8e-6ec9077a8efb-kube-api-access-tr24x\") pod \"ingress-canary-8dtp8\" (UID: \"8c3755dc-b805-4f6a-8c8e-6ec9077a8efb\") " pod="openshift-ingress-canary/ingress-canary-8dtp8" Jul 07 14:26:38.580212 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:38.580082 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3d597bc9-74bf-4809-a33d-0a28dec34cb1-tmp-dir\") pod \"dns-default-5ngmq\" (UID: \"3d597bc9-74bf-4809-a33d-0a28dec34cb1\") " pod="openshift-dns/dns-default-5ngmq" Jul 07 14:26:38.580212 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:38.580150 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/648e2b2f-08a5-4a83-b9aa-319ab2527ae4-trusted-ca\") pod \"image-registry-687956bbfb-m9bjn\" (UID: \"648e2b2f-08a5-4a83-b9aa-319ab2527ae4\") " pod="openshift-image-registry/image-registry-687956bbfb-m9bjn" Jul 07 14:26:38.580309 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:38.580226 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/648e2b2f-08a5-4a83-b9aa-319ab2527ae4-installation-pull-secrets\") pod \"image-registry-687956bbfb-m9bjn\" (UID: \"648e2b2f-08a5-4a83-b9aa-319ab2527ae4\") " pod="openshift-image-registry/image-registry-687956bbfb-m9bjn" Jul 07 14:26:38.580309 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:38.580254 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/648e2b2f-08a5-4a83-b9aa-319ab2527ae4-bound-sa-token\") pod \"image-registry-687956bbfb-m9bjn\" (UID: \"648e2b2f-08a5-4a83-b9aa-319ab2527ae4\") " pod="openshift-image-registry/image-registry-687956bbfb-m9bjn" Jul 07 14:26:38.580309 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:38.580280 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/648e2b2f-08a5-4a83-b9aa-319ab2527ae4-registry-tls\") pod \"image-registry-687956bbfb-m9bjn\" (UID: \"648e2b2f-08a5-4a83-b9aa-319ab2527ae4\") " pod="openshift-image-registry/image-registry-687956bbfb-m9bjn" Jul 07 14:26:38.580309 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:38.580302 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/648e2b2f-08a5-4a83-b9aa-319ab2527ae4-ca-trust-extracted\") pod \"image-registry-687956bbfb-m9bjn\" (UID: \"648e2b2f-08a5-4a83-b9aa-319ab2527ae4\") " pod="openshift-image-registry/image-registry-687956bbfb-m9bjn" Jul 07 14:26:38.580491 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:38.580330 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3d597bc9-74bf-4809-a33d-0a28dec34cb1-metrics-tls\") pod \"dns-default-5ngmq\" (UID: \"3d597bc9-74bf-4809-a33d-0a28dec34cb1\") " pod="openshift-dns/dns-default-5ngmq" Jul 07 14:26:38.580491 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:38.580373 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8c3755dc-b805-4f6a-8c8e-6ec9077a8efb-cert\") pod \"ingress-canary-8dtp8\" (UID: \"8c3755dc-b805-4f6a-8c8e-6ec9077a8efb\") " pod="openshift-ingress-canary/ingress-canary-8dtp8" Jul 07 14:26:38.580491 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:38.580447 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h4tnw\" (UniqueName: \"kubernetes.io/projected/648e2b2f-08a5-4a83-b9aa-319ab2527ae4-kube-api-access-h4tnw\") pod \"image-registry-687956bbfb-m9bjn\" (UID: \"648e2b2f-08a5-4a83-b9aa-319ab2527ae4\") " pod="openshift-image-registry/image-registry-687956bbfb-m9bjn" Jul 07 14:26:38.580642 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:38.580487 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/648e2b2f-08a5-4a83-b9aa-319ab2527ae4-registry-certificates\") pod \"image-registry-687956bbfb-m9bjn\" (UID: \"648e2b2f-08a5-4a83-b9aa-319ab2527ae4\") " pod="openshift-image-registry/image-registry-687956bbfb-m9bjn" Jul 07 14:26:38.583636 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:38.583616 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-8dtp8"] Jul 07 14:26:38.592781 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:38.592756 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-5ngmq"] Jul 07 14:26:38.681841 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:38.681813 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rl7qw\" (UniqueName: \"kubernetes.io/projected/3d597bc9-74bf-4809-a33d-0a28dec34cb1-kube-api-access-rl7qw\") pod \"dns-default-5ngmq\" (UID: \"3d597bc9-74bf-4809-a33d-0a28dec34cb1\") " pod="openshift-dns/dns-default-5ngmq" Jul 07 14:26:38.681955 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:38.681851 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/648e2b2f-08a5-4a83-b9aa-319ab2527ae4-image-registry-private-configuration\") pod \"image-registry-687956bbfb-m9bjn\" (UID: \"648e2b2f-08a5-4a83-b9aa-319ab2527ae4\") " pod="openshift-image-registry/image-registry-687956bbfb-m9bjn" Jul 07 14:26:38.681955 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:38.681877 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tr24x\" (UniqueName: \"kubernetes.io/projected/8c3755dc-b805-4f6a-8c8e-6ec9077a8efb-kube-api-access-tr24x\") pod \"ingress-canary-8dtp8\" (UID: \"8c3755dc-b805-4f6a-8c8e-6ec9077a8efb\") " pod="openshift-ingress-canary/ingress-canary-8dtp8" Jul 07 14:26:38.681955 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:38.681903 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3d597bc9-74bf-4809-a33d-0a28dec34cb1-tmp-dir\") pod \"dns-default-5ngmq\" (UID: \"3d597bc9-74bf-4809-a33d-0a28dec34cb1\") " pod="openshift-dns/dns-default-5ngmq" Jul 07 14:26:38.681955 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:38.681928 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/648e2b2f-08a5-4a83-b9aa-319ab2527ae4-trusted-ca\") pod \"image-registry-687956bbfb-m9bjn\" (UID: \"648e2b2f-08a5-4a83-b9aa-319ab2527ae4\") " pod="openshift-image-registry/image-registry-687956bbfb-m9bjn" Jul 07 14:26:38.682110 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:38.681986 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/648e2b2f-08a5-4a83-b9aa-319ab2527ae4-installation-pull-secrets\") pod \"image-registry-687956bbfb-m9bjn\" (UID: \"648e2b2f-08a5-4a83-b9aa-319ab2527ae4\") " pod="openshift-image-registry/image-registry-687956bbfb-m9bjn" Jul 07 14:26:38.682110 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:38.682009 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/648e2b2f-08a5-4a83-b9aa-319ab2527ae4-bound-sa-token\") pod \"image-registry-687956bbfb-m9bjn\" (UID: \"648e2b2f-08a5-4a83-b9aa-319ab2527ae4\") " pod="openshift-image-registry/image-registry-687956bbfb-m9bjn" Jul 07 14:26:38.682110 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:38.682034 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/648e2b2f-08a5-4a83-b9aa-319ab2527ae4-registry-tls\") pod \"image-registry-687956bbfb-m9bjn\" (UID: \"648e2b2f-08a5-4a83-b9aa-319ab2527ae4\") " pod="openshift-image-registry/image-registry-687956bbfb-m9bjn" Jul 07 14:26:38.682110 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:38.682055 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/648e2b2f-08a5-4a83-b9aa-319ab2527ae4-ca-trust-extracted\") pod \"image-registry-687956bbfb-m9bjn\" (UID: \"648e2b2f-08a5-4a83-b9aa-319ab2527ae4\") " pod="openshift-image-registry/image-registry-687956bbfb-m9bjn" Jul 07 14:26:38.682110 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:38.682082 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3d597bc9-74bf-4809-a33d-0a28dec34cb1-metrics-tls\") pod \"dns-default-5ngmq\" (UID: \"3d597bc9-74bf-4809-a33d-0a28dec34cb1\") " pod="openshift-dns/dns-default-5ngmq" Jul 07 14:26:38.682110 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:38.682103 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8c3755dc-b805-4f6a-8c8e-6ec9077a8efb-cert\") pod \"ingress-canary-8dtp8\" (UID: \"8c3755dc-b805-4f6a-8c8e-6ec9077a8efb\") " pod="openshift-ingress-canary/ingress-canary-8dtp8" Jul 07 14:26:38.683033 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:38.682566 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3d597bc9-74bf-4809-a33d-0a28dec34cb1-tmp-dir\") pod \"dns-default-5ngmq\" (UID: \"3d597bc9-74bf-4809-a33d-0a28dec34cb1\") " pod="openshift-dns/dns-default-5ngmq" Jul 07 14:26:38.683033 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:38.682619 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h4tnw\" (UniqueName: \"kubernetes.io/projected/648e2b2f-08a5-4a83-b9aa-319ab2527ae4-kube-api-access-h4tnw\") pod \"image-registry-687956bbfb-m9bjn\" (UID: \"648e2b2f-08a5-4a83-b9aa-319ab2527ae4\") " pod="openshift-image-registry/image-registry-687956bbfb-m9bjn" Jul 07 14:26:38.683033 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:38.682678 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/648e2b2f-08a5-4a83-b9aa-319ab2527ae4-registry-certificates\") pod \"image-registry-687956bbfb-m9bjn\" (UID: \"648e2b2f-08a5-4a83-b9aa-319ab2527ae4\") " pod="openshift-image-registry/image-registry-687956bbfb-m9bjn" Jul 07 14:26:38.683033 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:38.682711 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/3d597bc9-74bf-4809-a33d-0a28dec34cb1-config-volume\") pod \"dns-default-5ngmq\" (UID: \"3d597bc9-74bf-4809-a33d-0a28dec34cb1\") " pod="openshift-dns/dns-default-5ngmq" Jul 07 14:26:38.683033 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:38.682718 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/648e2b2f-08a5-4a83-b9aa-319ab2527ae4-ca-trust-extracted\") pod \"image-registry-687956bbfb-m9bjn\" (UID: \"648e2b2f-08a5-4a83-b9aa-319ab2527ae4\") " pod="openshift-image-registry/image-registry-687956bbfb-m9bjn" Jul 07 14:26:38.683033 ip-10-0-134-42 kubenswrapper[2557]: E0707 14:26:38.682800 2557 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 07 14:26:38.683033 ip-10-0-134-42 kubenswrapper[2557]: E0707 14:26:38.682865 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3d597bc9-74bf-4809-a33d-0a28dec34cb1-metrics-tls podName:3d597bc9-74bf-4809-a33d-0a28dec34cb1 nodeName:}" failed. No retries permitted until 2026-07-07 14:26:39.182846828 +0000 UTC m=+35.891985203 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/3d597bc9-74bf-4809-a33d-0a28dec34cb1-metrics-tls") pod "dns-default-5ngmq" (UID: "3d597bc9-74bf-4809-a33d-0a28dec34cb1") : secret "dns-default-metrics-tls" not found Jul 07 14:26:38.683033 ip-10-0-134-42 kubenswrapper[2557]: E0707 14:26:38.682803 2557 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 07 14:26:38.683033 ip-10-0-134-42 kubenswrapper[2557]: E0707 14:26:38.682960 2557 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-687956bbfb-m9bjn: secret "image-registry-tls" not found Jul 07 14:26:38.683033 ip-10-0-134-42 kubenswrapper[2557]: E0707 14:26:38.682998 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/648e2b2f-08a5-4a83-b9aa-319ab2527ae4-registry-tls podName:648e2b2f-08a5-4a83-b9aa-319ab2527ae4 nodeName:}" failed. No retries permitted until 2026-07-07 14:26:39.182985185 +0000 UTC m=+35.892123557 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/648e2b2f-08a5-4a83-b9aa-319ab2527ae4-registry-tls") pod "image-registry-687956bbfb-m9bjn" (UID: "648e2b2f-08a5-4a83-b9aa-319ab2527ae4") : secret "image-registry-tls" not found Jul 07 14:26:38.683534 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:38.683214 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/3d597bc9-74bf-4809-a33d-0a28dec34cb1-config-volume\") pod \"dns-default-5ngmq\" (UID: \"3d597bc9-74bf-4809-a33d-0a28dec34cb1\") " pod="openshift-dns/dns-default-5ngmq" Jul 07 14:26:38.683534 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:38.683318 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/648e2b2f-08a5-4a83-b9aa-319ab2527ae4-trusted-ca\") pod \"image-registry-687956bbfb-m9bjn\" (UID: \"648e2b2f-08a5-4a83-b9aa-319ab2527ae4\") " pod="openshift-image-registry/image-registry-687956bbfb-m9bjn" Jul 07 14:26:38.683534 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:38.683414 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/648e2b2f-08a5-4a83-b9aa-319ab2527ae4-registry-certificates\") pod \"image-registry-687956bbfb-m9bjn\" (UID: \"648e2b2f-08a5-4a83-b9aa-319ab2527ae4\") " pod="openshift-image-registry/image-registry-687956bbfb-m9bjn" Jul 07 14:26:38.686828 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:38.686784 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/648e2b2f-08a5-4a83-b9aa-319ab2527ae4-installation-pull-secrets\") pod \"image-registry-687956bbfb-m9bjn\" (UID: \"648e2b2f-08a5-4a83-b9aa-319ab2527ae4\") " pod="openshift-image-registry/image-registry-687956bbfb-m9bjn" Jul 07 14:26:38.686828 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:38.686812 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/648e2b2f-08a5-4a83-b9aa-319ab2527ae4-image-registry-private-configuration\") pod \"image-registry-687956bbfb-m9bjn\" (UID: \"648e2b2f-08a5-4a83-b9aa-319ab2527ae4\") " pod="openshift-image-registry/image-registry-687956bbfb-m9bjn" Jul 07 14:26:38.686967 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:38.686946 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8c3755dc-b805-4f6a-8c8e-6ec9077a8efb-cert\") pod \"ingress-canary-8dtp8\" (UID: \"8c3755dc-b805-4f6a-8c8e-6ec9077a8efb\") " pod="openshift-ingress-canary/ingress-canary-8dtp8" Jul 07 14:26:38.689875 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:38.689828 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/648e2b2f-08a5-4a83-b9aa-319ab2527ae4-bound-sa-token\") pod \"image-registry-687956bbfb-m9bjn\" (UID: \"648e2b2f-08a5-4a83-b9aa-319ab2527ae4\") " pod="openshift-image-registry/image-registry-687956bbfb-m9bjn" Jul 07 14:26:38.689875 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:38.689837 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tr24x\" (UniqueName: \"kubernetes.io/projected/8c3755dc-b805-4f6a-8c8e-6ec9077a8efb-kube-api-access-tr24x\") pod \"ingress-canary-8dtp8\" (UID: \"8c3755dc-b805-4f6a-8c8e-6ec9077a8efb\") " pod="openshift-ingress-canary/ingress-canary-8dtp8" Jul 07 14:26:38.691014 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:38.690993 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h4tnw\" (UniqueName: \"kubernetes.io/projected/648e2b2f-08a5-4a83-b9aa-319ab2527ae4-kube-api-access-h4tnw\") pod \"image-registry-687956bbfb-m9bjn\" (UID: \"648e2b2f-08a5-4a83-b9aa-319ab2527ae4\") " pod="openshift-image-registry/image-registry-687956bbfb-m9bjn" Jul 07 14:26:38.691132 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:38.691052 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rl7qw\" (UniqueName: \"kubernetes.io/projected/3d597bc9-74bf-4809-a33d-0a28dec34cb1-kube-api-access-rl7qw\") pod \"dns-default-5ngmq\" (UID: \"3d597bc9-74bf-4809-a33d-0a28dec34cb1\") " pod="openshift-dns/dns-default-5ngmq" Jul 07 14:26:38.883069 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:38.883039 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-8dtp8" Jul 07 14:26:39.019562 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:39.019504 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-8dtp8"] Jul 07 14:26:39.025282 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:39.025248 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8c3755dc_b805_4f6a_8c8e_6ec9077a8efb.slice/crio-77fe232fc648e20856cf324ce6bf798f5cde6a9aa5686e19fe87a96dc390de72 WatchSource:0}: Error finding container 77fe232fc648e20856cf324ce6bf798f5cde6a9aa5686e19fe87a96dc390de72: Status 404 returned error can't find the container with id 77fe232fc648e20856cf324ce6bf798f5cde6a9aa5686e19fe87a96dc390de72 Jul 07 14:26:39.077750 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:39.077711 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-8dtp8" event={"ID":"8c3755dc-b805-4f6a-8c8e-6ec9077a8efb","Type":"ContainerStarted","Data":"77fe232fc648e20856cf324ce6bf798f5cde6a9aa5686e19fe87a96dc390de72"} Jul 07 14:26:39.186978 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:39.186910 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3d597bc9-74bf-4809-a33d-0a28dec34cb1-metrics-tls\") pod \"dns-default-5ngmq\" (UID: \"3d597bc9-74bf-4809-a33d-0a28dec34cb1\") " pod="openshift-dns/dns-default-5ngmq" Jul 07 14:26:39.187127 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:39.187082 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/648e2b2f-08a5-4a83-b9aa-319ab2527ae4-registry-tls\") pod \"image-registry-687956bbfb-m9bjn\" (UID: \"648e2b2f-08a5-4a83-b9aa-319ab2527ae4\") " pod="openshift-image-registry/image-registry-687956bbfb-m9bjn" Jul 07 14:26:39.187127 ip-10-0-134-42 kubenswrapper[2557]: E0707 14:26:39.187094 2557 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 07 14:26:39.187230 ip-10-0-134-42 kubenswrapper[2557]: E0707 14:26:39.187164 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3d597bc9-74bf-4809-a33d-0a28dec34cb1-metrics-tls podName:3d597bc9-74bf-4809-a33d-0a28dec34cb1 nodeName:}" failed. No retries permitted until 2026-07-07 14:26:40.187140849 +0000 UTC m=+36.896279208 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/3d597bc9-74bf-4809-a33d-0a28dec34cb1-metrics-tls") pod "dns-default-5ngmq" (UID: "3d597bc9-74bf-4809-a33d-0a28dec34cb1") : secret "dns-default-metrics-tls" not found Jul 07 14:26:39.187230 ip-10-0-134-42 kubenswrapper[2557]: E0707 14:26:39.187171 2557 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 07 14:26:39.187230 ip-10-0-134-42 kubenswrapper[2557]: E0707 14:26:39.187183 2557 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-687956bbfb-m9bjn: secret "image-registry-tls" not found Jul 07 14:26:39.187230 ip-10-0-134-42 kubenswrapper[2557]: E0707 14:26:39.187222 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/648e2b2f-08a5-4a83-b9aa-319ab2527ae4-registry-tls podName:648e2b2f-08a5-4a83-b9aa-319ab2527ae4 nodeName:}" failed. No retries permitted until 2026-07-07 14:26:40.187208473 +0000 UTC m=+36.896346832 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/648e2b2f-08a5-4a83-b9aa-319ab2527ae4-registry-tls") pod "image-registry-687956bbfb-m9bjn" (UID: "648e2b2f-08a5-4a83-b9aa-319ab2527ae4") : secret "image-registry-tls" not found Jul 07 14:26:39.280317 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:39.280284 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:26:39.436213 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:39.436177 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:26:39.518446 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:39.518370 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:26:39.600659 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:39.600630 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:26:39.663550 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:39.663508 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:26:39.771027 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:39.770957 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:26:39.828987 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:39.828950 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:26:39.829837 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:39.829816 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bqthq" Jul 07 14:26:39.829837 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:39.829852 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s4p8l" Jul 07 14:26:39.830006 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:39.829919 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hvlzx" Jul 07 14:26:39.831870 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:39.831848 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 07 14:26:39.831947 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:39.831878 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 07 14:26:39.832466 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:39.832449 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 07 14:26:39.832561 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:39.832536 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-29x7z\"" Jul 07 14:26:39.832622 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:39.832535 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 07 14:26:39.832672 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:39.832458 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-ps6h4\"" Jul 07 14:26:40.195381 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:40.195348 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/648e2b2f-08a5-4a83-b9aa-319ab2527ae4-registry-tls\") pod \"image-registry-687956bbfb-m9bjn\" (UID: \"648e2b2f-08a5-4a83-b9aa-319ab2527ae4\") " pod="openshift-image-registry/image-registry-687956bbfb-m9bjn" Jul 07 14:26:40.196274 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:40.195405 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3d597bc9-74bf-4809-a33d-0a28dec34cb1-metrics-tls\") pod \"dns-default-5ngmq\" (UID: \"3d597bc9-74bf-4809-a33d-0a28dec34cb1\") " pod="openshift-dns/dns-default-5ngmq" Jul 07 14:26:40.196274 ip-10-0-134-42 kubenswrapper[2557]: E0707 14:26:40.195510 2557 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 07 14:26:40.196274 ip-10-0-134-42 kubenswrapper[2557]: E0707 14:26:40.195538 2557 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 07 14:26:40.196274 ip-10-0-134-42 kubenswrapper[2557]: E0707 14:26:40.195547 2557 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-687956bbfb-m9bjn: secret "image-registry-tls" not found Jul 07 14:26:40.196274 ip-10-0-134-42 kubenswrapper[2557]: E0707 14:26:40.195607 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/648e2b2f-08a5-4a83-b9aa-319ab2527ae4-registry-tls podName:648e2b2f-08a5-4a83-b9aa-319ab2527ae4 nodeName:}" failed. No retries permitted until 2026-07-07 14:26:42.19558716 +0000 UTC m=+38.904725520 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/648e2b2f-08a5-4a83-b9aa-319ab2527ae4-registry-tls") pod "image-registry-687956bbfb-m9bjn" (UID: "648e2b2f-08a5-4a83-b9aa-319ab2527ae4") : secret "image-registry-tls" not found Jul 07 14:26:40.196274 ip-10-0-134-42 kubenswrapper[2557]: E0707 14:26:40.195623 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3d597bc9-74bf-4809-a33d-0a28dec34cb1-metrics-tls podName:3d597bc9-74bf-4809-a33d-0a28dec34cb1 nodeName:}" failed. No retries permitted until 2026-07-07 14:26:42.195615841 +0000 UTC m=+38.904754199 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/3d597bc9-74bf-4809-a33d-0a28dec34cb1-metrics-tls") pod "dns-default-5ngmq" (UID: "3d597bc9-74bf-4809-a33d-0a28dec34cb1") : secret "dns-default-metrics-tls" not found Jul 07 14:26:40.533538 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:40.533445 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:26:40.928018 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:40.927983 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:26:41.027614 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:41.027581 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:26:42.212955 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:42.212919 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/648e2b2f-08a5-4a83-b9aa-319ab2527ae4-registry-tls\") pod \"image-registry-687956bbfb-m9bjn\" (UID: \"648e2b2f-08a5-4a83-b9aa-319ab2527ae4\") " pod="openshift-image-registry/image-registry-687956bbfb-m9bjn" Jul 07 14:26:42.213370 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:42.212970 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3d597bc9-74bf-4809-a33d-0a28dec34cb1-metrics-tls\") pod \"dns-default-5ngmq\" (UID: \"3d597bc9-74bf-4809-a33d-0a28dec34cb1\") " pod="openshift-dns/dns-default-5ngmq" Jul 07 14:26:42.213370 ip-10-0-134-42 kubenswrapper[2557]: E0707 14:26:42.213041 2557 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 07 14:26:42.213370 ip-10-0-134-42 kubenswrapper[2557]: E0707 14:26:42.213054 2557 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-687956bbfb-m9bjn: secret "image-registry-tls" not found Jul 07 14:26:42.213370 ip-10-0-134-42 kubenswrapper[2557]: E0707 14:26:42.213113 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/648e2b2f-08a5-4a83-b9aa-319ab2527ae4-registry-tls podName:648e2b2f-08a5-4a83-b9aa-319ab2527ae4 nodeName:}" failed. No retries permitted until 2026-07-07 14:26:46.213095527 +0000 UTC m=+42.922233901 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/648e2b2f-08a5-4a83-b9aa-319ab2527ae4-registry-tls") pod "image-registry-687956bbfb-m9bjn" (UID: "648e2b2f-08a5-4a83-b9aa-319ab2527ae4") : secret "image-registry-tls" not found Jul 07 14:26:42.213370 ip-10-0-134-42 kubenswrapper[2557]: E0707 14:26:42.213142 2557 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 07 14:26:42.213370 ip-10-0-134-42 kubenswrapper[2557]: E0707 14:26:42.213208 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3d597bc9-74bf-4809-a33d-0a28dec34cb1-metrics-tls podName:3d597bc9-74bf-4809-a33d-0a28dec34cb1 nodeName:}" failed. No retries permitted until 2026-07-07 14:26:46.213190748 +0000 UTC m=+42.922329123 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/3d597bc9-74bf-4809-a33d-0a28dec34cb1-metrics-tls") pod "dns-default-5ngmq" (UID: "3d597bc9-74bf-4809-a33d-0a28dec34cb1") : secret "dns-default-metrics-tls" not found Jul 07 14:26:42.834076 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:42.834047 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:26:42.934679 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:42.934639 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:26:43.089710 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:43.089678 2557 generic.go:356] "Generic (PLEG): container finished" podID="d3b76148-a347-4b4e-9ba5-fb4c41ebc389" containerID="c744a8c7ceaf33f0dbd6842a0f17bf1bffa3501b591124b36163e1bf451e254f" exitCode=0 Jul 07 14:26:43.089861 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:43.089761 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-shb4c" event={"ID":"d3b76148-a347-4b4e-9ba5-fb4c41ebc389","Type":"ContainerDied","Data":"c744a8c7ceaf33f0dbd6842a0f17bf1bffa3501b591124b36163e1bf451e254f"} Jul 07 14:26:43.091108 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:43.091085 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-8dtp8" event={"ID":"8c3755dc-b805-4f6a-8c8e-6ec9077a8efb","Type":"ContainerStarted","Data":"3eeed998b09ae14c0792ca1dd9263614922f844cc90fc555e205cad65cd4ea02"} Jul 07 14:26:43.125298 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:43.125263 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-8dtp8" podStartSLOduration=1.357774401 podStartE2EDuration="5.125250563s" totalImagesPullingTime="3.767476162s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 14:26:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:26:43.124456081 +0000 UTC m=+39.833594460" watchObservedRunningTime="2026-07-07 14:26:43.125250563 +0000 UTC m=+39.834388942" Jul 07 14:26:44.097679 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:44.097640 2557 generic.go:356] "Generic (PLEG): container finished" podID="d3b76148-a347-4b4e-9ba5-fb4c41ebc389" containerID="50c5fe7cc71bb39f54ec35f55993201001c90cb6c15cf403d633483e62825642" exitCode=0 Jul 07 14:26:44.098095 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:44.097695 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-shb4c" event={"ID":"d3b76148-a347-4b4e-9ba5-fb4c41ebc389","Type":"ContainerDied","Data":"50c5fe7cc71bb39f54ec35f55993201001c90cb6c15cf403d633483e62825642"} Jul 07 14:26:45.103819 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:45.103781 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-shb4c" event={"ID":"d3b76148-a347-4b4e-9ba5-fb4c41ebc389","Type":"ContainerStarted","Data":"e2b9ed24210db1d07fe91b732585ce29265e9e229255ae05f5a114b42454cac5"} Jul 07 14:26:45.130278 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:45.130235 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-shb4c" podStartSLOduration=7.390997249 podStartE2EDuration="41.130218498s" totalImagesPullingTime="33.739221249s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 14:26:04 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:26:45.12870998 +0000 UTC m=+41.837848361" watchObservedRunningTime="2026-07-07 14:26:45.130218498 +0000 UTC m=+41.839356877" Jul 07 14:26:45.255842 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:45.255814 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:26:45.356900 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:45.356816 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:26:46.244126 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:46.244090 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/648e2b2f-08a5-4a83-b9aa-319ab2527ae4-registry-tls\") pod \"image-registry-687956bbfb-m9bjn\" (UID: \"648e2b2f-08a5-4a83-b9aa-319ab2527ae4\") " pod="openshift-image-registry/image-registry-687956bbfb-m9bjn" Jul 07 14:26:46.244126 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:46.244127 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3d597bc9-74bf-4809-a33d-0a28dec34cb1-metrics-tls\") pod \"dns-default-5ngmq\" (UID: \"3d597bc9-74bf-4809-a33d-0a28dec34cb1\") " pod="openshift-dns/dns-default-5ngmq" Jul 07 14:26:46.244539 ip-10-0-134-42 kubenswrapper[2557]: E0707 14:26:46.244226 2557 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 07 14:26:46.244539 ip-10-0-134-42 kubenswrapper[2557]: E0707 14:26:46.244233 2557 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 07 14:26:46.244539 ip-10-0-134-42 kubenswrapper[2557]: E0707 14:26:46.244250 2557 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-687956bbfb-m9bjn: secret "image-registry-tls" not found Jul 07 14:26:46.244539 ip-10-0-134-42 kubenswrapper[2557]: E0707 14:26:46.244274 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3d597bc9-74bf-4809-a33d-0a28dec34cb1-metrics-tls podName:3d597bc9-74bf-4809-a33d-0a28dec34cb1 nodeName:}" failed. No retries permitted until 2026-07-07 14:26:54.244261763 +0000 UTC m=+50.953400122 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/3d597bc9-74bf-4809-a33d-0a28dec34cb1-metrics-tls") pod "dns-default-5ngmq" (UID: "3d597bc9-74bf-4809-a33d-0a28dec34cb1") : secret "dns-default-metrics-tls" not found Jul 07 14:26:46.244539 ip-10-0-134-42 kubenswrapper[2557]: E0707 14:26:46.244293 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/648e2b2f-08a5-4a83-b9aa-319ab2527ae4-registry-tls podName:648e2b2f-08a5-4a83-b9aa-319ab2527ae4 nodeName:}" failed. No retries permitted until 2026-07-07 14:26:54.244280099 +0000 UTC m=+50.953418457 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/648e2b2f-08a5-4a83-b9aa-319ab2527ae4-registry-tls") pod "image-registry-687956bbfb-m9bjn" (UID: "648e2b2f-08a5-4a83-b9aa-319ab2527ae4") : secret "image-registry-tls" not found Jul 07 14:26:46.385053 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:46.385022 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:26:46.482912 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:46.482878 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:26:51.308097 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:51.308057 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:26:51.371301 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:51.371263 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:26:54.300595 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:54.300560 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/648e2b2f-08a5-4a83-b9aa-319ab2527ae4-registry-tls\") pod \"image-registry-687956bbfb-m9bjn\" (UID: \"648e2b2f-08a5-4a83-b9aa-319ab2527ae4\") " pod="openshift-image-registry/image-registry-687956bbfb-m9bjn" Jul 07 14:26:54.300595 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:54.300600 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3d597bc9-74bf-4809-a33d-0a28dec34cb1-metrics-tls\") pod \"dns-default-5ngmq\" (UID: \"3d597bc9-74bf-4809-a33d-0a28dec34cb1\") " pod="openshift-dns/dns-default-5ngmq" Jul 07 14:26:54.300998 ip-10-0-134-42 kubenswrapper[2557]: E0707 14:26:54.300708 2557 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 07 14:26:54.300998 ip-10-0-134-42 kubenswrapper[2557]: E0707 14:26:54.300726 2557 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-687956bbfb-m9bjn: secret "image-registry-tls" not found Jul 07 14:26:54.300998 ip-10-0-134-42 kubenswrapper[2557]: E0707 14:26:54.300735 2557 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 07 14:26:54.300998 ip-10-0-134-42 kubenswrapper[2557]: E0707 14:26:54.300791 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/648e2b2f-08a5-4a83-b9aa-319ab2527ae4-registry-tls podName:648e2b2f-08a5-4a83-b9aa-319ab2527ae4 nodeName:}" failed. No retries permitted until 2026-07-07 14:27:10.300776257 +0000 UTC m=+67.009914615 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/648e2b2f-08a5-4a83-b9aa-319ab2527ae4-registry-tls") pod "image-registry-687956bbfb-m9bjn" (UID: "648e2b2f-08a5-4a83-b9aa-319ab2527ae4") : secret "image-registry-tls" not found Jul 07 14:26:54.300998 ip-10-0-134-42 kubenswrapper[2557]: E0707 14:26:54.300806 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3d597bc9-74bf-4809-a33d-0a28dec34cb1-metrics-tls podName:3d597bc9-74bf-4809-a33d-0a28dec34cb1 nodeName:}" failed. No retries permitted until 2026-07-07 14:27:10.300799955 +0000 UTC m=+67.009938313 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/3d597bc9-74bf-4809-a33d-0a28dec34cb1-metrics-tls") pod "dns-default-5ngmq" (UID: "3d597bc9-74bf-4809-a33d-0a28dec34cb1") : secret "dns-default-metrics-tls" not found Jul 07 14:26:56.467052 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:56.467016 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:26:58.613138 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:58.613108 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-64786d9cbb-bkdv4"] Jul 07 14:26:58.638338 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:58.638315 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-64786d9cbb-bkdv4"] Jul 07 14:26:58.638460 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:58.638401 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-bkdv4" Jul 07 14:26:58.641559 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:58.641512 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 07 14:26:58.641679 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:58.641539 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\"" Jul 07 14:26:58.642099 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:58.642083 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 07 14:26:58.643225 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:58.643189 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\"" Jul 07 14:26:58.643350 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:58.643248 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-xj2t8\"" Jul 07 14:26:58.705083 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:58.705061 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-7c58ff4df-55gs2"] Jul 07 14:26:58.706728 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:58.706714 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-7c58ff4df-55gs2" Jul 07 14:26:58.710599 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:58.710575 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-5c847dcfc4-xpdmb"] Jul 07 14:26:58.711048 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:58.711026 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\"" Jul 07 14:26:58.711165 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:58.711050 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 07 14:26:58.711165 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:58.711074 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\"" Jul 07 14:26:58.711165 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:58.711116 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-6wnfq\"" Jul 07 14:26:58.711321 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:58.711294 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\"" Jul 07 14:26:58.711391 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:58.711376 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\"" Jul 07 14:26:58.711482 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:58.711468 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 07 14:26:58.712493 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:58.712476 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5c847dcfc4-xpdmb" Jul 07 14:26:58.716513 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:58.716496 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\"" Jul 07 14:26:58.716635 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:58.716561 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 07 14:26:58.716635 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:58.716603 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 07 14:26:58.716803 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:58.716788 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\"" Jul 07 14:26:58.716848 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:58.716802 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-gw7jk\"" Jul 07 14:26:58.722895 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:58.722878 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\"" Jul 07 14:26:58.728592 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:58.728568 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-7c58ff4df-55gs2"] Jul 07 14:26:58.729633 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:58.729613 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5c847dcfc4-xpdmb"] Jul 07 14:26:58.733065 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:58.733046 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/d75c5911-092c-408a-83a7-871dfb379d28-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-bkdv4\" (UID: \"d75c5911-092c-408a-83a7-871dfb379d28\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-bkdv4" Jul 07 14:26:58.733147 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:58.733079 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bc2n6\" (UniqueName: \"kubernetes.io/projected/d75c5911-092c-408a-83a7-871dfb379d28-kube-api-access-bc2n6\") pod \"cluster-monitoring-operator-64786d9cbb-bkdv4\" (UID: \"d75c5911-092c-408a-83a7-871dfb379d28\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-bkdv4" Jul 07 14:26:58.733183 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:58.733157 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/d75c5911-092c-408a-83a7-871dfb379d28-telemetry-config\") pod \"cluster-monitoring-operator-64786d9cbb-bkdv4\" (UID: \"d75c5911-092c-408a-83a7-871dfb379d28\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-bkdv4" Jul 07 14:26:58.834126 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:58.834094 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/450aeb2e-26e4-4ebb-8006-eb9c43c6e2d1-metrics-certs\") pod \"router-default-7c58ff4df-55gs2\" (UID: \"450aeb2e-26e4-4ebb-8006-eb9c43c6e2d1\") " pod="openshift-ingress/router-default-7c58ff4df-55gs2" Jul 07 14:26:58.834300 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:58.834138 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/450aeb2e-26e4-4ebb-8006-eb9c43c6e2d1-stats-auth\") pod \"router-default-7c58ff4df-55gs2\" (UID: \"450aeb2e-26e4-4ebb-8006-eb9c43c6e2d1\") " pod="openshift-ingress/router-default-7c58ff4df-55gs2" Jul 07 14:26:58.834300 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:58.834183 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/c7028698-fc37-401b-966d-359bad5c835e-serving-cert\") pod \"insights-operator-5c847dcfc4-xpdmb\" (UID: \"c7028698-fc37-401b-966d-359bad5c835e\") " pod="openshift-insights/insights-operator-5c847dcfc4-xpdmb" Jul 07 14:26:58.834300 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:58.834226 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/d75c5911-092c-408a-83a7-871dfb379d28-telemetry-config\") pod \"cluster-monitoring-operator-64786d9cbb-bkdv4\" (UID: \"d75c5911-092c-408a-83a7-871dfb379d28\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-bkdv4" Jul 07 14:26:58.834300 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:58.834276 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/d75c5911-092c-408a-83a7-871dfb379d28-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-bkdv4\" (UID: \"d75c5911-092c-408a-83a7-871dfb379d28\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-bkdv4" Jul 07 14:26:58.834466 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:58.834305 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/450aeb2e-26e4-4ebb-8006-eb9c43c6e2d1-default-certificate\") pod \"router-default-7c58ff4df-55gs2\" (UID: \"450aeb2e-26e4-4ebb-8006-eb9c43c6e2d1\") " pod="openshift-ingress/router-default-7c58ff4df-55gs2" Jul 07 14:26:58.834466 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:58.834322 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fgrmx\" (UniqueName: \"kubernetes.io/projected/c7028698-fc37-401b-966d-359bad5c835e-kube-api-access-fgrmx\") pod \"insights-operator-5c847dcfc4-xpdmb\" (UID: \"c7028698-fc37-401b-966d-359bad5c835e\") " pod="openshift-insights/insights-operator-5c847dcfc4-xpdmb" Jul 07 14:26:58.834466 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:58.834350 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/c7028698-fc37-401b-966d-359bad5c835e-snapshots\") pod \"insights-operator-5c847dcfc4-xpdmb\" (UID: \"c7028698-fc37-401b-966d-359bad5c835e\") " pod="openshift-insights/insights-operator-5c847dcfc4-xpdmb" Jul 07 14:26:58.834723 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:58.834697 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z2wl5\" (UniqueName: \"kubernetes.io/projected/450aeb2e-26e4-4ebb-8006-eb9c43c6e2d1-kube-api-access-z2wl5\") pod \"router-default-7c58ff4df-55gs2\" (UID: \"450aeb2e-26e4-4ebb-8006-eb9c43c6e2d1\") " pod="openshift-ingress/router-default-7c58ff4df-55gs2" Jul 07 14:26:58.834944 ip-10-0-134-42 kubenswrapper[2557]: E0707 14:26:58.834760 2557 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 07 14:26:58.835021 ip-10-0-134-42 kubenswrapper[2557]: E0707 14:26:58.835010 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d75c5911-092c-408a-83a7-871dfb379d28-cluster-monitoring-operator-tls podName:d75c5911-092c-408a-83a7-871dfb379d28 nodeName:}" failed. No retries permitted until 2026-07-07 14:26:59.334982254 +0000 UTC m=+56.044120616 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/d75c5911-092c-408a-83a7-871dfb379d28-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-bkdv4" (UID: "d75c5911-092c-408a-83a7-871dfb379d28") : secret "cluster-monitoring-operator-tls" not found Jul 07 14:26:58.835320 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:58.834892 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c7028698-fc37-401b-966d-359bad5c835e-trusted-ca-bundle\") pod \"insights-operator-5c847dcfc4-xpdmb\" (UID: \"c7028698-fc37-401b-966d-359bad5c835e\") " pod="openshift-insights/insights-operator-5c847dcfc4-xpdmb" Jul 07 14:26:58.835377 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:58.835329 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/d75c5911-092c-408a-83a7-871dfb379d28-telemetry-config\") pod \"cluster-monitoring-operator-64786d9cbb-bkdv4\" (UID: \"d75c5911-092c-408a-83a7-871dfb379d28\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-bkdv4" Jul 07 14:26:58.835377 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:58.835357 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c7028698-fc37-401b-966d-359bad5c835e-tmp\") pod \"insights-operator-5c847dcfc4-xpdmb\" (UID: \"c7028698-fc37-401b-966d-359bad5c835e\") " pod="openshift-insights/insights-operator-5c847dcfc4-xpdmb" Jul 07 14:26:58.835480 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:58.835398 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/450aeb2e-26e4-4ebb-8006-eb9c43c6e2d1-service-ca-bundle\") pod \"router-default-7c58ff4df-55gs2\" (UID: \"450aeb2e-26e4-4ebb-8006-eb9c43c6e2d1\") " pod="openshift-ingress/router-default-7c58ff4df-55gs2" Jul 07 14:26:58.835480 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:58.835458 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c7028698-fc37-401b-966d-359bad5c835e-service-ca-bundle\") pod \"insights-operator-5c847dcfc4-xpdmb\" (UID: \"c7028698-fc37-401b-966d-359bad5c835e\") " pod="openshift-insights/insights-operator-5c847dcfc4-xpdmb" Jul 07 14:26:58.835592 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:58.835534 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bc2n6\" (UniqueName: \"kubernetes.io/projected/d75c5911-092c-408a-83a7-871dfb379d28-kube-api-access-bc2n6\") pod \"cluster-monitoring-operator-64786d9cbb-bkdv4\" (UID: \"d75c5911-092c-408a-83a7-871dfb379d28\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-bkdv4" Jul 07 14:26:58.847489 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:58.847469 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bc2n6\" (UniqueName: \"kubernetes.io/projected/d75c5911-092c-408a-83a7-871dfb379d28-kube-api-access-bc2n6\") pod \"cluster-monitoring-operator-64786d9cbb-bkdv4\" (UID: \"d75c5911-092c-408a-83a7-871dfb379d28\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-bkdv4" Jul 07 14:26:58.936681 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:58.936630 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/450aeb2e-26e4-4ebb-8006-eb9c43c6e2d1-default-certificate\") pod \"router-default-7c58ff4df-55gs2\" (UID: \"450aeb2e-26e4-4ebb-8006-eb9c43c6e2d1\") " pod="openshift-ingress/router-default-7c58ff4df-55gs2" Jul 07 14:26:58.936681 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:58.936659 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fgrmx\" (UniqueName: \"kubernetes.io/projected/c7028698-fc37-401b-966d-359bad5c835e-kube-api-access-fgrmx\") pod \"insights-operator-5c847dcfc4-xpdmb\" (UID: \"c7028698-fc37-401b-966d-359bad5c835e\") " pod="openshift-insights/insights-operator-5c847dcfc4-xpdmb" Jul 07 14:26:58.936681 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:58.936678 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/c7028698-fc37-401b-966d-359bad5c835e-snapshots\") pod \"insights-operator-5c847dcfc4-xpdmb\" (UID: \"c7028698-fc37-401b-966d-359bad5c835e\") " pod="openshift-insights/insights-operator-5c847dcfc4-xpdmb" Jul 07 14:26:58.936916 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:58.936700 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z2wl5\" (UniqueName: \"kubernetes.io/projected/450aeb2e-26e4-4ebb-8006-eb9c43c6e2d1-kube-api-access-z2wl5\") pod \"router-default-7c58ff4df-55gs2\" (UID: \"450aeb2e-26e4-4ebb-8006-eb9c43c6e2d1\") " pod="openshift-ingress/router-default-7c58ff4df-55gs2" Jul 07 14:26:58.936916 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:58.936726 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c7028698-fc37-401b-966d-359bad5c835e-trusted-ca-bundle\") pod \"insights-operator-5c847dcfc4-xpdmb\" (UID: \"c7028698-fc37-401b-966d-359bad5c835e\") " pod="openshift-insights/insights-operator-5c847dcfc4-xpdmb" Jul 07 14:26:58.936916 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:58.936750 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c7028698-fc37-401b-966d-359bad5c835e-tmp\") pod \"insights-operator-5c847dcfc4-xpdmb\" (UID: \"c7028698-fc37-401b-966d-359bad5c835e\") " pod="openshift-insights/insights-operator-5c847dcfc4-xpdmb" Jul 07 14:26:58.936916 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:58.936772 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/450aeb2e-26e4-4ebb-8006-eb9c43c6e2d1-service-ca-bundle\") pod \"router-default-7c58ff4df-55gs2\" (UID: \"450aeb2e-26e4-4ebb-8006-eb9c43c6e2d1\") " pod="openshift-ingress/router-default-7c58ff4df-55gs2" Jul 07 14:26:58.936916 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:58.936807 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c7028698-fc37-401b-966d-359bad5c835e-service-ca-bundle\") pod \"insights-operator-5c847dcfc4-xpdmb\" (UID: \"c7028698-fc37-401b-966d-359bad5c835e\") " pod="openshift-insights/insights-operator-5c847dcfc4-xpdmb" Jul 07 14:26:58.936916 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:58.936864 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/450aeb2e-26e4-4ebb-8006-eb9c43c6e2d1-metrics-certs\") pod \"router-default-7c58ff4df-55gs2\" (UID: \"450aeb2e-26e4-4ebb-8006-eb9c43c6e2d1\") " pod="openshift-ingress/router-default-7c58ff4df-55gs2" Jul 07 14:26:58.936916 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:58.936896 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/450aeb2e-26e4-4ebb-8006-eb9c43c6e2d1-stats-auth\") pod \"router-default-7c58ff4df-55gs2\" (UID: \"450aeb2e-26e4-4ebb-8006-eb9c43c6e2d1\") " pod="openshift-ingress/router-default-7c58ff4df-55gs2" Jul 07 14:26:58.937257 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:58.936941 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/c7028698-fc37-401b-966d-359bad5c835e-serving-cert\") pod \"insights-operator-5c847dcfc4-xpdmb\" (UID: \"c7028698-fc37-401b-966d-359bad5c835e\") " pod="openshift-insights/insights-operator-5c847dcfc4-xpdmb" Jul 07 14:26:58.937257 ip-10-0-134-42 kubenswrapper[2557]: E0707 14:26:58.937001 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/450aeb2e-26e4-4ebb-8006-eb9c43c6e2d1-service-ca-bundle podName:450aeb2e-26e4-4ebb-8006-eb9c43c6e2d1 nodeName:}" failed. No retries permitted until 2026-07-07 14:26:59.436981867 +0000 UTC m=+56.146120239 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/450aeb2e-26e4-4ebb-8006-eb9c43c6e2d1-service-ca-bundle") pod "router-default-7c58ff4df-55gs2" (UID: "450aeb2e-26e4-4ebb-8006-eb9c43c6e2d1") : configmap references non-existent config key: service-ca.crt Jul 07 14:26:58.937257 ip-10-0-134-42 kubenswrapper[2557]: E0707 14:26:58.937081 2557 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 07 14:26:58.937257 ip-10-0-134-42 kubenswrapper[2557]: E0707 14:26:58.937143 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/450aeb2e-26e4-4ebb-8006-eb9c43c6e2d1-metrics-certs podName:450aeb2e-26e4-4ebb-8006-eb9c43c6e2d1 nodeName:}" failed. No retries permitted until 2026-07-07 14:26:59.437126967 +0000 UTC m=+56.146265329 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/450aeb2e-26e4-4ebb-8006-eb9c43c6e2d1-metrics-certs") pod "router-default-7c58ff4df-55gs2" (UID: "450aeb2e-26e4-4ebb-8006-eb9c43c6e2d1") : secret "router-metrics-certs-default" not found Jul 07 14:26:58.937447 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:58.937340 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/c7028698-fc37-401b-966d-359bad5c835e-snapshots\") pod \"insights-operator-5c847dcfc4-xpdmb\" (UID: \"c7028698-fc37-401b-966d-359bad5c835e\") " pod="openshift-insights/insights-operator-5c847dcfc4-xpdmb" Jul 07 14:26:58.937501 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:58.937487 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c7028698-fc37-401b-966d-359bad5c835e-service-ca-bundle\") pod \"insights-operator-5c847dcfc4-xpdmb\" (UID: \"c7028698-fc37-401b-966d-359bad5c835e\") " pod="openshift-insights/insights-operator-5c847dcfc4-xpdmb" Jul 07 14:26:58.937607 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:58.937556 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c7028698-fc37-401b-966d-359bad5c835e-tmp\") pod \"insights-operator-5c847dcfc4-xpdmb\" (UID: \"c7028698-fc37-401b-966d-359bad5c835e\") " pod="openshift-insights/insights-operator-5c847dcfc4-xpdmb" Jul 07 14:26:58.937684 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:58.937622 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c7028698-fc37-401b-966d-359bad5c835e-trusted-ca-bundle\") pod \"insights-operator-5c847dcfc4-xpdmb\" (UID: \"c7028698-fc37-401b-966d-359bad5c835e\") " pod="openshift-insights/insights-operator-5c847dcfc4-xpdmb" Jul 07 14:26:58.938954 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:58.938932 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/450aeb2e-26e4-4ebb-8006-eb9c43c6e2d1-default-certificate\") pod \"router-default-7c58ff4df-55gs2\" (UID: \"450aeb2e-26e4-4ebb-8006-eb9c43c6e2d1\") " pod="openshift-ingress/router-default-7c58ff4df-55gs2" Jul 07 14:26:58.939045 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:58.938987 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/c7028698-fc37-401b-966d-359bad5c835e-serving-cert\") pod \"insights-operator-5c847dcfc4-xpdmb\" (UID: \"c7028698-fc37-401b-966d-359bad5c835e\") " pod="openshift-insights/insights-operator-5c847dcfc4-xpdmb" Jul 07 14:26:58.939288 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:58.939268 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/450aeb2e-26e4-4ebb-8006-eb9c43c6e2d1-stats-auth\") pod \"router-default-7c58ff4df-55gs2\" (UID: \"450aeb2e-26e4-4ebb-8006-eb9c43c6e2d1\") " pod="openshift-ingress/router-default-7c58ff4df-55gs2" Jul 07 14:26:58.946463 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:58.946443 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z2wl5\" (UniqueName: \"kubernetes.io/projected/450aeb2e-26e4-4ebb-8006-eb9c43c6e2d1-kube-api-access-z2wl5\") pod \"router-default-7c58ff4df-55gs2\" (UID: \"450aeb2e-26e4-4ebb-8006-eb9c43c6e2d1\") " pod="openshift-ingress/router-default-7c58ff4df-55gs2" Jul 07 14:26:58.946614 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:58.946599 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fgrmx\" (UniqueName: \"kubernetes.io/projected/c7028698-fc37-401b-966d-359bad5c835e-kube-api-access-fgrmx\") pod \"insights-operator-5c847dcfc4-xpdmb\" (UID: \"c7028698-fc37-401b-966d-359bad5c835e\") " pod="openshift-insights/insights-operator-5c847dcfc4-xpdmb" Jul 07 14:26:59.020917 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:59.020894 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5c847dcfc4-xpdmb" Jul 07 14:26:59.147380 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:59.147353 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5c847dcfc4-xpdmb"] Jul 07 14:26:59.149977 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:26:59.149958 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc7028698_fc37_401b_966d_359bad5c835e.slice/crio-c45ad87266dd18a1eca5ece1828924e54dbc1fc6b253ecba1736ca73b1e2cdba WatchSource:0}: Error finding container c45ad87266dd18a1eca5ece1828924e54dbc1fc6b253ecba1736ca73b1e2cdba: Status 404 returned error can't find the container with id c45ad87266dd18a1eca5ece1828924e54dbc1fc6b253ecba1736ca73b1e2cdba Jul 07 14:26:59.153799 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:59.153771 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:26:59.245502 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:59.245425 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:26:59.340309 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:59.340277 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/d75c5911-092c-408a-83a7-871dfb379d28-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-bkdv4\" (UID: \"d75c5911-092c-408a-83a7-871dfb379d28\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-bkdv4" Jul 07 14:26:59.340433 ip-10-0-134-42 kubenswrapper[2557]: E0707 14:26:59.340409 2557 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 07 14:26:59.340488 ip-10-0-134-42 kubenswrapper[2557]: E0707 14:26:59.340474 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d75c5911-092c-408a-83a7-871dfb379d28-cluster-monitoring-operator-tls podName:d75c5911-092c-408a-83a7-871dfb379d28 nodeName:}" failed. No retries permitted until 2026-07-07 14:27:00.340454645 +0000 UTC m=+57.049593005 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/d75c5911-092c-408a-83a7-871dfb379d28-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-bkdv4" (UID: "d75c5911-092c-408a-83a7-871dfb379d28") : secret "cluster-monitoring-operator-tls" not found Jul 07 14:26:59.440750 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:59.440732 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/450aeb2e-26e4-4ebb-8006-eb9c43c6e2d1-service-ca-bundle\") pod \"router-default-7c58ff4df-55gs2\" (UID: \"450aeb2e-26e4-4ebb-8006-eb9c43c6e2d1\") " pod="openshift-ingress/router-default-7c58ff4df-55gs2" Jul 07 14:26:59.440860 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:26:59.440799 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/450aeb2e-26e4-4ebb-8006-eb9c43c6e2d1-metrics-certs\") pod \"router-default-7c58ff4df-55gs2\" (UID: \"450aeb2e-26e4-4ebb-8006-eb9c43c6e2d1\") " pod="openshift-ingress/router-default-7c58ff4df-55gs2" Jul 07 14:26:59.440913 ip-10-0-134-42 kubenswrapper[2557]: E0707 14:26:59.440885 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/450aeb2e-26e4-4ebb-8006-eb9c43c6e2d1-service-ca-bundle podName:450aeb2e-26e4-4ebb-8006-eb9c43c6e2d1 nodeName:}" failed. No retries permitted until 2026-07-07 14:27:00.440869879 +0000 UTC m=+57.150008237 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/450aeb2e-26e4-4ebb-8006-eb9c43c6e2d1-service-ca-bundle") pod "router-default-7c58ff4df-55gs2" (UID: "450aeb2e-26e4-4ebb-8006-eb9c43c6e2d1") : configmap references non-existent config key: service-ca.crt Jul 07 14:26:59.440913 ip-10-0-134-42 kubenswrapper[2557]: E0707 14:26:59.440906 2557 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 07 14:26:59.441017 ip-10-0-134-42 kubenswrapper[2557]: E0707 14:26:59.440973 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/450aeb2e-26e4-4ebb-8006-eb9c43c6e2d1-metrics-certs podName:450aeb2e-26e4-4ebb-8006-eb9c43c6e2d1 nodeName:}" failed. No retries permitted until 2026-07-07 14:27:00.440957229 +0000 UTC m=+57.150095591 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/450aeb2e-26e4-4ebb-8006-eb9c43c6e2d1-metrics-certs") pod "router-default-7c58ff4df-55gs2" (UID: "450aeb2e-26e4-4ebb-8006-eb9c43c6e2d1") : secret "router-metrics-certs-default" not found Jul 07 14:27:00.140815 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:00.140759 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5c847dcfc4-xpdmb" event={"ID":"c7028698-fc37-401b-966d-359bad5c835e","Type":"ContainerStarted","Data":"c45ad87266dd18a1eca5ece1828924e54dbc1fc6b253ecba1736ca73b1e2cdba"} Jul 07 14:27:00.347076 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:00.347041 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/d75c5911-092c-408a-83a7-871dfb379d28-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-bkdv4\" (UID: \"d75c5911-092c-408a-83a7-871dfb379d28\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-bkdv4" Jul 07 14:27:00.347233 ip-10-0-134-42 kubenswrapper[2557]: E0707 14:27:00.347153 2557 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 07 14:27:00.347233 ip-10-0-134-42 kubenswrapper[2557]: E0707 14:27:00.347204 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d75c5911-092c-408a-83a7-871dfb379d28-cluster-monitoring-operator-tls podName:d75c5911-092c-408a-83a7-871dfb379d28 nodeName:}" failed. No retries permitted until 2026-07-07 14:27:02.347191469 +0000 UTC m=+59.056329826 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/d75c5911-092c-408a-83a7-871dfb379d28-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-bkdv4" (UID: "d75c5911-092c-408a-83a7-871dfb379d28") : secret "cluster-monitoring-operator-tls" not found Jul 07 14:27:00.446471 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:00.446386 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:27:00.448258 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:00.448233 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/450aeb2e-26e4-4ebb-8006-eb9c43c6e2d1-metrics-certs\") pod \"router-default-7c58ff4df-55gs2\" (UID: \"450aeb2e-26e4-4ebb-8006-eb9c43c6e2d1\") " pod="openshift-ingress/router-default-7c58ff4df-55gs2" Jul 07 14:27:00.448394 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:00.448345 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/450aeb2e-26e4-4ebb-8006-eb9c43c6e2d1-service-ca-bundle\") pod \"router-default-7c58ff4df-55gs2\" (UID: \"450aeb2e-26e4-4ebb-8006-eb9c43c6e2d1\") " pod="openshift-ingress/router-default-7c58ff4df-55gs2" Jul 07 14:27:00.448394 ip-10-0-134-42 kubenswrapper[2557]: E0707 14:27:00.448376 2557 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 07 14:27:00.448485 ip-10-0-134-42 kubenswrapper[2557]: E0707 14:27:00.448431 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/450aeb2e-26e4-4ebb-8006-eb9c43c6e2d1-metrics-certs podName:450aeb2e-26e4-4ebb-8006-eb9c43c6e2d1 nodeName:}" failed. No retries permitted until 2026-07-07 14:27:02.448415733 +0000 UTC m=+59.157554107 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/450aeb2e-26e4-4ebb-8006-eb9c43c6e2d1-metrics-certs") pod "router-default-7c58ff4df-55gs2" (UID: "450aeb2e-26e4-4ebb-8006-eb9c43c6e2d1") : secret "router-metrics-certs-default" not found Jul 07 14:27:00.448560 ip-10-0-134-42 kubenswrapper[2557]: E0707 14:27:00.448541 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/450aeb2e-26e4-4ebb-8006-eb9c43c6e2d1-service-ca-bundle podName:450aeb2e-26e4-4ebb-8006-eb9c43c6e2d1 nodeName:}" failed. No retries permitted until 2026-07-07 14:27:02.448508207 +0000 UTC m=+59.157646568 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/450aeb2e-26e4-4ebb-8006-eb9c43c6e2d1-service-ca-bundle") pod "router-default-7c58ff4df-55gs2" (UID: "450aeb2e-26e4-4ebb-8006-eb9c43c6e2d1") : configmap references non-existent config key: service-ca.crt Jul 07 14:27:00.556505 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:00.556466 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:27:01.144536 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:01.144497 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5c847dcfc4-xpdmb" event={"ID":"c7028698-fc37-401b-966d-359bad5c835e","Type":"ContainerStarted","Data":"b01f476734c7fb166c440a9fff7d5898cce48b9d380b7424c1626df4669f7e41"} Jul 07 14:27:01.164201 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:01.164159 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-5c847dcfc4-xpdmb" podStartSLOduration=1.374436288 podStartE2EDuration="3.164146254s" totalImagesPullingTime="1.789709966s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 14:26:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:27:01.163280456 +0000 UTC m=+57.872418837" watchObservedRunningTime="2026-07-07 14:27:01.164146254 +0000 UTC m=+57.873284634" Jul 07 14:27:01.391233 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:01.391204 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:27:01.451049 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:01.451025 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:27:01.510567 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:01.510545 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:27:01.893123 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:01.893093 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-q75jf"] Jul 07 14:27:01.895459 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:01.895441 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-q75jf" Jul 07 14:27:01.902365 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:01.902221 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 07 14:27:01.902443 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:01.902434 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 07 14:27:01.903243 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:01.903217 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-fh5qh\"" Jul 07 14:27:01.911781 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:01.911761 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-q75jf"] Jul 07 14:27:01.960193 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:01.960170 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/d65b6030-55a6-4986-9642-a3bbe6be320c-data-volume\") pod \"insights-runtime-extractor-q75jf\" (UID: \"d65b6030-55a6-4986-9642-a3bbe6be320c\") " pod="openshift-insights/insights-runtime-extractor-q75jf" Jul 07 14:27:01.960294 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:01.960216 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/d65b6030-55a6-4986-9642-a3bbe6be320c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-q75jf\" (UID: \"d65b6030-55a6-4986-9642-a3bbe6be320c\") " pod="openshift-insights/insights-runtime-extractor-q75jf" Jul 07 14:27:01.960356 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:01.960324 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d65b6030-55a6-4986-9642-a3bbe6be320c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-q75jf\" (UID: \"d65b6030-55a6-4986-9642-a3bbe6be320c\") " pod="openshift-insights/insights-runtime-extractor-q75jf" Jul 07 14:27:01.960410 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:01.960387 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7ss29\" (UniqueName: \"kubernetes.io/projected/d65b6030-55a6-4986-9642-a3bbe6be320c-kube-api-access-7ss29\") pod \"insights-runtime-extractor-q75jf\" (UID: \"d65b6030-55a6-4986-9642-a3bbe6be320c\") " pod="openshift-insights/insights-runtime-extractor-q75jf" Jul 07 14:27:01.960466 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:01.960421 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/d65b6030-55a6-4986-9642-a3bbe6be320c-crio-socket\") pod \"insights-runtime-extractor-q75jf\" (UID: \"d65b6030-55a6-4986-9642-a3bbe6be320c\") " pod="openshift-insights/insights-runtime-extractor-q75jf" Jul 07 14:27:01.977147 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:01.977114 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:27:02.061481 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:02.061443 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/d65b6030-55a6-4986-9642-a3bbe6be320c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-q75jf\" (UID: \"d65b6030-55a6-4986-9642-a3bbe6be320c\") " pod="openshift-insights/insights-runtime-extractor-q75jf" Jul 07 14:27:02.061609 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:02.061508 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d65b6030-55a6-4986-9642-a3bbe6be320c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-q75jf\" (UID: \"d65b6030-55a6-4986-9642-a3bbe6be320c\") " pod="openshift-insights/insights-runtime-extractor-q75jf" Jul 07 14:27:02.061609 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:02.061569 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7ss29\" (UniqueName: \"kubernetes.io/projected/d65b6030-55a6-4986-9642-a3bbe6be320c-kube-api-access-7ss29\") pod \"insights-runtime-extractor-q75jf\" (UID: \"d65b6030-55a6-4986-9642-a3bbe6be320c\") " pod="openshift-insights/insights-runtime-extractor-q75jf" Jul 07 14:27:02.061609 ip-10-0-134-42 kubenswrapper[2557]: E0707 14:27:02.061580 2557 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 07 14:27:02.061609 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:02.061589 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/d65b6030-55a6-4986-9642-a3bbe6be320c-crio-socket\") pod \"insights-runtime-extractor-q75jf\" (UID: \"d65b6030-55a6-4986-9642-a3bbe6be320c\") " pod="openshift-insights/insights-runtime-extractor-q75jf" Jul 07 14:27:02.061745 ip-10-0-134-42 kubenswrapper[2557]: E0707 14:27:02.061610 2557 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 07 14:27:02.061745 ip-10-0-134-42 kubenswrapper[2557]: E0707 14:27:02.061641 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d65b6030-55a6-4986-9642-a3bbe6be320c-kube-rbac-proxy-cm podName:d65b6030-55a6-4986-9642-a3bbe6be320c nodeName:}" failed. No retries permitted until 2026-07-07 14:27:02.561622964 +0000 UTC m=+59.270761342 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/d65b6030-55a6-4986-9642-a3bbe6be320c-kube-rbac-proxy-cm") pod "insights-runtime-extractor-q75jf" (UID: "d65b6030-55a6-4986-9642-a3bbe6be320c") : configmap "kube-rbac-proxy" not found Jul 07 14:27:02.061745 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:02.061678 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/d65b6030-55a6-4986-9642-a3bbe6be320c-crio-socket\") pod \"insights-runtime-extractor-q75jf\" (UID: \"d65b6030-55a6-4986-9642-a3bbe6be320c\") " pod="openshift-insights/insights-runtime-extractor-q75jf" Jul 07 14:27:02.061745 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:02.061680 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/d65b6030-55a6-4986-9642-a3bbe6be320c-data-volume\") pod \"insights-runtime-extractor-q75jf\" (UID: \"d65b6030-55a6-4986-9642-a3bbe6be320c\") " pod="openshift-insights/insights-runtime-extractor-q75jf" Jul 07 14:27:02.061745 ip-10-0-134-42 kubenswrapper[2557]: E0707 14:27:02.061725 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d65b6030-55a6-4986-9642-a3bbe6be320c-insights-runtime-extractor-tls podName:d65b6030-55a6-4986-9642-a3bbe6be320c nodeName:}" failed. No retries permitted until 2026-07-07 14:27:02.561712267 +0000 UTC m=+59.270850636 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/d65b6030-55a6-4986-9642-a3bbe6be320c-insights-runtime-extractor-tls") pod "insights-runtime-extractor-q75jf" (UID: "d65b6030-55a6-4986-9642-a3bbe6be320c") : secret "insights-runtime-extractor-tls" not found Jul 07 14:27:02.061931 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:02.061917 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/d65b6030-55a6-4986-9642-a3bbe6be320c-data-volume\") pod \"insights-runtime-extractor-q75jf\" (UID: \"d65b6030-55a6-4986-9642-a3bbe6be320c\") " pod="openshift-insights/insights-runtime-extractor-q75jf" Jul 07 14:27:02.069337 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:02.069313 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7ss29\" (UniqueName: \"kubernetes.io/projected/d65b6030-55a6-4986-9642-a3bbe6be320c-kube-api-access-7ss29\") pod \"insights-runtime-extractor-q75jf\" (UID: \"d65b6030-55a6-4986-9642-a3bbe6be320c\") " pod="openshift-insights/insights-runtime-extractor-q75jf" Jul 07 14:27:02.076835 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:02.076811 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:27:02.365063 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:02.365028 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/d75c5911-092c-408a-83a7-871dfb379d28-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-bkdv4\" (UID: \"d75c5911-092c-408a-83a7-871dfb379d28\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-bkdv4" Jul 07 14:27:02.365404 ip-10-0-134-42 kubenswrapper[2557]: E0707 14:27:02.365158 2557 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 07 14:27:02.365404 ip-10-0-134-42 kubenswrapper[2557]: E0707 14:27:02.365212 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d75c5911-092c-408a-83a7-871dfb379d28-cluster-monitoring-operator-tls podName:d75c5911-092c-408a-83a7-871dfb379d28 nodeName:}" failed. No retries permitted until 2026-07-07 14:27:06.365198098 +0000 UTC m=+63.074336457 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/d75c5911-092c-408a-83a7-871dfb379d28-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-bkdv4" (UID: "d75c5911-092c-408a-83a7-871dfb379d28") : secret "cluster-monitoring-operator-tls" not found Jul 07 14:27:02.466358 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:02.466325 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/450aeb2e-26e4-4ebb-8006-eb9c43c6e2d1-service-ca-bundle\") pod \"router-default-7c58ff4df-55gs2\" (UID: \"450aeb2e-26e4-4ebb-8006-eb9c43c6e2d1\") " pod="openshift-ingress/router-default-7c58ff4df-55gs2" Jul 07 14:27:02.466491 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:02.466416 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/450aeb2e-26e4-4ebb-8006-eb9c43c6e2d1-metrics-certs\") pod \"router-default-7c58ff4df-55gs2\" (UID: \"450aeb2e-26e4-4ebb-8006-eb9c43c6e2d1\") " pod="openshift-ingress/router-default-7c58ff4df-55gs2" Jul 07 14:27:02.466576 ip-10-0-134-42 kubenswrapper[2557]: E0707 14:27:02.466546 2557 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 07 14:27:02.466618 ip-10-0-134-42 kubenswrapper[2557]: E0707 14:27:02.466548 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/450aeb2e-26e4-4ebb-8006-eb9c43c6e2d1-service-ca-bundle podName:450aeb2e-26e4-4ebb-8006-eb9c43c6e2d1 nodeName:}" failed. No retries permitted until 2026-07-07 14:27:06.466505035 +0000 UTC m=+63.175643416 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/450aeb2e-26e4-4ebb-8006-eb9c43c6e2d1-service-ca-bundle") pod "router-default-7c58ff4df-55gs2" (UID: "450aeb2e-26e4-4ebb-8006-eb9c43c6e2d1") : configmap references non-existent config key: service-ca.crt Jul 07 14:27:02.466659 ip-10-0-134-42 kubenswrapper[2557]: E0707 14:27:02.466617 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/450aeb2e-26e4-4ebb-8006-eb9c43c6e2d1-metrics-certs podName:450aeb2e-26e4-4ebb-8006-eb9c43c6e2d1 nodeName:}" failed. No retries permitted until 2026-07-07 14:27:06.466590805 +0000 UTC m=+63.175729167 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/450aeb2e-26e4-4ebb-8006-eb9c43c6e2d1-metrics-certs") pod "router-default-7c58ff4df-55gs2" (UID: "450aeb2e-26e4-4ebb-8006-eb9c43c6e2d1") : secret "router-metrics-certs-default" not found Jul 07 14:27:02.567016 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:02.566967 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/d65b6030-55a6-4986-9642-a3bbe6be320c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-q75jf\" (UID: \"d65b6030-55a6-4986-9642-a3bbe6be320c\") " pod="openshift-insights/insights-runtime-extractor-q75jf" Jul 07 14:27:02.567187 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:02.567037 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d65b6030-55a6-4986-9642-a3bbe6be320c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-q75jf\" (UID: \"d65b6030-55a6-4986-9642-a3bbe6be320c\") " pod="openshift-insights/insights-runtime-extractor-q75jf" Jul 07 14:27:02.567187 ip-10-0-134-42 kubenswrapper[2557]: E0707 14:27:02.567081 2557 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 07 14:27:02.567187 ip-10-0-134-42 kubenswrapper[2557]: E0707 14:27:02.567141 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d65b6030-55a6-4986-9642-a3bbe6be320c-kube-rbac-proxy-cm podName:d65b6030-55a6-4986-9642-a3bbe6be320c nodeName:}" failed. No retries permitted until 2026-07-07 14:27:03.567127583 +0000 UTC m=+60.276265942 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/d65b6030-55a6-4986-9642-a3bbe6be320c-kube-rbac-proxy-cm") pod "insights-runtime-extractor-q75jf" (UID: "d65b6030-55a6-4986-9642-a3bbe6be320c") : configmap "kube-rbac-proxy" not found Jul 07 14:27:02.567187 ip-10-0-134-42 kubenswrapper[2557]: E0707 14:27:02.567160 2557 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 07 14:27:02.567338 ip-10-0-134-42 kubenswrapper[2557]: E0707 14:27:02.567208 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d65b6030-55a6-4986-9642-a3bbe6be320c-insights-runtime-extractor-tls podName:d65b6030-55a6-4986-9642-a3bbe6be320c nodeName:}" failed. No retries permitted until 2026-07-07 14:27:03.567196382 +0000 UTC m=+60.276334740 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/d65b6030-55a6-4986-9642-a3bbe6be320c-insights-runtime-extractor-tls") pod "insights-runtime-extractor-q75jf" (UID: "d65b6030-55a6-4986-9642-a3bbe6be320c") : secret "insights-runtime-extractor-tls" not found Jul 07 14:27:03.285863 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:03.285836 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-wmt48_aa141f90-5856-4779-90d7-c6377b1e32d6/dns-node-resolver/0.log" Jul 07 14:27:03.426679 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:03.426649 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:27:03.526488 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:03.526461 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:27:03.576933 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:03.576910 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d65b6030-55a6-4986-9642-a3bbe6be320c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-q75jf\" (UID: \"d65b6030-55a6-4986-9642-a3bbe6be320c\") " pod="openshift-insights/insights-runtime-extractor-q75jf" Jul 07 14:27:03.577023 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:03.577008 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/d65b6030-55a6-4986-9642-a3bbe6be320c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-q75jf\" (UID: \"d65b6030-55a6-4986-9642-a3bbe6be320c\") " pod="openshift-insights/insights-runtime-extractor-q75jf" Jul 07 14:27:03.577100 ip-10-0-134-42 kubenswrapper[2557]: E0707 14:27:03.577082 2557 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 07 14:27:03.577168 ip-10-0-134-42 kubenswrapper[2557]: E0707 14:27:03.577157 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d65b6030-55a6-4986-9642-a3bbe6be320c-insights-runtime-extractor-tls podName:d65b6030-55a6-4986-9642-a3bbe6be320c nodeName:}" failed. No retries permitted until 2026-07-07 14:27:05.577136925 +0000 UTC m=+62.286275299 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/d65b6030-55a6-4986-9642-a3bbe6be320c-insights-runtime-extractor-tls") pod "insights-runtime-extractor-q75jf" (UID: "d65b6030-55a6-4986-9642-a3bbe6be320c") : secret "insights-runtime-extractor-tls" not found Jul 07 14:27:03.577231 ip-10-0-134-42 kubenswrapper[2557]: E0707 14:27:03.577089 2557 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 07 14:27:03.577231 ip-10-0-134-42 kubenswrapper[2557]: E0707 14:27:03.577213 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d65b6030-55a6-4986-9642-a3bbe6be320c-kube-rbac-proxy-cm podName:d65b6030-55a6-4986-9642-a3bbe6be320c nodeName:}" failed. No retries permitted until 2026-07-07 14:27:05.577202599 +0000 UTC m=+62.286340957 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/d65b6030-55a6-4986-9642-a3bbe6be320c-kube-rbac-proxy-cm") pod "insights-runtime-extractor-q75jf" (UID: "d65b6030-55a6-4986-9642-a3bbe6be320c") : configmap "kube-rbac-proxy" not found Jul 07 14:27:03.627051 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:03.627023 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:27:03.885494 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:03.885444 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-5g7bh_0b70b350-feef-419b-91c0-45a22ae7f30b/node-ca/0.log" Jul 07 14:27:04.485829 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:04.485803 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-8dtp8_8c3755dc-b805-4f6a-8c8e-6ec9077a8efb/serve-healthcheck-canary/0.log" Jul 07 14:27:05.075131 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:05.075103 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-scfg4" Jul 07 14:27:05.234423 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:05.234390 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:27:05.325359 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:05.325275 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:27:05.419255 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:05.419219 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:27:05.592641 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:05.592573 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d65b6030-55a6-4986-9642-a3bbe6be320c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-q75jf\" (UID: \"d65b6030-55a6-4986-9642-a3bbe6be320c\") " pod="openshift-insights/insights-runtime-extractor-q75jf" Jul 07 14:27:05.592972 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:05.592698 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/d65b6030-55a6-4986-9642-a3bbe6be320c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-q75jf\" (UID: \"d65b6030-55a6-4986-9642-a3bbe6be320c\") " pod="openshift-insights/insights-runtime-extractor-q75jf" Jul 07 14:27:05.592972 ip-10-0-134-42 kubenswrapper[2557]: E0707 14:27:05.592726 2557 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 07 14:27:05.592972 ip-10-0-134-42 kubenswrapper[2557]: E0707 14:27:05.592774 2557 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 07 14:27:05.592972 ip-10-0-134-42 kubenswrapper[2557]: E0707 14:27:05.592791 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d65b6030-55a6-4986-9642-a3bbe6be320c-insights-runtime-extractor-tls podName:d65b6030-55a6-4986-9642-a3bbe6be320c nodeName:}" failed. No retries permitted until 2026-07-07 14:27:09.592769991 +0000 UTC m=+66.301908350 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/d65b6030-55a6-4986-9642-a3bbe6be320c-insights-runtime-extractor-tls") pod "insights-runtime-extractor-q75jf" (UID: "d65b6030-55a6-4986-9642-a3bbe6be320c") : secret "insights-runtime-extractor-tls" not found Jul 07 14:27:05.592972 ip-10-0-134-42 kubenswrapper[2557]: E0707 14:27:05.592807 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d65b6030-55a6-4986-9642-a3bbe6be320c-kube-rbac-proxy-cm podName:d65b6030-55a6-4986-9642-a3bbe6be320c nodeName:}" failed. No retries permitted until 2026-07-07 14:27:09.592797175 +0000 UTC m=+66.301935545 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/d65b6030-55a6-4986-9642-a3bbe6be320c-kube-rbac-proxy-cm") pod "insights-runtime-extractor-q75jf" (UID: "d65b6030-55a6-4986-9642-a3bbe6be320c") : configmap "kube-rbac-proxy" not found Jul 07 14:27:06.400309 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:06.400265 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/d75c5911-092c-408a-83a7-871dfb379d28-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-bkdv4\" (UID: \"d75c5911-092c-408a-83a7-871dfb379d28\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-bkdv4" Jul 07 14:27:06.400496 ip-10-0-134-42 kubenswrapper[2557]: E0707 14:27:06.400433 2557 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 07 14:27:06.400601 ip-10-0-134-42 kubenswrapper[2557]: E0707 14:27:06.400505 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d75c5911-092c-408a-83a7-871dfb379d28-cluster-monitoring-operator-tls podName:d75c5911-092c-408a-83a7-871dfb379d28 nodeName:}" failed. No retries permitted until 2026-07-07 14:27:14.400484964 +0000 UTC m=+71.109623340 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/d75c5911-092c-408a-83a7-871dfb379d28-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-bkdv4" (UID: "d75c5911-092c-408a-83a7-871dfb379d28") : secret "cluster-monitoring-operator-tls" not found Jul 07 14:27:06.500974 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:06.500937 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/450aeb2e-26e4-4ebb-8006-eb9c43c6e2d1-metrics-certs\") pod \"router-default-7c58ff4df-55gs2\" (UID: \"450aeb2e-26e4-4ebb-8006-eb9c43c6e2d1\") " pod="openshift-ingress/router-default-7c58ff4df-55gs2" Jul 07 14:27:06.501247 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:06.501070 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/450aeb2e-26e4-4ebb-8006-eb9c43c6e2d1-service-ca-bundle\") pod \"router-default-7c58ff4df-55gs2\" (UID: \"450aeb2e-26e4-4ebb-8006-eb9c43c6e2d1\") " pod="openshift-ingress/router-default-7c58ff4df-55gs2" Jul 07 14:27:06.501247 ip-10-0-134-42 kubenswrapper[2557]: E0707 14:27:06.501100 2557 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 07 14:27:06.501247 ip-10-0-134-42 kubenswrapper[2557]: E0707 14:27:06.501161 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/450aeb2e-26e4-4ebb-8006-eb9c43c6e2d1-metrics-certs podName:450aeb2e-26e4-4ebb-8006-eb9c43c6e2d1 nodeName:}" failed. No retries permitted until 2026-07-07 14:27:14.501144047 +0000 UTC m=+71.210282407 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/450aeb2e-26e4-4ebb-8006-eb9c43c6e2d1-metrics-certs") pod "router-default-7c58ff4df-55gs2" (UID: "450aeb2e-26e4-4ebb-8006-eb9c43c6e2d1") : secret "router-metrics-certs-default" not found Jul 07 14:27:06.501412 ip-10-0-134-42 kubenswrapper[2557]: E0707 14:27:06.501245 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/450aeb2e-26e4-4ebb-8006-eb9c43c6e2d1-service-ca-bundle podName:450aeb2e-26e4-4ebb-8006-eb9c43c6e2d1 nodeName:}" failed. No retries permitted until 2026-07-07 14:27:14.501228989 +0000 UTC m=+71.210367347 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/450aeb2e-26e4-4ebb-8006-eb9c43c6e2d1-service-ca-bundle") pod "router-default-7c58ff4df-55gs2" (UID: "450aeb2e-26e4-4ebb-8006-eb9c43c6e2d1") : configmap references non-existent config key: service-ca.crt Jul 07 14:27:08.618005 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:08.617971 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q5g9n\" (UniqueName: \"kubernetes.io/projected/09a0edd9-f556-490d-a691-a0cb3dc3aff9-kube-api-access-q5g9n\") pod \"network-check-target-bqthq\" (UID: \"09a0edd9-f556-490d-a691-a0cb3dc3aff9\") " pod="openshift-network-diagnostics/network-check-target-bqthq" Jul 07 14:27:08.618549 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:08.618040 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/320ab15e-c868-4a37-a3e6-1cea72bb74a4-metrics-certs\") pod \"network-metrics-daemon-s4p8l\" (UID: \"320ab15e-c868-4a37-a3e6-1cea72bb74a4\") " pod="openshift-multus/network-metrics-daemon-s4p8l" Jul 07 14:27:08.618549 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:08.618104 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ce774983-4cbb-4526-ad84-72478efce781-original-pull-secret\") pod \"global-pull-secret-syncer-hvlzx\" (UID: \"ce774983-4cbb-4526-ad84-72478efce781\") " pod="kube-system/global-pull-secret-syncer-hvlzx" Jul 07 14:27:08.620184 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:08.620165 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 07 14:27:08.620514 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:08.620496 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 07 14:27:08.620608 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:08.620590 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 07 14:27:08.628992 ip-10-0-134-42 kubenswrapper[2557]: E0707 14:27:08.628968 2557 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 07 14:27:08.629141 ip-10-0-134-42 kubenswrapper[2557]: E0707 14:27:08.629031 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/320ab15e-c868-4a37-a3e6-1cea72bb74a4-metrics-certs podName:320ab15e-c868-4a37-a3e6-1cea72bb74a4 nodeName:}" failed. No retries permitted until 2026-07-07 14:28:12.629014804 +0000 UTC m=+129.338153176 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/320ab15e-c868-4a37-a3e6-1cea72bb74a4-metrics-certs") pod "network-metrics-daemon-s4p8l" (UID: "320ab15e-c868-4a37-a3e6-1cea72bb74a4") : secret "metrics-daemon-secret" not found Jul 07 14:27:08.630814 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:08.630800 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 07 14:27:08.631135 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:08.631118 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ce774983-4cbb-4526-ad84-72478efce781-original-pull-secret\") pod \"global-pull-secret-syncer-hvlzx\" (UID: \"ce774983-4cbb-4526-ad84-72478efce781\") " pod="kube-system/global-pull-secret-syncer-hvlzx" Jul 07 14:27:08.641825 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:08.641798 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q5g9n\" (UniqueName: \"kubernetes.io/projected/09a0edd9-f556-490d-a691-a0cb3dc3aff9-kube-api-access-q5g9n\") pod \"network-check-target-bqthq\" (UID: \"09a0edd9-f556-490d-a691-a0cb3dc3aff9\") " pod="openshift-network-diagnostics/network-check-target-bqthq" Jul 07 14:27:08.643754 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:08.643732 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-29x7z\"" Jul 07 14:27:08.652422 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:08.652401 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bqthq" Jul 07 14:27:08.652422 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:08.652412 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hvlzx" Jul 07 14:27:08.770185 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:08.770159 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-6mwgs"] Jul 07 14:27:08.772964 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:08.772932 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-6mwgs" Jul 07 14:27:08.773257 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:08.773146 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-68b757fb67-vd9g9"] Jul 07 14:27:08.774966 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:08.774920 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-2mlbw\"" Jul 07 14:27:08.774966 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:08.774948 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\"" Jul 07 14:27:08.775202 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:08.774920 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\"" Jul 07 14:27:08.775202 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:08.774951 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\"" Jul 07 14:27:08.775202 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:08.774955 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\"" Jul 07 14:27:08.775202 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:08.775153 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-68b757fb67-vd9g9" Jul 07 14:27:08.776895 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:08.776873 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\"" Jul 07 14:27:08.777111 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:08.777091 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\"" Jul 07 14:27:08.777203 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:08.777118 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\"" Jul 07 14:27:08.777361 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:08.777343 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\"" Jul 07 14:27:08.777417 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:08.777344 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-dqtz5\"" Jul 07 14:27:08.783804 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:08.783232 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-6mwgs"] Jul 07 14:27:08.786008 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:08.785921 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\"" Jul 07 14:27:08.792322 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:08.792301 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-bqthq"] Jul 07 14:27:08.793602 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:08.793581 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-68b757fb67-vd9g9"] Jul 07 14:27:08.796901 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:27:08.796879 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod09a0edd9_f556_490d_a691_a0cb3dc3aff9.slice/crio-ff548c442610aa105f947e4d5d73aab39019e81bf13aa2e9aab45757cb2e6e01 WatchSource:0}: Error finding container ff548c442610aa105f947e4d5d73aab39019e81bf13aa2e9aab45757cb2e6e01: Status 404 returned error can't find the container with id ff548c442610aa105f947e4d5d73aab39019e81bf13aa2e9aab45757cb2e6e01 Jul 07 14:27:08.806144 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:08.806123 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-hvlzx"] Jul 07 14:27:08.809565 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:27:08.809546 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podce774983_4cbb_4526_ad84_72478efce781.slice/crio-a71bca57b09b0ba03100dbbcc690ed924a289e14af03e6eef115a25b0235c39b WatchSource:0}: Error finding container a71bca57b09b0ba03100dbbcc690ed924a289e14af03e6eef115a25b0235c39b: Status 404 returned error can't find the container with id a71bca57b09b0ba03100dbbcc690ed924a289e14af03e6eef115a25b0235c39b Jul 07 14:27:08.872804 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:08.872724 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-rmmws"] Jul 07 14:27:08.875035 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:08.875020 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-rmmws" Jul 07 14:27:08.876943 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:08.876922 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\"" Jul 07 14:27:08.876943 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:08.876939 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\"" Jul 07 14:27:08.877130 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:08.876925 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\"" Jul 07 14:27:08.877130 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:08.877014 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-ld8s7\"" Jul 07 14:27:08.883316 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:08.883295 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-rmmws"] Jul 07 14:27:08.920889 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:08.920855 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7zjmq\" (UniqueName: \"kubernetes.io/projected/0fa80df4-c933-4758-b75f-3f70b7217c0d-kube-api-access-7zjmq\") pod \"console-operator-68b757fb67-vd9g9\" (UID: \"0fa80df4-c933-4758-b75f-3f70b7217c0d\") " pod="openshift-console-operator/console-operator-68b757fb67-vd9g9" Jul 07 14:27:08.921065 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:08.920899 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vg5qc\" (UniqueName: \"kubernetes.io/projected/cc5feae9-bad0-4740-9bb5-db0bec7c280f-kube-api-access-vg5qc\") pod \"service-ca-operator-5d46d9ffd4-6mwgs\" (UID: \"cc5feae9-bad0-4740-9bb5-db0bec7c280f\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-6mwgs" Jul 07 14:27:08.921065 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:08.920933 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/cc5feae9-bad0-4740-9bb5-db0bec7c280f-config\") pod \"service-ca-operator-5d46d9ffd4-6mwgs\" (UID: \"cc5feae9-bad0-4740-9bb5-db0bec7c280f\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-6mwgs" Jul 07 14:27:08.921065 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:08.920979 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/0fa80df4-c933-4758-b75f-3f70b7217c0d-trusted-ca\") pod \"console-operator-68b757fb67-vd9g9\" (UID: \"0fa80df4-c933-4758-b75f-3f70b7217c0d\") " pod="openshift-console-operator/console-operator-68b757fb67-vd9g9" Jul 07 14:27:08.921167 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:08.921082 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/0fa80df4-c933-4758-b75f-3f70b7217c0d-config\") pod \"console-operator-68b757fb67-vd9g9\" (UID: \"0fa80df4-c933-4758-b75f-3f70b7217c0d\") " pod="openshift-console-operator/console-operator-68b757fb67-vd9g9" Jul 07 14:27:08.921167 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:08.921123 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/0fa80df4-c933-4758-b75f-3f70b7217c0d-serving-cert\") pod \"console-operator-68b757fb67-vd9g9\" (UID: \"0fa80df4-c933-4758-b75f-3f70b7217c0d\") " pod="openshift-console-operator/console-operator-68b757fb67-vd9g9" Jul 07 14:27:08.921167 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:08.921161 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/cc5feae9-bad0-4740-9bb5-db0bec7c280f-serving-cert\") pod \"service-ca-operator-5d46d9ffd4-6mwgs\" (UID: \"cc5feae9-bad0-4740-9bb5-db0bec7c280f\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-6mwgs" Jul 07 14:27:08.972126 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:08.972094 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-kzmk9"] Jul 07 14:27:08.973955 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:08.973940 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-kzmk9" Jul 07 14:27:08.975335 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:08.975315 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-xqdc5"] Jul 07 14:27:08.976072 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:08.976056 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\"" Jul 07 14:27:08.976134 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:08.976058 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\"" Jul 07 14:27:08.976294 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:08.976277 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-8wvhq\"" Jul 07 14:27:08.977183 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:08.977167 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-xqdc5" Jul 07 14:27:08.978756 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:08.978737 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\"" Jul 07 14:27:08.978848 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:08.978740 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\"" Jul 07 14:27:08.978848 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:08.978762 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-fn7tr\"" Jul 07 14:27:08.978848 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:08.978831 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\"" Jul 07 14:27:08.979143 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:08.979124 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\"" Jul 07 14:27:08.982682 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:08.982657 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-kzmk9"] Jul 07 14:27:08.990693 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:08.990671 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-xqdc5"] Jul 07 14:27:09.021506 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:09.021468 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7zjmq\" (UniqueName: \"kubernetes.io/projected/0fa80df4-c933-4758-b75f-3f70b7217c0d-kube-api-access-7zjmq\") pod \"console-operator-68b757fb67-vd9g9\" (UID: \"0fa80df4-c933-4758-b75f-3f70b7217c0d\") " pod="openshift-console-operator/console-operator-68b757fb67-vd9g9" Jul 07 14:27:09.021732 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:09.021513 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vg5qc\" (UniqueName: \"kubernetes.io/projected/cc5feae9-bad0-4740-9bb5-db0bec7c280f-kube-api-access-vg5qc\") pod \"service-ca-operator-5d46d9ffd4-6mwgs\" (UID: \"cc5feae9-bad0-4740-9bb5-db0bec7c280f\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-6mwgs" Jul 07 14:27:09.021732 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:09.021714 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/cc5feae9-bad0-4740-9bb5-db0bec7c280f-config\") pod \"service-ca-operator-5d46d9ffd4-6mwgs\" (UID: \"cc5feae9-bad0-4740-9bb5-db0bec7c280f\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-6mwgs" Jul 07 14:27:09.021859 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:09.021742 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/0fa80df4-c933-4758-b75f-3f70b7217c0d-trusted-ca\") pod \"console-operator-68b757fb67-vd9g9\" (UID: \"0fa80df4-c933-4758-b75f-3f70b7217c0d\") " pod="openshift-console-operator/console-operator-68b757fb67-vd9g9" Jul 07 14:27:09.021859 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:09.021761 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t5zdx\" (UniqueName: \"kubernetes.io/projected/b6d003bd-5d0f-4e35-9bf1-f4eb10d6d1f8-kube-api-access-t5zdx\") pod \"cluster-samples-operator-56fcf86bf-rmmws\" (UID: \"b6d003bd-5d0f-4e35-9bf1-f4eb10d6d1f8\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-rmmws" Jul 07 14:27:09.021932 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:09.021891 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/b6d003bd-5d0f-4e35-9bf1-f4eb10d6d1f8-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-rmmws\" (UID: \"b6d003bd-5d0f-4e35-9bf1-f4eb10d6d1f8\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-rmmws" Jul 07 14:27:09.021971 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:09.021937 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/0fa80df4-c933-4758-b75f-3f70b7217c0d-config\") pod \"console-operator-68b757fb67-vd9g9\" (UID: \"0fa80df4-c933-4758-b75f-3f70b7217c0d\") " pod="openshift-console-operator/console-operator-68b757fb67-vd9g9" Jul 07 14:27:09.022021 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:09.021970 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/0fa80df4-c933-4758-b75f-3f70b7217c0d-serving-cert\") pod \"console-operator-68b757fb67-vd9g9\" (UID: \"0fa80df4-c933-4758-b75f-3f70b7217c0d\") " pod="openshift-console-operator/console-operator-68b757fb67-vd9g9" Jul 07 14:27:09.022072 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:09.022020 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/cc5feae9-bad0-4740-9bb5-db0bec7c280f-serving-cert\") pod \"service-ca-operator-5d46d9ffd4-6mwgs\" (UID: \"cc5feae9-bad0-4740-9bb5-db0bec7c280f\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-6mwgs" Jul 07 14:27:09.022334 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:09.022314 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/cc5feae9-bad0-4740-9bb5-db0bec7c280f-config\") pod \"service-ca-operator-5d46d9ffd4-6mwgs\" (UID: \"cc5feae9-bad0-4740-9bb5-db0bec7c280f\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-6mwgs" Jul 07 14:27:09.022749 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:09.022730 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/0fa80df4-c933-4758-b75f-3f70b7217c0d-trusted-ca\") pod \"console-operator-68b757fb67-vd9g9\" (UID: \"0fa80df4-c933-4758-b75f-3f70b7217c0d\") " pod="openshift-console-operator/console-operator-68b757fb67-vd9g9" Jul 07 14:27:09.023109 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:09.023088 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/0fa80df4-c933-4758-b75f-3f70b7217c0d-config\") pod \"console-operator-68b757fb67-vd9g9\" (UID: \"0fa80df4-c933-4758-b75f-3f70b7217c0d\") " pod="openshift-console-operator/console-operator-68b757fb67-vd9g9" Jul 07 14:27:09.024142 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:09.024113 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/cc5feae9-bad0-4740-9bb5-db0bec7c280f-serving-cert\") pod \"service-ca-operator-5d46d9ffd4-6mwgs\" (UID: \"cc5feae9-bad0-4740-9bb5-db0bec7c280f\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-6mwgs" Jul 07 14:27:09.024249 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:09.024234 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/0fa80df4-c933-4758-b75f-3f70b7217c0d-serving-cert\") pod \"console-operator-68b757fb67-vd9g9\" (UID: \"0fa80df4-c933-4758-b75f-3f70b7217c0d\") " pod="openshift-console-operator/console-operator-68b757fb67-vd9g9" Jul 07 14:27:09.028494 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:09.028475 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vg5qc\" (UniqueName: \"kubernetes.io/projected/cc5feae9-bad0-4740-9bb5-db0bec7c280f-kube-api-access-vg5qc\") pod \"service-ca-operator-5d46d9ffd4-6mwgs\" (UID: \"cc5feae9-bad0-4740-9bb5-db0bec7c280f\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-6mwgs" Jul 07 14:27:09.028760 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:09.028739 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7zjmq\" (UniqueName: \"kubernetes.io/projected/0fa80df4-c933-4758-b75f-3f70b7217c0d-kube-api-access-7zjmq\") pod \"console-operator-68b757fb67-vd9g9\" (UID: \"0fa80df4-c933-4758-b75f-3f70b7217c0d\") " pod="openshift-console-operator/console-operator-68b757fb67-vd9g9" Jul 07 14:27:09.085747 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:09.085714 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-6mwgs" Jul 07 14:27:09.093472 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:09.093436 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-68b757fb67-vd9g9" Jul 07 14:27:09.123471 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:09.123326 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/b6d003bd-5d0f-4e35-9bf1-f4eb10d6d1f8-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-rmmws\" (UID: \"b6d003bd-5d0f-4e35-9bf1-f4eb10d6d1f8\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-rmmws" Jul 07 14:27:09.123471 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:09.123369 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/71300bd3-684d-46ed-b639-fcbc757ec917-serving-cert\") pod \"kube-storage-version-migrator-operator-846bf668f7-xqdc5\" (UID: \"71300bd3-684d-46ed-b639-fcbc757ec917\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-xqdc5" Jul 07 14:27:09.123471 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:09.123394 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dgqwd\" (UniqueName: \"kubernetes.io/projected/71300bd3-684d-46ed-b639-fcbc757ec917-kube-api-access-dgqwd\") pod \"kube-storage-version-migrator-operator-846bf668f7-xqdc5\" (UID: \"71300bd3-684d-46ed-b639-fcbc757ec917\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-xqdc5" Jul 07 14:27:09.123741 ip-10-0-134-42 kubenswrapper[2557]: E0707 14:27:09.123488 2557 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 07 14:27:09.123741 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:09.123541 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/71300bd3-684d-46ed-b639-fcbc757ec917-config\") pod \"kube-storage-version-migrator-operator-846bf668f7-xqdc5\" (UID: \"71300bd3-684d-46ed-b639-fcbc757ec917\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-xqdc5" Jul 07 14:27:09.123741 ip-10-0-134-42 kubenswrapper[2557]: E0707 14:27:09.123570 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b6d003bd-5d0f-4e35-9bf1-f4eb10d6d1f8-samples-operator-tls podName:b6d003bd-5d0f-4e35-9bf1-f4eb10d6d1f8 nodeName:}" failed. No retries permitted until 2026-07-07 14:27:09.623554884 +0000 UTC m=+66.332693242 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/b6d003bd-5d0f-4e35-9bf1-f4eb10d6d1f8-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-rmmws" (UID: "b6d003bd-5d0f-4e35-9bf1-f4eb10d6d1f8") : secret "samples-operator-tls" not found Jul 07 14:27:09.123741 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:09.123709 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9cv8g\" (UniqueName: \"kubernetes.io/projected/b77d08a8-a421-4412-9879-b094ef9bb36e-kube-api-access-9cv8g\") pod \"volume-data-source-validator-5d456fdbcd-kzmk9\" (UID: \"b77d08a8-a421-4412-9879-b094ef9bb36e\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-kzmk9" Jul 07 14:27:09.123902 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:09.123811 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t5zdx\" (UniqueName: \"kubernetes.io/projected/b6d003bd-5d0f-4e35-9bf1-f4eb10d6d1f8-kube-api-access-t5zdx\") pod \"cluster-samples-operator-56fcf86bf-rmmws\" (UID: \"b6d003bd-5d0f-4e35-9bf1-f4eb10d6d1f8\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-rmmws" Jul 07 14:27:09.134804 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:09.134764 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t5zdx\" (UniqueName: \"kubernetes.io/projected/b6d003bd-5d0f-4e35-9bf1-f4eb10d6d1f8-kube-api-access-t5zdx\") pod \"cluster-samples-operator-56fcf86bf-rmmws\" (UID: \"b6d003bd-5d0f-4e35-9bf1-f4eb10d6d1f8\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-rmmws" Jul 07 14:27:09.165233 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:09.165189 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-bqthq" event={"ID":"09a0edd9-f556-490d-a691-a0cb3dc3aff9","Type":"ContainerStarted","Data":"ff548c442610aa105f947e4d5d73aab39019e81bf13aa2e9aab45757cb2e6e01"} Jul 07 14:27:09.166563 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:09.166534 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-hvlzx" event={"ID":"ce774983-4cbb-4526-ad84-72478efce781","Type":"ContainerStarted","Data":"a71bca57b09b0ba03100dbbcc690ed924a289e14af03e6eef115a25b0235c39b"} Jul 07 14:27:09.217654 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:09.217623 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-6mwgs"] Jul 07 14:27:09.220779 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:27:09.220747 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcc5feae9_bad0_4740_9bb5_db0bec7c280f.slice/crio-c1f52e41c588024b97a0fe6a8a47ed3f1c19f059eb53790a3d888c2e0e8a3660 WatchSource:0}: Error finding container c1f52e41c588024b97a0fe6a8a47ed3f1c19f059eb53790a3d888c2e0e8a3660: Status 404 returned error can't find the container with id c1f52e41c588024b97a0fe6a8a47ed3f1c19f059eb53790a3d888c2e0e8a3660 Jul 07 14:27:09.224833 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:09.224803 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/71300bd3-684d-46ed-b639-fcbc757ec917-config\") pod \"kube-storage-version-migrator-operator-846bf668f7-xqdc5\" (UID: \"71300bd3-684d-46ed-b639-fcbc757ec917\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-xqdc5" Jul 07 14:27:09.224931 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:09.224898 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9cv8g\" (UniqueName: \"kubernetes.io/projected/b77d08a8-a421-4412-9879-b094ef9bb36e-kube-api-access-9cv8g\") pod \"volume-data-source-validator-5d456fdbcd-kzmk9\" (UID: \"b77d08a8-a421-4412-9879-b094ef9bb36e\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-kzmk9" Jul 07 14:27:09.225075 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:09.225057 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/71300bd3-684d-46ed-b639-fcbc757ec917-serving-cert\") pod \"kube-storage-version-migrator-operator-846bf668f7-xqdc5\" (UID: \"71300bd3-684d-46ed-b639-fcbc757ec917\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-xqdc5" Jul 07 14:27:09.225120 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:09.225092 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dgqwd\" (UniqueName: \"kubernetes.io/projected/71300bd3-684d-46ed-b639-fcbc757ec917-kube-api-access-dgqwd\") pod \"kube-storage-version-migrator-operator-846bf668f7-xqdc5\" (UID: \"71300bd3-684d-46ed-b639-fcbc757ec917\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-xqdc5" Jul 07 14:27:09.225426 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:09.225395 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/71300bd3-684d-46ed-b639-fcbc757ec917-config\") pod \"kube-storage-version-migrator-operator-846bf668f7-xqdc5\" (UID: \"71300bd3-684d-46ed-b639-fcbc757ec917\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-xqdc5" Jul 07 14:27:09.227333 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:09.227311 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/71300bd3-684d-46ed-b639-fcbc757ec917-serving-cert\") pod \"kube-storage-version-migrator-operator-846bf668f7-xqdc5\" (UID: \"71300bd3-684d-46ed-b639-fcbc757ec917\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-xqdc5" Jul 07 14:27:09.231638 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:09.231609 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-68b757fb67-vd9g9"] Jul 07 14:27:09.234813 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:27:09.234778 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0fa80df4_c933_4758_b75f_3f70b7217c0d.slice/crio-3c412e40da7d86c83c8488fd501bd3485a9eefc1b6d507548a0629947b7bab8e WatchSource:0}: Error finding container 3c412e40da7d86c83c8488fd501bd3485a9eefc1b6d507548a0629947b7bab8e: Status 404 returned error can't find the container with id 3c412e40da7d86c83c8488fd501bd3485a9eefc1b6d507548a0629947b7bab8e Jul 07 14:27:09.235233 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:09.235200 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dgqwd\" (UniqueName: \"kubernetes.io/projected/71300bd3-684d-46ed-b639-fcbc757ec917-kube-api-access-dgqwd\") pod \"kube-storage-version-migrator-operator-846bf668f7-xqdc5\" (UID: \"71300bd3-684d-46ed-b639-fcbc757ec917\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-xqdc5" Jul 07 14:27:09.235703 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:09.235685 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9cv8g\" (UniqueName: \"kubernetes.io/projected/b77d08a8-a421-4412-9879-b094ef9bb36e-kube-api-access-9cv8g\") pod \"volume-data-source-validator-5d456fdbcd-kzmk9\" (UID: \"b77d08a8-a421-4412-9879-b094ef9bb36e\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-kzmk9" Jul 07 14:27:09.284244 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:09.284206 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-kzmk9" Jul 07 14:27:09.291022 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:09.290991 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-xqdc5" Jul 07 14:27:09.434842 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:09.434752 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-xqdc5"] Jul 07 14:27:09.438982 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:27:09.438950 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod71300bd3_684d_46ed_b639_fcbc757ec917.slice/crio-8572182c4f30eca8a9b5b80ee0ec3e7f7c429bbe54ef25ff6f7b7cc9789945f0 WatchSource:0}: Error finding container 8572182c4f30eca8a9b5b80ee0ec3e7f7c429bbe54ef25ff6f7b7cc9789945f0: Status 404 returned error can't find the container with id 8572182c4f30eca8a9b5b80ee0ec3e7f7c429bbe54ef25ff6f7b7cc9789945f0 Jul 07 14:27:09.452047 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:09.452017 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-kzmk9"] Jul 07 14:27:09.455363 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:27:09.455333 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb77d08a8_a421_4412_9879_b094ef9bb36e.slice/crio-c34751dffb79d5a829e0390ea1e2b196e3989bcab6f484d1e8e58f2e20f555d6 WatchSource:0}: Error finding container c34751dffb79d5a829e0390ea1e2b196e3989bcab6f484d1e8e58f2e20f555d6: Status 404 returned error can't find the container with id c34751dffb79d5a829e0390ea1e2b196e3989bcab6f484d1e8e58f2e20f555d6 Jul 07 14:27:09.605997 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:09.605958 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:27:09.629198 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:09.629115 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/d65b6030-55a6-4986-9642-a3bbe6be320c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-q75jf\" (UID: \"d65b6030-55a6-4986-9642-a3bbe6be320c\") " pod="openshift-insights/insights-runtime-extractor-q75jf" Jul 07 14:27:09.629669 ip-10-0-134-42 kubenswrapper[2557]: E0707 14:27:09.629246 2557 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 07 14:27:09.629669 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:09.629265 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d65b6030-55a6-4986-9642-a3bbe6be320c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-q75jf\" (UID: \"d65b6030-55a6-4986-9642-a3bbe6be320c\") " pod="openshift-insights/insights-runtime-extractor-q75jf" Jul 07 14:27:09.629669 ip-10-0-134-42 kubenswrapper[2557]: E0707 14:27:09.629320 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d65b6030-55a6-4986-9642-a3bbe6be320c-kube-rbac-proxy-cm podName:d65b6030-55a6-4986-9642-a3bbe6be320c nodeName:}" failed. No retries permitted until 2026-07-07 14:27:17.629299123 +0000 UTC m=+74.338437482 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/d65b6030-55a6-4986-9642-a3bbe6be320c-kube-rbac-proxy-cm") pod "insights-runtime-extractor-q75jf" (UID: "d65b6030-55a6-4986-9642-a3bbe6be320c") : configmap "kube-rbac-proxy" not found Jul 07 14:27:09.629669 ip-10-0-134-42 kubenswrapper[2557]: E0707 14:27:09.629371 2557 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 07 14:27:09.629669 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:09.629385 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/b6d003bd-5d0f-4e35-9bf1-f4eb10d6d1f8-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-rmmws\" (UID: \"b6d003bd-5d0f-4e35-9bf1-f4eb10d6d1f8\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-rmmws" Jul 07 14:27:09.629669 ip-10-0-134-42 kubenswrapper[2557]: E0707 14:27:09.629419 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d65b6030-55a6-4986-9642-a3bbe6be320c-insights-runtime-extractor-tls podName:d65b6030-55a6-4986-9642-a3bbe6be320c nodeName:}" failed. No retries permitted until 2026-07-07 14:27:17.629404899 +0000 UTC m=+74.338543258 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/d65b6030-55a6-4986-9642-a3bbe6be320c-insights-runtime-extractor-tls") pod "insights-runtime-extractor-q75jf" (UID: "d65b6030-55a6-4986-9642-a3bbe6be320c") : secret "insights-runtime-extractor-tls" not found Jul 07 14:27:09.629669 ip-10-0-134-42 kubenswrapper[2557]: E0707 14:27:09.629466 2557 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 07 14:27:09.629669 ip-10-0-134-42 kubenswrapper[2557]: E0707 14:27:09.629510 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b6d003bd-5d0f-4e35-9bf1-f4eb10d6d1f8-samples-operator-tls podName:b6d003bd-5d0f-4e35-9bf1-f4eb10d6d1f8 nodeName:}" failed. No retries permitted until 2026-07-07 14:27:10.62949663 +0000 UTC m=+67.338634993 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/b6d003bd-5d0f-4e35-9bf1-f4eb10d6d1f8-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-rmmws" (UID: "b6d003bd-5d0f-4e35-9bf1-f4eb10d6d1f8") : secret "samples-operator-tls" not found Jul 07 14:27:10.173353 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:10.173226 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-kzmk9" event={"ID":"b77d08a8-a421-4412-9879-b094ef9bb36e","Type":"ContainerStarted","Data":"c34751dffb79d5a829e0390ea1e2b196e3989bcab6f484d1e8e58f2e20f555d6"} Jul 07 14:27:10.175250 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:10.175169 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-vd9g9" event={"ID":"0fa80df4-c933-4758-b75f-3f70b7217c0d","Type":"ContainerStarted","Data":"3c412e40da7d86c83c8488fd501bd3485a9eefc1b6d507548a0629947b7bab8e"} Jul 07 14:27:10.177493 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:10.177442 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-xqdc5" event={"ID":"71300bd3-684d-46ed-b639-fcbc757ec917","Type":"ContainerStarted","Data":"8572182c4f30eca8a9b5b80ee0ec3e7f7c429bbe54ef25ff6f7b7cc9789945f0"} Jul 07 14:27:10.180179 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:10.180130 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-6mwgs" event={"ID":"cc5feae9-bad0-4740-9bb5-db0bec7c280f","Type":"ContainerStarted","Data":"c1f52e41c588024b97a0fe6a8a47ed3f1c19f059eb53790a3d888c2e0e8a3660"} Jul 07 14:27:10.339680 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:10.339631 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/648e2b2f-08a5-4a83-b9aa-319ab2527ae4-registry-tls\") pod \"image-registry-687956bbfb-m9bjn\" (UID: \"648e2b2f-08a5-4a83-b9aa-319ab2527ae4\") " pod="openshift-image-registry/image-registry-687956bbfb-m9bjn" Jul 07 14:27:10.339872 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:10.339764 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3d597bc9-74bf-4809-a33d-0a28dec34cb1-metrics-tls\") pod \"dns-default-5ngmq\" (UID: \"3d597bc9-74bf-4809-a33d-0a28dec34cb1\") " pod="openshift-dns/dns-default-5ngmq" Jul 07 14:27:10.339939 ip-10-0-134-42 kubenswrapper[2557]: E0707 14:27:10.339896 2557 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 07 14:27:10.339995 ip-10-0-134-42 kubenswrapper[2557]: E0707 14:27:10.339958 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3d597bc9-74bf-4809-a33d-0a28dec34cb1-metrics-tls podName:3d597bc9-74bf-4809-a33d-0a28dec34cb1 nodeName:}" failed. No retries permitted until 2026-07-07 14:27:42.339940497 +0000 UTC m=+99.049078858 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/3d597bc9-74bf-4809-a33d-0a28dec34cb1-metrics-tls") pod "dns-default-5ngmq" (UID: "3d597bc9-74bf-4809-a33d-0a28dec34cb1") : secret "dns-default-metrics-tls" not found Jul 07 14:27:10.340316 ip-10-0-134-42 kubenswrapper[2557]: E0707 14:27:10.340171 2557 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 07 14:27:10.340316 ip-10-0-134-42 kubenswrapper[2557]: E0707 14:27:10.340194 2557 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-687956bbfb-m9bjn: secret "image-registry-tls" not found Jul 07 14:27:10.340316 ip-10-0-134-42 kubenswrapper[2557]: E0707 14:27:10.340255 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/648e2b2f-08a5-4a83-b9aa-319ab2527ae4-registry-tls podName:648e2b2f-08a5-4a83-b9aa-319ab2527ae4 nodeName:}" failed. No retries permitted until 2026-07-07 14:27:42.340237994 +0000 UTC m=+99.049376355 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/648e2b2f-08a5-4a83-b9aa-319ab2527ae4-registry-tls") pod "image-registry-687956bbfb-m9bjn" (UID: "648e2b2f-08a5-4a83-b9aa-319ab2527ae4") : secret "image-registry-tls" not found Jul 07 14:27:10.643697 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:10.643464 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/b6d003bd-5d0f-4e35-9bf1-f4eb10d6d1f8-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-rmmws\" (UID: \"b6d003bd-5d0f-4e35-9bf1-f4eb10d6d1f8\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-rmmws" Jul 07 14:27:10.643697 ip-10-0-134-42 kubenswrapper[2557]: E0707 14:27:10.643649 2557 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 07 14:27:10.644178 ip-10-0-134-42 kubenswrapper[2557]: E0707 14:27:10.643729 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b6d003bd-5d0f-4e35-9bf1-f4eb10d6d1f8-samples-operator-tls podName:b6d003bd-5d0f-4e35-9bf1-f4eb10d6d1f8 nodeName:}" failed. No retries permitted until 2026-07-07 14:27:12.643707287 +0000 UTC m=+69.352845646 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/b6d003bd-5d0f-4e35-9bf1-f4eb10d6d1f8-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-rmmws" (UID: "b6d003bd-5d0f-4e35-9bf1-f4eb10d6d1f8") : secret "samples-operator-tls" not found Jul 07 14:27:12.187318 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:12.187281 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-kzmk9" event={"ID":"b77d08a8-a421-4412-9879-b094ef9bb36e","Type":"ContainerStarted","Data":"f50d859a43e11d5c1f397536947432963b5d52942a60e0b761711b5cff8344d6"} Jul 07 14:27:12.202286 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:12.202197 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-kzmk9" podStartSLOduration=2.419574561 podStartE2EDuration="4.202179251s" totalImagesPullingTime="1.78260469s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 14:27:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:27:12.201499533 +0000 UTC m=+68.910637913" watchObservedRunningTime="2026-07-07 14:27:12.202179251 +0000 UTC m=+68.911317633" Jul 07 14:27:12.290140 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:12.290102 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:27:12.381426 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:12.381382 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:27:12.665374 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:12.665334 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/b6d003bd-5d0f-4e35-9bf1-f4eb10d6d1f8-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-rmmws\" (UID: \"b6d003bd-5d0f-4e35-9bf1-f4eb10d6d1f8\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-rmmws" Jul 07 14:27:12.665589 ip-10-0-134-42 kubenswrapper[2557]: E0707 14:27:12.665498 2557 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 07 14:27:12.665589 ip-10-0-134-42 kubenswrapper[2557]: E0707 14:27:12.665586 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b6d003bd-5d0f-4e35-9bf1-f4eb10d6d1f8-samples-operator-tls podName:b6d003bd-5d0f-4e35-9bf1-f4eb10d6d1f8 nodeName:}" failed. No retries permitted until 2026-07-07 14:27:16.66556495 +0000 UTC m=+73.374703326 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/b6d003bd-5d0f-4e35-9bf1-f4eb10d6d1f8-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-rmmws" (UID: "b6d003bd-5d0f-4e35-9bf1-f4eb10d6d1f8") : secret "samples-operator-tls" not found Jul 07 14:27:14.481430 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:14.481386 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/d75c5911-092c-408a-83a7-871dfb379d28-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-bkdv4\" (UID: \"d75c5911-092c-408a-83a7-871dfb379d28\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-bkdv4" Jul 07 14:27:14.481874 ip-10-0-134-42 kubenswrapper[2557]: E0707 14:27:14.481574 2557 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 07 14:27:14.481874 ip-10-0-134-42 kubenswrapper[2557]: E0707 14:27:14.481673 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d75c5911-092c-408a-83a7-871dfb379d28-cluster-monitoring-operator-tls podName:d75c5911-092c-408a-83a7-871dfb379d28 nodeName:}" failed. No retries permitted until 2026-07-07 14:27:30.481652923 +0000 UTC m=+87.190791286 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/d75c5911-092c-408a-83a7-871dfb379d28-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-bkdv4" (UID: "d75c5911-092c-408a-83a7-871dfb379d28") : secret "cluster-monitoring-operator-tls" not found Jul 07 14:27:14.582256 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:14.582212 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/450aeb2e-26e4-4ebb-8006-eb9c43c6e2d1-service-ca-bundle\") pod \"router-default-7c58ff4df-55gs2\" (UID: \"450aeb2e-26e4-4ebb-8006-eb9c43c6e2d1\") " pod="openshift-ingress/router-default-7c58ff4df-55gs2" Jul 07 14:27:14.582431 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:14.582318 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/450aeb2e-26e4-4ebb-8006-eb9c43c6e2d1-metrics-certs\") pod \"router-default-7c58ff4df-55gs2\" (UID: \"450aeb2e-26e4-4ebb-8006-eb9c43c6e2d1\") " pod="openshift-ingress/router-default-7c58ff4df-55gs2" Jul 07 14:27:14.582431 ip-10-0-134-42 kubenswrapper[2557]: E0707 14:27:14.582404 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/450aeb2e-26e4-4ebb-8006-eb9c43c6e2d1-service-ca-bundle podName:450aeb2e-26e4-4ebb-8006-eb9c43c6e2d1 nodeName:}" failed. No retries permitted until 2026-07-07 14:27:30.582381329 +0000 UTC m=+87.291519689 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/450aeb2e-26e4-4ebb-8006-eb9c43c6e2d1-service-ca-bundle") pod "router-default-7c58ff4df-55gs2" (UID: "450aeb2e-26e4-4ebb-8006-eb9c43c6e2d1") : configmap references non-existent config key: service-ca.crt Jul 07 14:27:14.582563 ip-10-0-134-42 kubenswrapper[2557]: E0707 14:27:14.582462 2557 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 07 14:27:14.582563 ip-10-0-134-42 kubenswrapper[2557]: E0707 14:27:14.582530 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/450aeb2e-26e4-4ebb-8006-eb9c43c6e2d1-metrics-certs podName:450aeb2e-26e4-4ebb-8006-eb9c43c6e2d1 nodeName:}" failed. No retries permitted until 2026-07-07 14:27:30.582505734 +0000 UTC m=+87.291644101 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/450aeb2e-26e4-4ebb-8006-eb9c43c6e2d1-metrics-certs") pod "router-default-7c58ff4df-55gs2" (UID: "450aeb2e-26e4-4ebb-8006-eb9c43c6e2d1") : secret "router-metrics-certs-default" not found Jul 07 14:27:15.096348 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:15.096309 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:27:16.199653 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:16.199617 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-bqthq" event={"ID":"09a0edd9-f556-490d-a691-a0cb3dc3aff9","Type":"ContainerStarted","Data":"ef78452bd4cf5f200aa3efd817d16019e97ef4ab210bd280eb41fec8a62219ff"} Jul 07 14:27:16.200085 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:16.199721 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-bqthq" Jul 07 14:27:16.201321 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:16.201302 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-vd9g9_0fa80df4-c933-4758-b75f-3f70b7217c0d/console-operator/0.log" Jul 07 14:27:16.201414 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:16.201334 2557 generic.go:356] "Generic (PLEG): container finished" podID="0fa80df4-c933-4758-b75f-3f70b7217c0d" containerID="92c8506c0d1493744f61561254951f127346d49d59f83ea902e6c2c69e4dbdee" exitCode=255 Jul 07 14:27:16.201414 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:16.201404 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-vd9g9" event={"ID":"0fa80df4-c933-4758-b75f-3f70b7217c0d","Type":"ContainerDied","Data":"92c8506c0d1493744f61561254951f127346d49d59f83ea902e6c2c69e4dbdee"} Jul 07 14:27:16.201623 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:16.201611 2557 scope.go:117] "RemoveContainer" containerID="92c8506c0d1493744f61561254951f127346d49d59f83ea902e6c2c69e4dbdee" Jul 07 14:27:16.202885 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:16.202861 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-hvlzx" event={"ID":"ce774983-4cbb-4526-ad84-72478efce781","Type":"ContainerStarted","Data":"7ec191bc0c6281d9c074695c21c49e1e84dfcf5809a66163b33d46e0db9ed2eb"} Jul 07 14:27:16.204472 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:16.204448 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-xqdc5" event={"ID":"71300bd3-684d-46ed-b639-fcbc757ec917","Type":"ContainerStarted","Data":"9862f1deadb52d7d9b217cd19a657d4b7bdc6a08004c46ee07e964278e3b9c46"} Jul 07 14:27:16.205835 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:16.205818 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-6mwgs" event={"ID":"cc5feae9-bad0-4740-9bb5-db0bec7c280f","Type":"ContainerStarted","Data":"aee205755a35946847a34a95ccc0016c91662b053d4d4f89f5c01e56e59f8f6d"} Jul 07 14:27:16.220262 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:16.220222 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-bqthq" podStartSLOduration=64.393442583 podStartE2EDuration="1m11.220209767s" totalImagesPullingTime="6.826767184s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 14:26:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:27:16.219584785 +0000 UTC m=+72.928723167" watchObservedRunningTime="2026-07-07 14:27:16.220209767 +0000 UTC m=+72.929348193" Jul 07 14:27:16.236792 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:16.236745 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-xqdc5" podStartSLOduration=2.055155568 podStartE2EDuration="8.236726677s" totalImagesPullingTime="6.181571109s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 14:27:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:27:16.235695692 +0000 UTC m=+72.944834072" watchObservedRunningTime="2026-07-07 14:27:16.236726677 +0000 UTC m=+72.945865041" Jul 07 14:27:16.252176 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:16.251925 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-6mwgs" podStartSLOduration=1.85122329 podStartE2EDuration="8.251905723s" totalImagesPullingTime="6.400682433s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 14:27:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:27:16.250791809 +0000 UTC m=+72.959930189" watchObservedRunningTime="2026-07-07 14:27:16.251905723 +0000 UTC m=+72.961044110" Jul 07 14:27:16.286632 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:16.286573 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-hvlzx" podStartSLOduration=64.471348459 podStartE2EDuration="1m11.286553245s" totalImagesPullingTime="6.815204786s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 14:26:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:27:16.284936781 +0000 UTC m=+72.994075163" watchObservedRunningTime="2026-07-07 14:27:16.286553245 +0000 UTC m=+72.995691626" Jul 07 14:27:16.705894 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:16.705861 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/b6d003bd-5d0f-4e35-9bf1-f4eb10d6d1f8-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-rmmws\" (UID: \"b6d003bd-5d0f-4e35-9bf1-f4eb10d6d1f8\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-rmmws" Jul 07 14:27:16.706058 ip-10-0-134-42 kubenswrapper[2557]: E0707 14:27:16.705995 2557 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 07 14:27:16.706106 ip-10-0-134-42 kubenswrapper[2557]: E0707 14:27:16.706073 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b6d003bd-5d0f-4e35-9bf1-f4eb10d6d1f8-samples-operator-tls podName:b6d003bd-5d0f-4e35-9bf1-f4eb10d6d1f8 nodeName:}" failed. No retries permitted until 2026-07-07 14:27:24.706053025 +0000 UTC m=+81.415191399 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/b6d003bd-5d0f-4e35-9bf1-f4eb10d6d1f8-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-rmmws" (UID: "b6d003bd-5d0f-4e35-9bf1-f4eb10d6d1f8") : secret "samples-operator-tls" not found Jul 07 14:27:16.747942 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:16.747908 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:27:16.817090 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:16.817056 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:27:16.917858 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:16.917769 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:27:17.210662 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:17.210594 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-vd9g9_0fa80df4-c933-4758-b75f-3f70b7217c0d/console-operator/1.log" Jul 07 14:27:17.211049 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:17.210967 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-vd9g9_0fa80df4-c933-4758-b75f-3f70b7217c0d/console-operator/0.log" Jul 07 14:27:17.211049 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:17.210997 2557 generic.go:356] "Generic (PLEG): container finished" podID="0fa80df4-c933-4758-b75f-3f70b7217c0d" containerID="e3430facd46f3ca9a9ab9a90884875e6e9a19b293c97f8b1ce0e9f83e1fe2d77" exitCode=255 Jul 07 14:27:17.211049 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:17.211029 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-vd9g9" event={"ID":"0fa80df4-c933-4758-b75f-3f70b7217c0d","Type":"ContainerDied","Data":"e3430facd46f3ca9a9ab9a90884875e6e9a19b293c97f8b1ce0e9f83e1fe2d77"} Jul 07 14:27:17.211203 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:17.211125 2557 scope.go:117] "RemoveContainer" containerID="92c8506c0d1493744f61561254951f127346d49d59f83ea902e6c2c69e4dbdee" Jul 07 14:27:17.211335 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:17.211315 2557 scope.go:117] "RemoveContainer" containerID="e3430facd46f3ca9a9ab9a90884875e6e9a19b293c97f8b1ce0e9f83e1fe2d77" Jul 07 14:27:17.211577 ip-10-0-134-42 kubenswrapper[2557]: E0707 14:27:17.211555 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-68b757fb67-vd9g9_openshift-console-operator(0fa80df4-c933-4758-b75f-3f70b7217c0d)\"" pod="openshift-console-operator/console-operator-68b757fb67-vd9g9" podUID="0fa80df4-c933-4758-b75f-3f70b7217c0d" Jul 07 14:27:17.715196 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:17.715157 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/d65b6030-55a6-4986-9642-a3bbe6be320c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-q75jf\" (UID: \"d65b6030-55a6-4986-9642-a3bbe6be320c\") " pod="openshift-insights/insights-runtime-extractor-q75jf" Jul 07 14:27:17.715357 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:17.715221 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d65b6030-55a6-4986-9642-a3bbe6be320c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-q75jf\" (UID: \"d65b6030-55a6-4986-9642-a3bbe6be320c\") " pod="openshift-insights/insights-runtime-extractor-q75jf" Jul 07 14:27:17.715357 ip-10-0-134-42 kubenswrapper[2557]: E0707 14:27:17.715321 2557 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 07 14:27:17.715466 ip-10-0-134-42 kubenswrapper[2557]: E0707 14:27:17.715381 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d65b6030-55a6-4986-9642-a3bbe6be320c-insights-runtime-extractor-tls podName:d65b6030-55a6-4986-9642-a3bbe6be320c nodeName:}" failed. No retries permitted until 2026-07-07 14:27:33.715369048 +0000 UTC m=+90.424507406 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/d65b6030-55a6-4986-9642-a3bbe6be320c-insights-runtime-extractor-tls") pod "insights-runtime-extractor-q75jf" (UID: "d65b6030-55a6-4986-9642-a3bbe6be320c") : secret "insights-runtime-extractor-tls" not found Jul 07 14:27:17.715687 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:17.715668 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/d65b6030-55a6-4986-9642-a3bbe6be320c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-q75jf\" (UID: \"d65b6030-55a6-4986-9642-a3bbe6be320c\") " pod="openshift-insights/insights-runtime-extractor-q75jf" Jul 07 14:27:17.842784 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:17.842747 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:27:17.932202 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:17.932165 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:27:18.215529 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:18.215490 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-vd9g9_0fa80df4-c933-4758-b75f-3f70b7217c0d/console-operator/1.log" Jul 07 14:27:18.215899 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:18.215831 2557 scope.go:117] "RemoveContainer" containerID="e3430facd46f3ca9a9ab9a90884875e6e9a19b293c97f8b1ce0e9f83e1fe2d77" Jul 07 14:27:18.215998 ip-10-0-134-42 kubenswrapper[2557]: E0707 14:27:18.215979 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-68b757fb67-vd9g9_openshift-console-operator(0fa80df4-c933-4758-b75f-3f70b7217c0d)\"" pod="openshift-console-operator/console-operator-68b757fb67-vd9g9" podUID="0fa80df4-c933-4758-b75f-3f70b7217c0d" Jul 07 14:27:18.521683 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:18.521592 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:27:18.625171 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:18.625141 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:27:18.774328 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:18.774236 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:27:18.874992 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:18.874963 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:27:19.094341 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:19.094297 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-68b757fb67-vd9g9" Jul 07 14:27:19.094503 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:19.094365 2557 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-68b757fb67-vd9g9" Jul 07 14:27:19.224911 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:19.222713 2557 scope.go:117] "RemoveContainer" containerID="e3430facd46f3ca9a9ab9a90884875e6e9a19b293c97f8b1ce0e9f83e1fe2d77" Jul 07 14:27:19.224911 ip-10-0-134-42 kubenswrapper[2557]: E0707 14:27:19.222971 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-68b757fb67-vd9g9_openshift-console-operator(0fa80df4-c933-4758-b75f-3f70b7217c0d)\"" pod="openshift-console-operator/console-operator-68b757fb67-vd9g9" podUID="0fa80df4-c933-4758-b75f-3f70b7217c0d" Jul 07 14:27:19.280902 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:19.280867 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:27:19.381162 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:19.381083 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:27:19.480999 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:19.480963 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:27:19.558718 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:19.558687 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:27:20.151116 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:20.151084 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:27:20.595269 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:20.595232 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:27:20.697690 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:20.697659 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:27:21.152821 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:21.152774 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:27:21.320794 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:21.320761 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:27:21.405351 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:21.405264 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:27:24.774450 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:24.774403 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/b6d003bd-5d0f-4e35-9bf1-f4eb10d6d1f8-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-rmmws\" (UID: \"b6d003bd-5d0f-4e35-9bf1-f4eb10d6d1f8\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-rmmws" Jul 07 14:27:24.774852 ip-10-0-134-42 kubenswrapper[2557]: E0707 14:27:24.774550 2557 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 07 14:27:24.774852 ip-10-0-134-42 kubenswrapper[2557]: E0707 14:27:24.774604 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b6d003bd-5d0f-4e35-9bf1-f4eb10d6d1f8-samples-operator-tls podName:b6d003bd-5d0f-4e35-9bf1-f4eb10d6d1f8 nodeName:}" failed. No retries permitted until 2026-07-07 14:27:40.7745917 +0000 UTC m=+97.483730059 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/b6d003bd-5d0f-4e35-9bf1-f4eb10d6d1f8-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-rmmws" (UID: "b6d003bd-5d0f-4e35-9bf1-f4eb10d6d1f8") : secret "samples-operator-tls" not found Jul 07 14:27:24.880536 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:24.880493 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:27:24.971431 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:24.971363 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:27:25.071657 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:25.071611 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:27:25.172756 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:25.172727 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:27:25.362215 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:25.362129 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:27:25.437158 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:25.437124 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:27:25.585342 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:25.585310 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:27:25.752352 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:25.752265 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:27:25.943776 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:25.943743 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:27:27.178181 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:27.178150 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:27:27.359424 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:27.359381 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:27:27.557783 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:27.557699 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:27:27.761067 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:27.761033 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:27:27.950792 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:27.950755 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:27:28.548635 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:28.548605 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:27:28.799340 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:28.799255 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:27:28.941790 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:28.941757 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:27:29.342309 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:29.342273 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:27:29.558848 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:29.558815 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:27:29.753657 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:29.753565 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:27:30.143778 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:30.143747 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:27:30.523874 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:30.523787 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/d75c5911-092c-408a-83a7-871dfb379d28-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-bkdv4\" (UID: \"d75c5911-092c-408a-83a7-871dfb379d28\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-bkdv4" Jul 07 14:27:30.524378 ip-10-0-134-42 kubenswrapper[2557]: E0707 14:27:30.524360 2557 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 07 14:27:30.524434 ip-10-0-134-42 kubenswrapper[2557]: E0707 14:27:30.524425 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d75c5911-092c-408a-83a7-871dfb379d28-cluster-monitoring-operator-tls podName:d75c5911-092c-408a-83a7-871dfb379d28 nodeName:}" failed. No retries permitted until 2026-07-07 14:28:02.524409711 +0000 UTC m=+119.233548070 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/d75c5911-092c-408a-83a7-871dfb379d28-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-bkdv4" (UID: "d75c5911-092c-408a-83a7-871dfb379d28") : secret "cluster-monitoring-operator-tls" not found Jul 07 14:27:30.624909 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:30.624882 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/450aeb2e-26e4-4ebb-8006-eb9c43c6e2d1-service-ca-bundle\") pod \"router-default-7c58ff4df-55gs2\" (UID: \"450aeb2e-26e4-4ebb-8006-eb9c43c6e2d1\") " pod="openshift-ingress/router-default-7c58ff4df-55gs2" Jul 07 14:27:30.625289 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:30.624932 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/450aeb2e-26e4-4ebb-8006-eb9c43c6e2d1-metrics-certs\") pod \"router-default-7c58ff4df-55gs2\" (UID: \"450aeb2e-26e4-4ebb-8006-eb9c43c6e2d1\") " pod="openshift-ingress/router-default-7c58ff4df-55gs2" Jul 07 14:27:30.625289 ip-10-0-134-42 kubenswrapper[2557]: E0707 14:27:30.625054 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/450aeb2e-26e4-4ebb-8006-eb9c43c6e2d1-service-ca-bundle podName:450aeb2e-26e4-4ebb-8006-eb9c43c6e2d1 nodeName:}" failed. No retries permitted until 2026-07-07 14:28:02.625037963 +0000 UTC m=+119.334176321 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/450aeb2e-26e4-4ebb-8006-eb9c43c6e2d1-service-ca-bundle") pod "router-default-7c58ff4df-55gs2" (UID: "450aeb2e-26e4-4ebb-8006-eb9c43c6e2d1") : configmap references non-existent config key: service-ca.crt Jul 07 14:27:30.627237 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:30.627214 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/450aeb2e-26e4-4ebb-8006-eb9c43c6e2d1-metrics-certs\") pod \"router-default-7c58ff4df-55gs2\" (UID: \"450aeb2e-26e4-4ebb-8006-eb9c43c6e2d1\") " pod="openshift-ingress/router-default-7c58ff4df-55gs2" Jul 07 14:27:30.942882 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:30.942845 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:27:31.433758 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:31.433718 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:27:31.489802 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:31.489768 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:27:31.550628 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:31.550591 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:27:31.741684 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:31.741607 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:27:32.158658 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:32.158627 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:27:32.829842 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:32.829814 2557 scope.go:117] "RemoveContainer" containerID="e3430facd46f3ca9a9ab9a90884875e6e9a19b293c97f8b1ce0e9f83e1fe2d77" Jul 07 14:27:33.054187 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:33.054160 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:27:33.265792 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:33.265720 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-vd9g9_0fa80df4-c933-4758-b75f-3f70b7217c0d/console-operator/2.log" Jul 07 14:27:33.266118 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:33.266103 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-vd9g9_0fa80df4-c933-4758-b75f-3f70b7217c0d/console-operator/1.log" Jul 07 14:27:33.266160 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:33.266135 2557 generic.go:356] "Generic (PLEG): container finished" podID="0fa80df4-c933-4758-b75f-3f70b7217c0d" containerID="1b21e983634ddec8a770a45829845565580db6407a3d5469b494e3bbd49db5af" exitCode=255 Jul 07 14:27:33.266194 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:33.266182 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-vd9g9" event={"ID":"0fa80df4-c933-4758-b75f-3f70b7217c0d","Type":"ContainerDied","Data":"1b21e983634ddec8a770a45829845565580db6407a3d5469b494e3bbd49db5af"} Jul 07 14:27:33.266237 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:33.266227 2557 scope.go:117] "RemoveContainer" containerID="e3430facd46f3ca9a9ab9a90884875e6e9a19b293c97f8b1ce0e9f83e1fe2d77" Jul 07 14:27:33.266498 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:33.266482 2557 scope.go:117] "RemoveContainer" containerID="1b21e983634ddec8a770a45829845565580db6407a3d5469b494e3bbd49db5af" Jul 07 14:27:33.266686 ip-10-0-134-42 kubenswrapper[2557]: E0707 14:27:33.266667 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-68b757fb67-vd9g9_openshift-console-operator(0fa80df4-c933-4758-b75f-3f70b7217c0d)\"" pod="openshift-console-operator/console-operator-68b757fb67-vd9g9" podUID="0fa80df4-c933-4758-b75f-3f70b7217c0d" Jul 07 14:27:33.324677 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:33.324650 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:27:33.395944 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:33.395924 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:27:33.543348 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:33.543300 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:27:33.745070 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:33.745033 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:27:33.752234 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:33.752212 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d65b6030-55a6-4986-9642-a3bbe6be320c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-q75jf\" (UID: \"d65b6030-55a6-4986-9642-a3bbe6be320c\") " pod="openshift-insights/insights-runtime-extractor-q75jf" Jul 07 14:27:33.754426 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:33.754404 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d65b6030-55a6-4986-9642-a3bbe6be320c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-q75jf\" (UID: \"d65b6030-55a6-4986-9642-a3bbe6be320c\") " pod="openshift-insights/insights-runtime-extractor-q75jf" Jul 07 14:27:34.007852 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:34.007826 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-fh5qh\"" Jul 07 14:27:34.016312 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:34.016297 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-q75jf" Jul 07 14:27:34.136713 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:34.136685 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-q75jf"] Jul 07 14:27:34.140508 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:27:34.140486 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd65b6030_55a6_4986_9642_a3bbe6be320c.slice/crio-774c645dc1c0b5d4234eb2728cb7a8c62faf86799ea7073ae8fcb59182e9379d WatchSource:0}: Error finding container 774c645dc1c0b5d4234eb2728cb7a8c62faf86799ea7073ae8fcb59182e9379d: Status 404 returned error can't find the container with id 774c645dc1c0b5d4234eb2728cb7a8c62faf86799ea7073ae8fcb59182e9379d Jul 07 14:27:34.154162 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:34.154135 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:27:34.269934 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:34.269878 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-vd9g9_0fa80df4-c933-4758-b75f-3f70b7217c0d/console-operator/2.log" Jul 07 14:27:34.270924 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:34.270902 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-q75jf" event={"ID":"d65b6030-55a6-4986-9642-a3bbe6be320c","Type":"ContainerStarted","Data":"774c645dc1c0b5d4234eb2728cb7a8c62faf86799ea7073ae8fcb59182e9379d"} Jul 07 14:27:34.796841 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:34.796762 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:27:35.348095 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:35.348062 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:27:35.416949 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:35.416915 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:27:35.537064 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:35.537014 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:27:36.297689 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:36.297649 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:27:36.375385 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:36.375348 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:27:38.285266 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:38.285231 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-q75jf" event={"ID":"d65b6030-55a6-4986-9642-a3bbe6be320c","Type":"ContainerStarted","Data":"55da6f4580caa79717756e80f4023371b2e2e4783a6ac2c090dfea3feb96bf40"} Jul 07 14:27:39.093572 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:39.093543 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-68b757fb67-vd9g9" Jul 07 14:27:39.093724 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:39.093602 2557 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-68b757fb67-vd9g9" Jul 07 14:27:39.093909 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:39.093898 2557 scope.go:117] "RemoveContainer" containerID="1b21e983634ddec8a770a45829845565580db6407a3d5469b494e3bbd49db5af" Jul 07 14:27:39.094058 ip-10-0-134-42 kubenswrapper[2557]: E0707 14:27:39.094041 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-68b757fb67-vd9g9_openshift-console-operator(0fa80df4-c933-4758-b75f-3f70b7217c0d)\"" pod="openshift-console-operator/console-operator-68b757fb67-vd9g9" podUID="0fa80df4-c933-4758-b75f-3f70b7217c0d" Jul 07 14:27:39.290946 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:39.290905 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-q75jf" event={"ID":"d65b6030-55a6-4986-9642-a3bbe6be320c","Type":"ContainerStarted","Data":"c80e9685faf8d6d0f9ae082140bcf144a0c9a600be745a43f4dcfa10cec72fc8"} Jul 07 14:27:40.815119 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:40.815079 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/b6d003bd-5d0f-4e35-9bf1-f4eb10d6d1f8-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-rmmws\" (UID: \"b6d003bd-5d0f-4e35-9bf1-f4eb10d6d1f8\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-rmmws" Jul 07 14:27:40.817660 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:40.817628 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/b6d003bd-5d0f-4e35-9bf1-f4eb10d6d1f8-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-rmmws\" (UID: \"b6d003bd-5d0f-4e35-9bf1-f4eb10d6d1f8\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-rmmws" Jul 07 14:27:40.984065 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:40.984028 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-rmmws" Jul 07 14:27:41.187975 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:41.187950 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-rmmws"] Jul 07 14:27:41.299082 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:41.299052 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-q75jf" event={"ID":"d65b6030-55a6-4986-9642-a3bbe6be320c","Type":"ContainerStarted","Data":"869237201ec807b8f558f26bd5d7a37bd99e7c7718c5d682ea235be1a20ef4cb"} Jul 07 14:27:41.300116 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:41.300091 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-rmmws" event={"ID":"b6d003bd-5d0f-4e35-9bf1-f4eb10d6d1f8","Type":"ContainerStarted","Data":"e2a2a9b2d332e81fadd465d705293a263c6c3e2a3f3edfdaab17cfa36591b27d"} Jul 07 14:27:41.326506 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:41.326456 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-989f8b8dc-fzbff"] Jul 07 14:27:41.328298 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:41.328285 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-fzbff" Jul 07 14:27:41.330245 ip-10-0-134-42 kubenswrapper[2557]: E0707 14:27:41.330224 2557 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"network-diagnostics-dockercfg-h8zkk\" is forbidden: User \"system:node:ip-10-0-134-42.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"openshift-network-diagnostics\": no relationship found between node 'ip-10-0-134-42.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-h8zkk\"" type="*v1.Secret" Jul 07 14:27:41.340943 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:41.340906 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-q75jf" podStartSLOduration=33.482552232 podStartE2EDuration="40.340895366s" totalImagesPullingTime="6.858343134s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 14:27:01 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:27:41.340320232 +0000 UTC m=+98.049458612" watchObservedRunningTime="2026-07-07 14:27:41.340895366 +0000 UTC m=+98.050033746" Jul 07 14:27:41.344781 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:41.344758 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-989f8b8dc-fzbff"] Jul 07 14:27:41.401195 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:41.401162 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:27:41.419290 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:41.419269 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t7l9n\" (UniqueName: \"kubernetes.io/projected/d1b1199b-8a58-4376-ad02-ce6ff9ba4b35-kube-api-access-t7l9n\") pod \"network-check-source-989f8b8dc-fzbff\" (UID: \"d1b1199b-8a58-4376-ad02-ce6ff9ba4b35\") " pod="openshift-network-diagnostics/network-check-source-989f8b8dc-fzbff" Jul 07 14:27:41.445999 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:41.445974 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-6bffdf9778-2hh7m"] Jul 07 14:27:41.448174 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:41.448159 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6bffdf9778-2hh7m" Jul 07 14:27:41.450914 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:41.450888 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\"" Jul 07 14:27:41.451021 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:41.451003 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\"" Jul 07 14:27:41.451119 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:41.451098 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-52tgs\"" Jul 07 14:27:41.474112 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:41.474094 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6bffdf9778-2hh7m"] Jul 07 14:27:41.519938 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:41.519914 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/7a17e389-0667-415a-b38d-216a19842db3-nginx-conf\") pod \"networking-console-plugin-6bffdf9778-2hh7m\" (UID: \"7a17e389-0667-415a-b38d-216a19842db3\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-2hh7m" Jul 07 14:27:41.520053 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:41.519945 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t7l9n\" (UniqueName: \"kubernetes.io/projected/d1b1199b-8a58-4376-ad02-ce6ff9ba4b35-kube-api-access-t7l9n\") pod \"network-check-source-989f8b8dc-fzbff\" (UID: \"d1b1199b-8a58-4376-ad02-ce6ff9ba4b35\") " pod="openshift-network-diagnostics/network-check-source-989f8b8dc-fzbff" Jul 07 14:27:41.520053 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:41.519964 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/7a17e389-0667-415a-b38d-216a19842db3-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-2hh7m\" (UID: \"7a17e389-0667-415a-b38d-216a19842db3\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-2hh7m" Jul 07 14:27:41.528179 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:41.528154 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t7l9n\" (UniqueName: \"kubernetes.io/projected/d1b1199b-8a58-4376-ad02-ce6ff9ba4b35-kube-api-access-t7l9n\") pod \"network-check-source-989f8b8dc-fzbff\" (UID: \"d1b1199b-8a58-4376-ad02-ce6ff9ba4b35\") " pod="openshift-network-diagnostics/network-check-source-989f8b8dc-fzbff" Jul 07 14:27:41.532213 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:41.532174 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:27:41.620792 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:41.620738 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/7a17e389-0667-415a-b38d-216a19842db3-nginx-conf\") pod \"networking-console-plugin-6bffdf9778-2hh7m\" (UID: \"7a17e389-0667-415a-b38d-216a19842db3\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-2hh7m" Jul 07 14:27:41.620792 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:41.620769 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/7a17e389-0667-415a-b38d-216a19842db3-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-2hh7m\" (UID: \"7a17e389-0667-415a-b38d-216a19842db3\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-2hh7m" Jul 07 14:27:41.621293 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:41.621274 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/7a17e389-0667-415a-b38d-216a19842db3-nginx-conf\") pod \"networking-console-plugin-6bffdf9778-2hh7m\" (UID: \"7a17e389-0667-415a-b38d-216a19842db3\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-2hh7m" Jul 07 14:27:41.622857 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:41.622841 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/7a17e389-0667-415a-b38d-216a19842db3-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-2hh7m\" (UID: \"7a17e389-0667-415a-b38d-216a19842db3\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-2hh7m" Jul 07 14:27:41.629153 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:41.629134 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:27:41.757615 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:41.757592 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6bffdf9778-2hh7m" Jul 07 14:27:41.902787 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:41.902680 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6bffdf9778-2hh7m"] Jul 07 14:27:41.907297 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:27:41.907257 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7a17e389_0667_415a_b38d_216a19842db3.slice/crio-f62fb24f65367a3be9390d83153c1d5d359e73031e0126c3423d86f2e8323d5f WatchSource:0}: Error finding container f62fb24f65367a3be9390d83153c1d5d359e73031e0126c3423d86f2e8323d5f: Status 404 returned error can't find the container with id f62fb24f65367a3be9390d83153c1d5d359e73031e0126c3423d86f2e8323d5f Jul 07 14:27:42.304863 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:42.304786 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6bffdf9778-2hh7m" event={"ID":"7a17e389-0667-415a-b38d-216a19842db3","Type":"ContainerStarted","Data":"f62fb24f65367a3be9390d83153c1d5d359e73031e0126c3423d86f2e8323d5f"} Jul 07 14:27:42.326480 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:42.326456 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-h8zkk\"" Jul 07 14:27:42.336906 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:42.336891 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-fzbff" Jul 07 14:27:42.427164 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:42.427133 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3d597bc9-74bf-4809-a33d-0a28dec34cb1-metrics-tls\") pod \"dns-default-5ngmq\" (UID: \"3d597bc9-74bf-4809-a33d-0a28dec34cb1\") " pod="openshift-dns/dns-default-5ngmq" Jul 07 14:27:42.427272 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:42.427256 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/648e2b2f-08a5-4a83-b9aa-319ab2527ae4-registry-tls\") pod \"image-registry-687956bbfb-m9bjn\" (UID: \"648e2b2f-08a5-4a83-b9aa-319ab2527ae4\") " pod="openshift-image-registry/image-registry-687956bbfb-m9bjn" Jul 07 14:27:42.429558 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:42.429536 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3d597bc9-74bf-4809-a33d-0a28dec34cb1-metrics-tls\") pod \"dns-default-5ngmq\" (UID: \"3d597bc9-74bf-4809-a33d-0a28dec34cb1\") " pod="openshift-dns/dns-default-5ngmq" Jul 07 14:27:42.429661 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:42.429564 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/648e2b2f-08a5-4a83-b9aa-319ab2527ae4-registry-tls\") pod \"image-registry-687956bbfb-m9bjn\" (UID: \"648e2b2f-08a5-4a83-b9aa-319ab2527ae4\") " pod="openshift-image-registry/image-registry-687956bbfb-m9bjn" Jul 07 14:27:42.461026 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:42.460999 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-989f8b8dc-fzbff"] Jul 07 14:27:42.462921 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:42.462898 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-6pj2m\"" Jul 07 14:27:42.463674 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:27:42.463652 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd1b1199b_8a58_4376_ad02_ce6ff9ba4b35.slice/crio-6ce5fb4eeff18587359808c98b8daa491e2afe25bab08caa3a780161c1341ca8 WatchSource:0}: Error finding container 6ce5fb4eeff18587359808c98b8daa491e2afe25bab08caa3a780161c1341ca8: Status 404 returned error can't find the container with id 6ce5fb4eeff18587359808c98b8daa491e2afe25bab08caa3a780161c1341ca8 Jul 07 14:27:42.471909 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:42.471884 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-687956bbfb-m9bjn" Jul 07 14:27:42.491981 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:42.491956 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-ksq8k\"" Jul 07 14:27:42.500346 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:42.500325 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-5ngmq" Jul 07 14:27:42.644495 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:42.644340 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-687956bbfb-m9bjn"] Jul 07 14:27:42.667035 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:42.666987 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-5ngmq"] Jul 07 14:27:43.094152 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:27:43.094122 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3d597bc9_74bf_4809_a33d_0a28dec34cb1.slice/crio-c51ebb384d3a35a1ce947bd061e8bd407e0e4e6e69b9bdb1e24fa5bcfe1311e0 WatchSource:0}: Error finding container c51ebb384d3a35a1ce947bd061e8bd407e0e4e6e69b9bdb1e24fa5bcfe1311e0: Status 404 returned error can't find the container with id c51ebb384d3a35a1ce947bd061e8bd407e0e4e6e69b9bdb1e24fa5bcfe1311e0 Jul 07 14:27:43.094793 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:27:43.094768 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod648e2b2f_08a5_4a83_b9aa_319ab2527ae4.slice/crio-3a70913a2798e9c42e0fdea44b58daf897ec4385e7134dd72e435a23a7859367 WatchSource:0}: Error finding container 3a70913a2798e9c42e0fdea44b58daf897ec4385e7134dd72e435a23a7859367: Status 404 returned error can't find the container with id 3a70913a2798e9c42e0fdea44b58daf897ec4385e7134dd72e435a23a7859367 Jul 07 14:27:43.309739 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:43.309697 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-5ngmq" event={"ID":"3d597bc9-74bf-4809-a33d-0a28dec34cb1","Type":"ContainerStarted","Data":"c51ebb384d3a35a1ce947bd061e8bd407e0e4e6e69b9bdb1e24fa5bcfe1311e0"} Jul 07 14:27:43.311455 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:43.311421 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-fzbff" event={"ID":"d1b1199b-8a58-4376-ad02-ce6ff9ba4b35","Type":"ContainerStarted","Data":"060f52e598705f6c0a61b9399a835183d44839d3c80a0fe5cf93934b73c8f9b9"} Jul 07 14:27:43.311610 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:43.311475 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-fzbff" event={"ID":"d1b1199b-8a58-4376-ad02-ce6ff9ba4b35","Type":"ContainerStarted","Data":"6ce5fb4eeff18587359808c98b8daa491e2afe25bab08caa3a780161c1341ca8"} Jul 07 14:27:43.312948 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:43.312917 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-687956bbfb-m9bjn" event={"ID":"648e2b2f-08a5-4a83-b9aa-319ab2527ae4","Type":"ContainerStarted","Data":"3a70913a2798e9c42e0fdea44b58daf897ec4385e7134dd72e435a23a7859367"} Jul 07 14:27:43.328400 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:43.328359 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-fzbff" podStartSLOduration=2.328345527 podStartE2EDuration="2.328345527s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 14:27:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:27:43.326876383 +0000 UTC m=+100.036014765" watchObservedRunningTime="2026-07-07 14:27:43.328345527 +0000 UTC m=+100.037483885" Jul 07 14:27:44.318707 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:44.318669 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6bffdf9778-2hh7m" event={"ID":"7a17e389-0667-415a-b38d-216a19842db3","Type":"ContainerStarted","Data":"9c4f093e4990252e22964af954ff19c764dfe8a12bec7f5cdfbf74ef62aff01b"} Jul 07 14:27:44.320802 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:44.320774 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-687956bbfb-m9bjn" event={"ID":"648e2b2f-08a5-4a83-b9aa-319ab2527ae4","Type":"ContainerStarted","Data":"28b8972dd748c52490c47687d88cfec1ba1c791207ba74bc8bb05ba48af16d27"} Jul 07 14:27:44.320908 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:44.320825 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-687956bbfb-m9bjn" Jul 07 14:27:44.322942 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:44.322911 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-rmmws" event={"ID":"b6d003bd-5d0f-4e35-9bf1-f4eb10d6d1f8","Type":"ContainerStarted","Data":"ee2b3fee6d772dd2652727fa07f6051810e03f02c6516cb8bdc781d5f7d2b62c"} Jul 07 14:27:44.323059 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:44.322958 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-rmmws" event={"ID":"b6d003bd-5d0f-4e35-9bf1-f4eb10d6d1f8","Type":"ContainerStarted","Data":"1c5d286478565b6ff36f31a7e2fcd9a7663fcf3c7f410411a86f2a1c24dda3b9"} Jul 07 14:27:44.335390 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:44.335344 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-6bffdf9778-2hh7m" podStartSLOduration=1.828582868 podStartE2EDuration="3.3353273s" totalImagesPullingTime="1.506744432s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 14:27:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:27:44.334828746 +0000 UTC m=+101.043967128" watchObservedRunningTime="2026-07-07 14:27:44.3353273 +0000 UTC m=+101.044465682" Jul 07 14:27:44.381505 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:44.381292 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-rmmws" podStartSLOduration=34.193834668 podStartE2EDuration="36.381275202s" totalImagesPullingTime="2.187440534s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 14:27:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:27:44.381098229 +0000 UTC m=+101.090236611" watchObservedRunningTime="2026-07-07 14:27:44.381275202 +0000 UTC m=+101.090413583" Jul 07 14:27:44.382371 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:44.382330 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-687956bbfb-m9bjn" podStartSLOduration=99.382315925 podStartE2EDuration="1m39.382315925s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 14:26:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:27:44.362982777 +0000 UTC m=+101.072121158" watchObservedRunningTime="2026-07-07 14:27:44.382315925 +0000 UTC m=+101.091454307" Jul 07 14:27:44.452678 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:44.452630 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:27:44.531553 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:44.531511 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:27:44.716567 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:44.716538 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:27:44.816064 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:44.816035 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:27:45.328731 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:45.328694 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-5ngmq" event={"ID":"3d597bc9-74bf-4809-a33d-0a28dec34cb1","Type":"ContainerStarted","Data":"c7ea692d2a8fe46fbaaef79b34937cee19b5b1ed65e434443fa272f85cbad003"} Jul 07 14:27:45.329169 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:45.328756 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-5ngmq" event={"ID":"3d597bc9-74bf-4809-a33d-0a28dec34cb1","Type":"ContainerStarted","Data":"36e2fe522e8e82f1cb481d47ae7591936902a2e7de2c17eee7497df133b58a1b"} Jul 07 14:27:46.331971 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:46.331943 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-5ngmq" Jul 07 14:27:47.214538 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:47.214495 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-bqthq" Jul 07 14:27:47.231304 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:47.231254 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-5ngmq" podStartSLOduration=67.75522298 podStartE2EDuration="1m9.231239125s" totalImagesPullingTime="1.476016145s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 14:26:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:27:45.349824858 +0000 UTC m=+102.058963237" watchObservedRunningTime="2026-07-07 14:27:47.231239125 +0000 UTC m=+103.940377512" Jul 07 14:27:47.643026 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:47.642987 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:27:47.706070 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:47.706043 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:27:50.830216 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:50.830185 2557 scope.go:117] "RemoveContainer" containerID="1b21e983634ddec8a770a45829845565580db6407a3d5469b494e3bbd49db5af" Jul 07 14:27:50.830659 ip-10-0-134-42 kubenswrapper[2557]: E0707 14:27:50.830335 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-68b757fb67-vd9g9_openshift-console-operator(0fa80df4-c933-4758-b75f-3f70b7217c0d)\"" pod="openshift-console-operator/console-operator-68b757fb67-vd9g9" podUID="0fa80df4-c933-4758-b75f-3f70b7217c0d" Jul 07 14:27:55.641500 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:55.641460 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:27:55.714120 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:55.714089 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:27:56.337170 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:27:56.337139 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-5ngmq" Jul 07 14:28:02.588388 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:02.588348 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/d75c5911-092c-408a-83a7-871dfb379d28-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-bkdv4\" (UID: \"d75c5911-092c-408a-83a7-871dfb379d28\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-bkdv4" Jul 07 14:28:02.590666 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:02.590639 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/d75c5911-092c-408a-83a7-871dfb379d28-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-bkdv4\" (UID: \"d75c5911-092c-408a-83a7-871dfb379d28\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-bkdv4" Jul 07 14:28:02.689191 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:02.689162 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/450aeb2e-26e4-4ebb-8006-eb9c43c6e2d1-service-ca-bundle\") pod \"router-default-7c58ff4df-55gs2\" (UID: \"450aeb2e-26e4-4ebb-8006-eb9c43c6e2d1\") " pod="openshift-ingress/router-default-7c58ff4df-55gs2" Jul 07 14:28:02.689741 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:02.689722 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/450aeb2e-26e4-4ebb-8006-eb9c43c6e2d1-service-ca-bundle\") pod \"router-default-7c58ff4df-55gs2\" (UID: \"450aeb2e-26e4-4ebb-8006-eb9c43c6e2d1\") " pod="openshift-ingress/router-default-7c58ff4df-55gs2" Jul 07 14:28:02.849065 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:02.849001 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-xj2t8\"" Jul 07 14:28:02.857468 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:02.857450 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-bkdv4" Jul 07 14:28:02.918229 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:02.918189 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-6wnfq\"" Jul 07 14:28:02.926963 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:02.926943 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-7c58ff4df-55gs2" Jul 07 14:28:02.982763 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:02.982735 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-64786d9cbb-bkdv4"] Jul 07 14:28:02.984478 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:28:02.984453 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd75c5911_092c_408a_83a7_871dfb379d28.slice/crio-8680f0c980b061ecf83cc0900881cbc96cf5367a4053e72f4947324fb9d16854 WatchSource:0}: Error finding container 8680f0c980b061ecf83cc0900881cbc96cf5367a4053e72f4947324fb9d16854: Status 404 returned error can't find the container with id 8680f0c980b061ecf83cc0900881cbc96cf5367a4053e72f4947324fb9d16854 Jul 07 14:28:03.052272 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:03.052242 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-7c58ff4df-55gs2"] Jul 07 14:28:03.054991 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:28:03.054967 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod450aeb2e_26e4_4ebb_8006_eb9c43c6e2d1.slice/crio-5430d821b6feb61949adf1d57ac84298cccde11d7d9664561d9db08b44ac93e2 WatchSource:0}: Error finding container 5430d821b6feb61949adf1d57ac84298cccde11d7d9664561d9db08b44ac93e2: Status 404 returned error can't find the container with id 5430d821b6feb61949adf1d57ac84298cccde11d7d9664561d9db08b44ac93e2 Jul 07 14:28:03.387710 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:03.387632 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-bkdv4" event={"ID":"d75c5911-092c-408a-83a7-871dfb379d28","Type":"ContainerStarted","Data":"8680f0c980b061ecf83cc0900881cbc96cf5367a4053e72f4947324fb9d16854"} Jul 07 14:28:03.389002 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:03.388975 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-7c58ff4df-55gs2" event={"ID":"450aeb2e-26e4-4ebb-8006-eb9c43c6e2d1","Type":"ContainerStarted","Data":"4074542b7dd1a4fe299b77fcc29eb8bff8b84894ad4ef982ce60e9ca9f93ab22"} Jul 07 14:28:03.389119 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:03.389021 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-7c58ff4df-55gs2" event={"ID":"450aeb2e-26e4-4ebb-8006-eb9c43c6e2d1","Type":"ContainerStarted","Data":"5430d821b6feb61949adf1d57ac84298cccde11d7d9664561d9db08b44ac93e2"} Jul 07 14:28:03.525789 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:03.525741 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-7c58ff4df-55gs2" podStartSLOduration=65.525723541 podStartE2EDuration="1m5.525723541s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 14:26:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:28:03.407786811 +0000 UTC m=+120.116925191" watchObservedRunningTime="2026-07-07 14:28:03.525723541 +0000 UTC m=+120.234861922" Jul 07 14:28:03.526131 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:03.526105 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-687956bbfb-m9bjn"] Jul 07 14:28:03.531070 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:03.531048 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-687956bbfb-m9bjn" Jul 07 14:28:03.605920 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:03.605893 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:28:03.661017 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:03.660930 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:28:03.831847 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:03.831818 2557 scope.go:117] "RemoveContainer" containerID="1b21e983634ddec8a770a45829845565580db6407a3d5469b494e3bbd49db5af" Jul 07 14:28:03.927654 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:03.927465 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-7c58ff4df-55gs2" Jul 07 14:28:03.929923 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:03.929901 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-7c58ff4df-55gs2" Jul 07 14:28:03.933203 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:03.933094 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:28:03.993943 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:03.993896 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:28:04.395156 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:04.395121 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-vd9g9_0fa80df4-c933-4758-b75f-3f70b7217c0d/console-operator/2.log" Jul 07 14:28:04.395327 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:04.395261 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-vd9g9" event={"ID":"0fa80df4-c933-4758-b75f-3f70b7217c0d","Type":"ContainerStarted","Data":"581a848a90e4c44d9d7b6c6f54a63056d006ae4b3786f382c6074cfd16fa1f12"} Jul 07 14:28:04.395561 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:04.395536 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-7c58ff4df-55gs2" Jul 07 14:28:04.395779 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:04.395762 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-68b757fb67-vd9g9" Jul 07 14:28:04.396979 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:04.396953 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-7c58ff4df-55gs2" Jul 07 14:28:04.415312 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:04.415266 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-68b757fb67-vd9g9" podStartSLOduration=50.03488946 podStartE2EDuration="56.41524924s" totalImagesPullingTime="6.38035978s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 14:27:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:28:04.413205888 +0000 UTC m=+121.122344286" watchObservedRunningTime="2026-07-07 14:28:04.41524924 +0000 UTC m=+121.124387621" Jul 07 14:28:04.937624 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:04.937576 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-68b757fb67-vd9g9" Jul 07 14:28:05.077078 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:05.077029 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:28:05.179964 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:05.179928 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:28:05.253994 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:05.253965 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-fpkmh"] Jul 07 14:28:05.254188 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:05.254163 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:28:05.256164 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:05.256148 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-fpkmh" Jul 07 14:28:05.257846 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:05.257826 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\"" Jul 07 14:28:05.257945 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:05.257929 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-j5hp5\"" Jul 07 14:28:05.264578 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:05.264557 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-fpkmh"] Jul 07 14:28:05.335115 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:05.335052 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:28:05.400143 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:05.400114 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-bkdv4" event={"ID":"d75c5911-092c-408a-83a7-871dfb379d28","Type":"ContainerStarted","Data":"057d2feb3dd0b8ce19255e53e711693c03390b9e4f7a5126a7b2f681ebdbfd5d"} Jul 07 14:28:05.414143 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:05.414120 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/e3ead245-ab3d-41f4-9e62-c256b1968e9c-tls-certificates\") pod \"prometheus-operator-admission-webhook-d559d66d5-fpkmh\" (UID: \"e3ead245-ab3d-41f4-9e62-c256b1968e9c\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-fpkmh" Jul 07 14:28:05.415314 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:05.415281 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-bkdv4" podStartSLOduration=65.657169162 podStartE2EDuration="1m7.415267563s" totalImagesPullingTime="1.758098401s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 14:26:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:28:05.41478595 +0000 UTC m=+122.123924347" watchObservedRunningTime="2026-07-07 14:28:05.415267563 +0000 UTC m=+122.124405941" Jul 07 14:28:05.422255 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:05.422232 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:28:05.494341 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:05.494317 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:28:05.514600 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:05.514570 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/e3ead245-ab3d-41f4-9e62-c256b1968e9c-tls-certificates\") pod \"prometheus-operator-admission-webhook-d559d66d5-fpkmh\" (UID: \"e3ead245-ab3d-41f4-9e62-c256b1968e9c\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-fpkmh" Jul 07 14:28:05.516910 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:05.516890 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/e3ead245-ab3d-41f4-9e62-c256b1968e9c-tls-certificates\") pod \"prometheus-operator-admission-webhook-d559d66d5-fpkmh\" (UID: \"e3ead245-ab3d-41f4-9e62-c256b1968e9c\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-fpkmh" Jul 07 14:28:05.564997 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:05.564964 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-fpkmh" Jul 07 14:28:05.680881 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:05.680854 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-fpkmh"] Jul 07 14:28:05.683863 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:28:05.683837 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode3ead245_ab3d_41f4_9e62_c256b1968e9c.slice/crio-ea640f78a30f67d4762e60532e90ec9af37f9132d07a11e97fb727cbda9ded88 WatchSource:0}: Error finding container ea640f78a30f67d4762e60532e90ec9af37f9132d07a11e97fb727cbda9ded88: Status 404 returned error can't find the container with id ea640f78a30f67d4762e60532e90ec9af37f9132d07a11e97fb727cbda9ded88 Jul 07 14:28:05.794503 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:05.794472 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:28:05.855203 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:05.855170 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:28:06.202361 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:06.202318 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:28:06.262112 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:06.262085 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:28:06.404813 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:06.404782 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-fpkmh" event={"ID":"e3ead245-ab3d-41f4-9e62-c256b1968e9c","Type":"ContainerStarted","Data":"ea640f78a30f67d4762e60532e90ec9af37f9132d07a11e97fb727cbda9ded88"} Jul 07 14:28:06.421570 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:06.421541 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:28:06.478805 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:06.478742 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:28:06.602223 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:06.602196 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:28:06.992621 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:06.992590 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:28:07.055565 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:07.055536 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:28:07.157852 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:07.157826 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:28:08.414856 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:08.414814 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-fpkmh" event={"ID":"e3ead245-ab3d-41f4-9e62-c256b1968e9c","Type":"ContainerStarted","Data":"25b41caf1ffbad0b7cfac76972dd213e0b2e4389b585b1f0663c473cbf097742"} Jul 07 14:28:08.415234 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:08.415049 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-fpkmh" Jul 07 14:28:08.419573 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:08.419552 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-fpkmh" Jul 07 14:28:08.432990 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:08.432953 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-fpkmh" podStartSLOduration=1.669468988 podStartE2EDuration="3.43294111s" totalImagesPullingTime="1.763472122s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 14:28:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:28:08.432104045 +0000 UTC m=+125.141242426" watchObservedRunningTime="2026-07-07 14:28:08.43294111 +0000 UTC m=+125.142079903" Jul 07 14:28:08.826835 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:08.826800 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:28:08.890609 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:08.890572 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:28:09.274705 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:09.274629 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:28:09.374797 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:09.374759 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:28:09.841655 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:09.841626 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:28:12.316053 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:12.316022 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:28:12.666172 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:12.666142 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/320ab15e-c868-4a37-a3e6-1cea72bb74a4-metrics-certs\") pod \"network-metrics-daemon-s4p8l\" (UID: \"320ab15e-c868-4a37-a3e6-1cea72bb74a4\") " pod="openshift-multus/network-metrics-daemon-s4p8l" Jul 07 14:28:12.668578 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:12.668562 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/320ab15e-c868-4a37-a3e6-1cea72bb74a4-metrics-certs\") pod \"network-metrics-daemon-s4p8l\" (UID: \"320ab15e-c868-4a37-a3e6-1cea72bb74a4\") " pod="openshift-multus/network-metrics-daemon-s4p8l" Jul 07 14:28:12.849493 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:12.849466 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-ps6h4\"" Jul 07 14:28:12.857434 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:12.857412 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s4p8l" Jul 07 14:28:12.992693 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:12.992666 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-s4p8l"] Jul 07 14:28:12.995920 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:28:12.995897 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod320ab15e_c868_4a37_a3e6_1cea72bb74a4.slice/crio-7c12ac6036a1291d15bb14da9ce07c8229898912ca1e3e2d4531038cbf92a120 WatchSource:0}: Error finding container 7c12ac6036a1291d15bb14da9ce07c8229898912ca1e3e2d4531038cbf92a120: Status 404 returned error can't find the container with id 7c12ac6036a1291d15bb14da9ce07c8229898912ca1e3e2d4531038cbf92a120 Jul 07 14:28:13.433057 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:13.433018 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-s4p8l" event={"ID":"320ab15e-c868-4a37-a3e6-1cea72bb74a4","Type":"ContainerStarted","Data":"7c12ac6036a1291d15bb14da9ce07c8229898912ca1e3e2d4531038cbf92a120"} Jul 07 14:28:14.132344 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:14.132309 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:28:14.231337 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:14.231259 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:28:14.331642 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:14.331614 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:28:14.438829 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:14.438745 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-s4p8l" event={"ID":"320ab15e-c868-4a37-a3e6-1cea72bb74a4","Type":"ContainerStarted","Data":"cd95761bd5a529b71eb062ca85479912870626b84ce0178e3db90e7aa42e5db1"} Jul 07 14:28:14.438829 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:14.438813 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-s4p8l" event={"ID":"320ab15e-c868-4a37-a3e6-1cea72bb74a4","Type":"ContainerStarted","Data":"0cda0d5a7323ed7b67e7a66b6245139e04d5dd8352b2297f30d67e0ec4838d82"} Jul 07 14:28:14.459036 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:14.458991 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-s4p8l" podStartSLOduration=129.269136612 podStartE2EDuration="2m10.458977376s" totalImagesPullingTime="1.189840764s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 14:26:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:28:14.457683164 +0000 UTC m=+131.166821544" watchObservedRunningTime="2026-07-07 14:28:14.458977376 +0000 UTC m=+131.168115755" Jul 07 14:28:14.459275 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:14.459251 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:28:14.563387 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:14.563360 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:28:14.665504 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:14.665474 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:28:14.665504 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:14.665497 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-54cf96dfcf-hdnqk"] Jul 07 14:28:14.667953 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:14.667939 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-hdnqk" Jul 07 14:28:14.669668 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:14.669634 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\"" Jul 07 14:28:14.669872 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:14.669853 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\"" Jul 07 14:28:14.669946 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:14.669892 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-fb9m2\"" Jul 07 14:28:14.669992 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:14.669957 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 07 14:28:14.679035 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:14.679015 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-54cf96dfcf-hdnqk"] Jul 07 14:28:14.684113 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:14.684092 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/860b5cea-cdae-499f-b01c-1fdf449ab505-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-54cf96dfcf-hdnqk\" (UID: \"860b5cea-cdae-499f-b01c-1fdf449ab505\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-hdnqk" Jul 07 14:28:14.684219 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:14.684128 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nsz6j\" (UniqueName: \"kubernetes.io/projected/860b5cea-cdae-499f-b01c-1fdf449ab505-kube-api-access-nsz6j\") pod \"openshift-state-metrics-54cf96dfcf-hdnqk\" (UID: \"860b5cea-cdae-499f-b01c-1fdf449ab505\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-hdnqk" Jul 07 14:28:14.684219 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:14.684156 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/860b5cea-cdae-499f-b01c-1fdf449ab505-openshift-state-metrics-tls\") pod \"openshift-state-metrics-54cf96dfcf-hdnqk\" (UID: \"860b5cea-cdae-499f-b01c-1fdf449ab505\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-hdnqk" Jul 07 14:28:14.684337 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:14.684223 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/860b5cea-cdae-499f-b01c-1fdf449ab505-metrics-client-ca\") pod \"openshift-state-metrics-54cf96dfcf-hdnqk\" (UID: \"860b5cea-cdae-499f-b01c-1fdf449ab505\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-hdnqk" Jul 07 14:28:14.689024 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:14.688973 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-ftkv4"] Jul 07 14:28:14.691084 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:14.691066 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-ftkv4" Jul 07 14:28:14.692773 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:14.692752 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 07 14:28:14.692870 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:14.692816 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 07 14:28:14.692942 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:14.692819 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 07 14:28:14.693105 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:14.693087 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-nnrwj\"" Jul 07 14:28:14.760760 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:14.760740 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:28:14.785555 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:14.785514 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/b7247b62-46dd-4b19-8afd-52f281ee1f66-node-exporter-textfile\") pod \"node-exporter-ftkv4\" (UID: \"b7247b62-46dd-4b19-8afd-52f281ee1f66\") " pod="openshift-monitoring/node-exporter-ftkv4" Jul 07 14:28:14.785655 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:14.785563 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hdwz4\" (UniqueName: \"kubernetes.io/projected/b7247b62-46dd-4b19-8afd-52f281ee1f66-kube-api-access-hdwz4\") pod \"node-exporter-ftkv4\" (UID: \"b7247b62-46dd-4b19-8afd-52f281ee1f66\") " pod="openshift-monitoring/node-exporter-ftkv4" Jul 07 14:28:14.785655 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:14.785588 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nsz6j\" (UniqueName: \"kubernetes.io/projected/860b5cea-cdae-499f-b01c-1fdf449ab505-kube-api-access-nsz6j\") pod \"openshift-state-metrics-54cf96dfcf-hdnqk\" (UID: \"860b5cea-cdae-499f-b01c-1fdf449ab505\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-hdnqk" Jul 07 14:28:14.785736 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:14.785656 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/b7247b62-46dd-4b19-8afd-52f281ee1f66-root\") pod \"node-exporter-ftkv4\" (UID: \"b7247b62-46dd-4b19-8afd-52f281ee1f66\") " pod="openshift-monitoring/node-exporter-ftkv4" Jul 07 14:28:14.785736 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:14.785700 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/860b5cea-cdae-499f-b01c-1fdf449ab505-openshift-state-metrics-tls\") pod \"openshift-state-metrics-54cf96dfcf-hdnqk\" (UID: \"860b5cea-cdae-499f-b01c-1fdf449ab505\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-hdnqk" Jul 07 14:28:14.785822 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:14.785745 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/b7247b62-46dd-4b19-8afd-52f281ee1f66-node-exporter-accelerators-collector-config\") pod \"node-exporter-ftkv4\" (UID: \"b7247b62-46dd-4b19-8afd-52f281ee1f66\") " pod="openshift-monitoring/node-exporter-ftkv4" Jul 07 14:28:14.785822 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:14.785782 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b7247b62-46dd-4b19-8afd-52f281ee1f66-metrics-client-ca\") pod \"node-exporter-ftkv4\" (UID: \"b7247b62-46dd-4b19-8afd-52f281ee1f66\") " pod="openshift-monitoring/node-exporter-ftkv4" Jul 07 14:28:14.785822 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:14.785810 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/860b5cea-cdae-499f-b01c-1fdf449ab505-metrics-client-ca\") pod \"openshift-state-metrics-54cf96dfcf-hdnqk\" (UID: \"860b5cea-cdae-499f-b01c-1fdf449ab505\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-hdnqk" Jul 07 14:28:14.785822 ip-10-0-134-42 kubenswrapper[2557]: E0707 14:28:14.785816 2557 secret.go:189] Couldn't get secret openshift-monitoring/openshift-state-metrics-tls: secret "openshift-state-metrics-tls" not found Jul 07 14:28:14.785972 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:14.785838 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b7247b62-46dd-4b19-8afd-52f281ee1f66-sys\") pod \"node-exporter-ftkv4\" (UID: \"b7247b62-46dd-4b19-8afd-52f281ee1f66\") " pod="openshift-monitoring/node-exporter-ftkv4" Jul 07 14:28:14.785972 ip-10-0-134-42 kubenswrapper[2557]: E0707 14:28:14.785869 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/860b5cea-cdae-499f-b01c-1fdf449ab505-openshift-state-metrics-tls podName:860b5cea-cdae-499f-b01c-1fdf449ab505 nodeName:}" failed. No retries permitted until 2026-07-07 14:28:15.285852979 +0000 UTC m=+131.994991354 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "openshift-state-metrics-tls" (UniqueName: "kubernetes.io/secret/860b5cea-cdae-499f-b01c-1fdf449ab505-openshift-state-metrics-tls") pod "openshift-state-metrics-54cf96dfcf-hdnqk" (UID: "860b5cea-cdae-499f-b01c-1fdf449ab505") : secret "openshift-state-metrics-tls" not found Jul 07 14:28:14.785972 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:14.785946 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b7247b62-46dd-4b19-8afd-52f281ee1f66-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-ftkv4\" (UID: \"b7247b62-46dd-4b19-8afd-52f281ee1f66\") " pod="openshift-monitoring/node-exporter-ftkv4" Jul 07 14:28:14.786129 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:14.785982 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/b7247b62-46dd-4b19-8afd-52f281ee1f66-node-exporter-tls\") pod \"node-exporter-ftkv4\" (UID: \"b7247b62-46dd-4b19-8afd-52f281ee1f66\") " pod="openshift-monitoring/node-exporter-ftkv4" Jul 07 14:28:14.786129 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:14.786015 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/860b5cea-cdae-499f-b01c-1fdf449ab505-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-54cf96dfcf-hdnqk\" (UID: \"860b5cea-cdae-499f-b01c-1fdf449ab505\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-hdnqk" Jul 07 14:28:14.786129 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:14.786040 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/b7247b62-46dd-4b19-8afd-52f281ee1f66-node-exporter-wtmp\") pod \"node-exporter-ftkv4\" (UID: \"b7247b62-46dd-4b19-8afd-52f281ee1f66\") " pod="openshift-monitoring/node-exporter-ftkv4" Jul 07 14:28:14.786381 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:14.786362 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/860b5cea-cdae-499f-b01c-1fdf449ab505-metrics-client-ca\") pod \"openshift-state-metrics-54cf96dfcf-hdnqk\" (UID: \"860b5cea-cdae-499f-b01c-1fdf449ab505\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-hdnqk" Jul 07 14:28:14.788386 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:14.788363 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/860b5cea-cdae-499f-b01c-1fdf449ab505-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-54cf96dfcf-hdnqk\" (UID: \"860b5cea-cdae-499f-b01c-1fdf449ab505\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-hdnqk" Jul 07 14:28:14.799702 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:14.799684 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nsz6j\" (UniqueName: \"kubernetes.io/projected/860b5cea-cdae-499f-b01c-1fdf449ab505-kube-api-access-nsz6j\") pod \"openshift-state-metrics-54cf96dfcf-hdnqk\" (UID: \"860b5cea-cdae-499f-b01c-1fdf449ab505\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-hdnqk" Jul 07 14:28:14.886837 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:14.886815 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/b7247b62-46dd-4b19-8afd-52f281ee1f66-node-exporter-textfile\") pod \"node-exporter-ftkv4\" (UID: \"b7247b62-46dd-4b19-8afd-52f281ee1f66\") " pod="openshift-monitoring/node-exporter-ftkv4" Jul 07 14:28:14.886924 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:14.886844 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hdwz4\" (UniqueName: \"kubernetes.io/projected/b7247b62-46dd-4b19-8afd-52f281ee1f66-kube-api-access-hdwz4\") pod \"node-exporter-ftkv4\" (UID: \"b7247b62-46dd-4b19-8afd-52f281ee1f66\") " pod="openshift-monitoring/node-exporter-ftkv4" Jul 07 14:28:14.886924 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:14.886866 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/b7247b62-46dd-4b19-8afd-52f281ee1f66-root\") pod \"node-exporter-ftkv4\" (UID: \"b7247b62-46dd-4b19-8afd-52f281ee1f66\") " pod="openshift-monitoring/node-exporter-ftkv4" Jul 07 14:28:14.886924 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:14.886899 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/b7247b62-46dd-4b19-8afd-52f281ee1f66-node-exporter-accelerators-collector-config\") pod \"node-exporter-ftkv4\" (UID: \"b7247b62-46dd-4b19-8afd-52f281ee1f66\") " pod="openshift-monitoring/node-exporter-ftkv4" Jul 07 14:28:14.887115 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:14.886926 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b7247b62-46dd-4b19-8afd-52f281ee1f66-metrics-client-ca\") pod \"node-exporter-ftkv4\" (UID: \"b7247b62-46dd-4b19-8afd-52f281ee1f66\") " pod="openshift-monitoring/node-exporter-ftkv4" Jul 07 14:28:14.887115 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:14.886952 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b7247b62-46dd-4b19-8afd-52f281ee1f66-sys\") pod \"node-exporter-ftkv4\" (UID: \"b7247b62-46dd-4b19-8afd-52f281ee1f66\") " pod="openshift-monitoring/node-exporter-ftkv4" Jul 07 14:28:14.887115 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:14.886964 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/b7247b62-46dd-4b19-8afd-52f281ee1f66-root\") pod \"node-exporter-ftkv4\" (UID: \"b7247b62-46dd-4b19-8afd-52f281ee1f66\") " pod="openshift-monitoring/node-exporter-ftkv4" Jul 07 14:28:14.887115 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:14.886980 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b7247b62-46dd-4b19-8afd-52f281ee1f66-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-ftkv4\" (UID: \"b7247b62-46dd-4b19-8afd-52f281ee1f66\") " pod="openshift-monitoring/node-exporter-ftkv4" Jul 07 14:28:14.887115 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:14.887028 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/b7247b62-46dd-4b19-8afd-52f281ee1f66-node-exporter-tls\") pod \"node-exporter-ftkv4\" (UID: \"b7247b62-46dd-4b19-8afd-52f281ee1f66\") " pod="openshift-monitoring/node-exporter-ftkv4" Jul 07 14:28:14.887115 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:14.887082 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/b7247b62-46dd-4b19-8afd-52f281ee1f66-node-exporter-wtmp\") pod \"node-exporter-ftkv4\" (UID: \"b7247b62-46dd-4b19-8afd-52f281ee1f66\") " pod="openshift-monitoring/node-exporter-ftkv4" Jul 07 14:28:14.887416 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:14.887116 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b7247b62-46dd-4b19-8afd-52f281ee1f66-sys\") pod \"node-exporter-ftkv4\" (UID: \"b7247b62-46dd-4b19-8afd-52f281ee1f66\") " pod="openshift-monitoring/node-exporter-ftkv4" Jul 07 14:28:14.887416 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:14.887180 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/b7247b62-46dd-4b19-8afd-52f281ee1f66-node-exporter-textfile\") pod \"node-exporter-ftkv4\" (UID: \"b7247b62-46dd-4b19-8afd-52f281ee1f66\") " pod="openshift-monitoring/node-exporter-ftkv4" Jul 07 14:28:14.887416 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:14.887318 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/b7247b62-46dd-4b19-8afd-52f281ee1f66-node-exporter-wtmp\") pod \"node-exporter-ftkv4\" (UID: \"b7247b62-46dd-4b19-8afd-52f281ee1f66\") " pod="openshift-monitoring/node-exporter-ftkv4" Jul 07 14:28:14.887563 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:14.887453 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/b7247b62-46dd-4b19-8afd-52f281ee1f66-node-exporter-accelerators-collector-config\") pod \"node-exporter-ftkv4\" (UID: \"b7247b62-46dd-4b19-8afd-52f281ee1f66\") " pod="openshift-monitoring/node-exporter-ftkv4" Jul 07 14:28:14.887742 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:14.887714 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b7247b62-46dd-4b19-8afd-52f281ee1f66-metrics-client-ca\") pod \"node-exporter-ftkv4\" (UID: \"b7247b62-46dd-4b19-8afd-52f281ee1f66\") " pod="openshift-monitoring/node-exporter-ftkv4" Jul 07 14:28:14.889313 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:14.889294 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b7247b62-46dd-4b19-8afd-52f281ee1f66-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-ftkv4\" (UID: \"b7247b62-46dd-4b19-8afd-52f281ee1f66\") " pod="openshift-monitoring/node-exporter-ftkv4" Jul 07 14:28:14.889591 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:14.889571 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/b7247b62-46dd-4b19-8afd-52f281ee1f66-node-exporter-tls\") pod \"node-exporter-ftkv4\" (UID: \"b7247b62-46dd-4b19-8afd-52f281ee1f66\") " pod="openshift-monitoring/node-exporter-ftkv4" Jul 07 14:28:14.897049 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:14.897030 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hdwz4\" (UniqueName: \"kubernetes.io/projected/b7247b62-46dd-4b19-8afd-52f281ee1f66-kube-api-access-hdwz4\") pod \"node-exporter-ftkv4\" (UID: \"b7247b62-46dd-4b19-8afd-52f281ee1f66\") " pod="openshift-monitoring/node-exporter-ftkv4" Jul 07 14:28:14.928113 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:14.928087 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:28:14.999432 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:14.999383 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-ftkv4" Jul 07 14:28:15.012974 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:28:15.012950 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb7247b62_46dd_4b19_8afd_52f281ee1f66.slice/crio-f92720d3e8b222b8b651d7440b381c8a952287fdd104578d2b3a67c38b8102a0 WatchSource:0}: Error finding container f92720d3e8b222b8b651d7440b381c8a952287fdd104578d2b3a67c38b8102a0: Status 404 returned error can't find the container with id f92720d3e8b222b8b651d7440b381c8a952287fdd104578d2b3a67c38b8102a0 Jul 07 14:28:15.025325 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:15.025305 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:28:15.127932 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:15.127903 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:28:15.290826 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:15.290736 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/860b5cea-cdae-499f-b01c-1fdf449ab505-openshift-state-metrics-tls\") pod \"openshift-state-metrics-54cf96dfcf-hdnqk\" (UID: \"860b5cea-cdae-499f-b01c-1fdf449ab505\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-hdnqk" Jul 07 14:28:15.293247 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:15.293220 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/860b5cea-cdae-499f-b01c-1fdf449ab505-openshift-state-metrics-tls\") pod \"openshift-state-metrics-54cf96dfcf-hdnqk\" (UID: \"860b5cea-cdae-499f-b01c-1fdf449ab505\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-hdnqk" Jul 07 14:28:15.443533 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:15.443486 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-ftkv4" event={"ID":"b7247b62-46dd-4b19-8afd-52f281ee1f66","Type":"ContainerStarted","Data":"f92720d3e8b222b8b651d7440b381c8a952287fdd104578d2b3a67c38b8102a0"} Jul 07 14:28:15.499648 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:15.499612 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:28:15.576845 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:15.576821 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-hdnqk" Jul 07 14:28:15.701448 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:15.701369 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:28:15.746556 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:15.746512 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-54cf96dfcf-hdnqk"] Jul 07 14:28:15.750498 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:28:15.750469 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod860b5cea_cdae_499f_b01c_1fdf449ab505.slice/crio-4c94f8e8ee9df2b87916ec4073c92186d5eca335df9125b609e40d05162c27e7 WatchSource:0}: Error finding container 4c94f8e8ee9df2b87916ec4073c92186d5eca335df9125b609e40d05162c27e7: Status 404 returned error can't find the container with id 4c94f8e8ee9df2b87916ec4073c92186d5eca335df9125b609e40d05162c27e7 Jul 07 14:28:15.809555 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:15.809511 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:28:15.923315 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:15.923253 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:28:16.019820 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:16.019794 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:28:16.448911 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:16.448875 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-hdnqk" event={"ID":"860b5cea-cdae-499f-b01c-1fdf449ab505","Type":"ContainerStarted","Data":"98f8910f9aacdafa008da9620764a2835a33bf9796eeb9a4dabab2f44493c760"} Jul 07 14:28:16.449301 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:16.448955 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-hdnqk" event={"ID":"860b5cea-cdae-499f-b01c-1fdf449ab505","Type":"ContainerStarted","Data":"d716ec7af17bce402cb8be3bc78d7d6448083fe631fd6da97d4f9306f8454555"} Jul 07 14:28:16.449301 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:16.448975 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-hdnqk" event={"ID":"860b5cea-cdae-499f-b01c-1fdf449ab505","Type":"ContainerStarted","Data":"4c94f8e8ee9df2b87916ec4073c92186d5eca335df9125b609e40d05162c27e7"} Jul 07 14:28:17.454339 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:17.454306 2557 generic.go:356] "Generic (PLEG): container finished" podID="b7247b62-46dd-4b19-8afd-52f281ee1f66" containerID="39d5746cb4455b19338480afe403247e7730c3b91591b8e74f8c475e45fb4e06" exitCode=0 Jul 07 14:28:17.454783 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:17.454390 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-ftkv4" event={"ID":"b7247b62-46dd-4b19-8afd-52f281ee1f66","Type":"ContainerDied","Data":"39d5746cb4455b19338480afe403247e7730c3b91591b8e74f8c475e45fb4e06"} Jul 07 14:28:17.456603 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:17.456580 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-hdnqk" event={"ID":"860b5cea-cdae-499f-b01c-1fdf449ab505","Type":"ContainerStarted","Data":"e8076e6117b5cc0def03435945d08fbb2f631b69f03ef03bbd61feac962cdd6b"} Jul 07 14:28:17.513938 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:17.513902 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-hdnqk" podStartSLOduration=2.214184665 podStartE2EDuration="3.513887478s" totalImagesPullingTime="1.299702813s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 14:28:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:28:17.512775699 +0000 UTC m=+134.221914078" watchObservedRunningTime="2026-07-07 14:28:17.513887478 +0000 UTC m=+134.223025857" Jul 07 14:28:17.723988 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:17.723956 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:28:17.822128 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:17.822093 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:28:17.922185 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:17.922156 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:28:18.039212 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:18.039178 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-6f79756669-sbqr9"] Jul 07 14:28:18.041253 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:18.041232 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6f79756669-sbqr9" Jul 07 14:28:18.043663 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:18.043641 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 07 14:28:18.043791 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:18.043728 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 07 14:28:18.043791 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:18.043785 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 07 14:28:18.043904 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:18.043857 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-k5x4k\"" Jul 07 14:28:18.044097 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:18.044081 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 07 14:28:18.044160 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:18.044114 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 07 14:28:18.044396 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:18.044384 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 07 14:28:18.044614 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:18.044601 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 07 14:28:18.057679 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:18.057659 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6f79756669-sbqr9"] Jul 07 14:28:18.115393 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:18.115335 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4e5725c5-2d41-4aa0-9270-6131a1365be7-oauth-serving-cert\") pod \"console-6f79756669-sbqr9\" (UID: \"4e5725c5-2d41-4aa0-9270-6131a1365be7\") " pod="openshift-console/console-6f79756669-sbqr9" Jul 07 14:28:18.115393 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:18.115365 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4e5725c5-2d41-4aa0-9270-6131a1365be7-console-config\") pod \"console-6f79756669-sbqr9\" (UID: \"4e5725c5-2d41-4aa0-9270-6131a1365be7\") " pod="openshift-console/console-6f79756669-sbqr9" Jul 07 14:28:18.115393 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:18.115387 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4e5725c5-2d41-4aa0-9270-6131a1365be7-console-serving-cert\") pod \"console-6f79756669-sbqr9\" (UID: \"4e5725c5-2d41-4aa0-9270-6131a1365be7\") " pod="openshift-console/console-6f79756669-sbqr9" Jul 07 14:28:18.115612 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:18.115402 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-snx4z\" (UniqueName: \"kubernetes.io/projected/4e5725c5-2d41-4aa0-9270-6131a1365be7-kube-api-access-snx4z\") pod \"console-6f79756669-sbqr9\" (UID: \"4e5725c5-2d41-4aa0-9270-6131a1365be7\") " pod="openshift-console/console-6f79756669-sbqr9" Jul 07 14:28:18.115612 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:18.115466 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4e5725c5-2d41-4aa0-9270-6131a1365be7-console-oauth-config\") pod \"console-6f79756669-sbqr9\" (UID: \"4e5725c5-2d41-4aa0-9270-6131a1365be7\") " pod="openshift-console/console-6f79756669-sbqr9" Jul 07 14:28:18.115612 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:18.115484 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4e5725c5-2d41-4aa0-9270-6131a1365be7-service-ca\") pod \"console-6f79756669-sbqr9\" (UID: \"4e5725c5-2d41-4aa0-9270-6131a1365be7\") " pod="openshift-console/console-6f79756669-sbqr9" Jul 07 14:28:18.216255 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:18.216222 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4e5725c5-2d41-4aa0-9270-6131a1365be7-oauth-serving-cert\") pod \"console-6f79756669-sbqr9\" (UID: \"4e5725c5-2d41-4aa0-9270-6131a1365be7\") " pod="openshift-console/console-6f79756669-sbqr9" Jul 07 14:28:18.216255 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:18.216258 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4e5725c5-2d41-4aa0-9270-6131a1365be7-console-config\") pod \"console-6f79756669-sbqr9\" (UID: \"4e5725c5-2d41-4aa0-9270-6131a1365be7\") " pod="openshift-console/console-6f79756669-sbqr9" Jul 07 14:28:18.216492 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:18.216284 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4e5725c5-2d41-4aa0-9270-6131a1365be7-console-serving-cert\") pod \"console-6f79756669-sbqr9\" (UID: \"4e5725c5-2d41-4aa0-9270-6131a1365be7\") " pod="openshift-console/console-6f79756669-sbqr9" Jul 07 14:28:18.216492 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:18.216326 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-snx4z\" (UniqueName: \"kubernetes.io/projected/4e5725c5-2d41-4aa0-9270-6131a1365be7-kube-api-access-snx4z\") pod \"console-6f79756669-sbqr9\" (UID: \"4e5725c5-2d41-4aa0-9270-6131a1365be7\") " pod="openshift-console/console-6f79756669-sbqr9" Jul 07 14:28:18.216492 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:18.216351 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4e5725c5-2d41-4aa0-9270-6131a1365be7-console-oauth-config\") pod \"console-6f79756669-sbqr9\" (UID: \"4e5725c5-2d41-4aa0-9270-6131a1365be7\") " pod="openshift-console/console-6f79756669-sbqr9" Jul 07 14:28:18.216492 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:18.216374 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4e5725c5-2d41-4aa0-9270-6131a1365be7-service-ca\") pod \"console-6f79756669-sbqr9\" (UID: \"4e5725c5-2d41-4aa0-9270-6131a1365be7\") " pod="openshift-console/console-6f79756669-sbqr9" Jul 07 14:28:18.216996 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:18.216974 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4e5725c5-2d41-4aa0-9270-6131a1365be7-oauth-serving-cert\") pod \"console-6f79756669-sbqr9\" (UID: \"4e5725c5-2d41-4aa0-9270-6131a1365be7\") " pod="openshift-console/console-6f79756669-sbqr9" Jul 07 14:28:18.217127 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:18.217105 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4e5725c5-2d41-4aa0-9270-6131a1365be7-service-ca\") pod \"console-6f79756669-sbqr9\" (UID: \"4e5725c5-2d41-4aa0-9270-6131a1365be7\") " pod="openshift-console/console-6f79756669-sbqr9" Jul 07 14:28:18.217207 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:18.217192 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4e5725c5-2d41-4aa0-9270-6131a1365be7-console-config\") pod \"console-6f79756669-sbqr9\" (UID: \"4e5725c5-2d41-4aa0-9270-6131a1365be7\") " pod="openshift-console/console-6f79756669-sbqr9" Jul 07 14:28:18.218714 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:18.218689 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4e5725c5-2d41-4aa0-9270-6131a1365be7-console-serving-cert\") pod \"console-6f79756669-sbqr9\" (UID: \"4e5725c5-2d41-4aa0-9270-6131a1365be7\") " pod="openshift-console/console-6f79756669-sbqr9" Jul 07 14:28:18.218817 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:18.218729 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4e5725c5-2d41-4aa0-9270-6131a1365be7-console-oauth-config\") pod \"console-6f79756669-sbqr9\" (UID: \"4e5725c5-2d41-4aa0-9270-6131a1365be7\") " pod="openshift-console/console-6f79756669-sbqr9" Jul 07 14:28:18.224420 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:18.224395 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-snx4z\" (UniqueName: \"kubernetes.io/projected/4e5725c5-2d41-4aa0-9270-6131a1365be7-kube-api-access-snx4z\") pod \"console-6f79756669-sbqr9\" (UID: \"4e5725c5-2d41-4aa0-9270-6131a1365be7\") " pod="openshift-console/console-6f79756669-sbqr9" Jul 07 14:28:18.244851 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:18.244825 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:28:18.350154 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:18.350124 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6f79756669-sbqr9" Jul 07 14:28:18.462055 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:18.462024 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-ftkv4" event={"ID":"b7247b62-46dd-4b19-8afd-52f281ee1f66","Type":"ContainerStarted","Data":"c1211b006e93204008b6349a9619c2af8bfddbb1ec6f556aab6a557a90ac7a61"} Jul 07 14:28:18.462364 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:18.462081 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-ftkv4" event={"ID":"b7247b62-46dd-4b19-8afd-52f281ee1f66","Type":"ContainerStarted","Data":"b403e5be48b1ada65b221a9b00545e28b938287408b2a57a9f2e0b30eca9ea15"} Jul 07 14:28:18.473083 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:18.473057 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6f79756669-sbqr9"] Jul 07 14:28:18.477621 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:28:18.477597 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4e5725c5_2d41_4aa0_9270_6131a1365be7.slice/crio-44c46224ddc5575ccfc04a0570c6036b68409fa254deba7a0aa49268433eeb9e WatchSource:0}: Error finding container 44c46224ddc5575ccfc04a0570c6036b68409fa254deba7a0aa49268433eeb9e: Status 404 returned error can't find the container with id 44c46224ddc5575ccfc04a0570c6036b68409fa254deba7a0aa49268433eeb9e Jul 07 14:28:18.491858 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:18.491820 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-ftkv4" podStartSLOduration=2.868390445 podStartE2EDuration="4.491805195s" totalImagesPullingTime="1.545300464s" totalInitContainerRuntime="78.114286ms" isStatefulPod=true podCreationTimestamp="2026-07-07 14:28:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:28:18.489999772 +0000 UTC m=+135.199138151" watchObservedRunningTime="2026-07-07 14:28:18.491805195 +0000 UTC m=+135.200943574" Jul 07 14:28:18.631950 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:18.631922 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:28:19.444165 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:19.444128 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:28:19.467832 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:19.467799 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6f79756669-sbqr9" event={"ID":"4e5725c5-2d41-4aa0-9270-6131a1365be7","Type":"ContainerStarted","Data":"44c46224ddc5575ccfc04a0570c6036b68409fa254deba7a0aa49268433eeb9e"} Jul 07 14:28:19.544618 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:19.544586 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:28:19.910886 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:19.910848 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:28:20.965909 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:20.965853 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:28:21.327047 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:21.327022 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:28:21.428289 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:21.428259 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:28:21.476993 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:21.476965 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6f79756669-sbqr9" event={"ID":"4e5725c5-2d41-4aa0-9270-6131a1365be7","Type":"ContainerStarted","Data":"4cb76d7736e6511d4a714df5f2b2bd769d2ee0d545c6414b93f7581612275c64"} Jul 07 14:28:21.478277 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:21.478256 2557 generic.go:356] "Generic (PLEG): container finished" podID="c7028698-fc37-401b-966d-359bad5c835e" containerID="b01f476734c7fb166c440a9fff7d5898cce48b9d380b7424c1626df4669f7e41" exitCode=0 Jul 07 14:28:21.478364 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:21.478312 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5c847dcfc4-xpdmb" event={"ID":"c7028698-fc37-401b-966d-359bad5c835e","Type":"ContainerDied","Data":"b01f476734c7fb166c440a9fff7d5898cce48b9d380b7424c1626df4669f7e41"} Jul 07 14:28:21.478600 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:21.478587 2557 scope.go:117] "RemoveContainer" containerID="b01f476734c7fb166c440a9fff7d5898cce48b9d380b7424c1626df4669f7e41" Jul 07 14:28:21.535591 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:21.535484 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-6f79756669-sbqr9" podStartSLOduration=0.769682975 podStartE2EDuration="3.535468734s" totalImagesPullingTime="2.765785759s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 14:28:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:28:21.533598043 +0000 UTC m=+138.242736423" watchObservedRunningTime="2026-07-07 14:28:21.535468734 +0000 UTC m=+138.244607113" Jul 07 14:28:22.239863 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:22.239816 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:28:22.484256 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:22.484217 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5c847dcfc4-xpdmb" event={"ID":"c7028698-fc37-401b-966d-359bad5c835e","Type":"ContainerStarted","Data":"887bb5832711cada6efa8bc1cf75daaa75e54840612d294f00f7d57d9c87b702"} Jul 07 14:28:23.233780 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:23.233742 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:28:24.709222 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:24.709188 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-64786d9cbb-bkdv4_d75c5911-092c-408a-83a7-871dfb379d28/cluster-monitoring-operator/0.log" Jul 07 14:28:25.906957 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:25.906930 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-ftkv4_b7247b62-46dd-4b19-8afd-52f281ee1f66/init-textfile/0.log" Jul 07 14:28:26.107755 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:26.107722 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-ftkv4_b7247b62-46dd-4b19-8afd-52f281ee1f66/node-exporter/0.log" Jul 07 14:28:26.306574 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:26.306487 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-ftkv4_b7247b62-46dd-4b19-8afd-52f281ee1f66/kube-rbac-proxy/0.log" Jul 07 14:28:27.503932 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:27.503903 2557 generic.go:356] "Generic (PLEG): container finished" podID="cc5feae9-bad0-4740-9bb5-db0bec7c280f" containerID="aee205755a35946847a34a95ccc0016c91662b053d4d4f89f5c01e56e59f8f6d" exitCode=0 Jul 07 14:28:27.504271 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:27.503978 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-6mwgs" event={"ID":"cc5feae9-bad0-4740-9bb5-db0bec7c280f","Type":"ContainerDied","Data":"aee205755a35946847a34a95ccc0016c91662b053d4d4f89f5c01e56e59f8f6d"} Jul 07 14:28:27.504311 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:27.504275 2557 scope.go:117] "RemoveContainer" containerID="aee205755a35946847a34a95ccc0016c91662b053d4d4f89f5c01e56e59f8f6d" Jul 07 14:28:27.708929 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:27.708901 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-54cf96dfcf-hdnqk_860b5cea-cdae-499f-b01c-1fdf449ab505/kube-rbac-proxy-main/0.log" Jul 07 14:28:27.906827 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:27.906800 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-54cf96dfcf-hdnqk_860b5cea-cdae-499f-b01c-1fdf449ab505/kube-rbac-proxy-self/0.log" Jul 07 14:28:28.108113 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:28.108086 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-54cf96dfcf-hdnqk_860b5cea-cdae-499f-b01c-1fdf449ab505/openshift-state-metrics/0.log" Jul 07 14:28:28.213290 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:28.213218 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:28:28.233355 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:28.233327 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-6fbf9dbdc9-b45gx"] Jul 07 14:28:28.236460 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:28.236445 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6fbf9dbdc9-b45gx" Jul 07 14:28:28.248897 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:28.248877 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 07 14:28:28.249647 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:28.249628 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6fbf9dbdc9-b45gx"] Jul 07 14:28:28.296271 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:28.296248 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d58a9997-5009-4986-ac9a-5313ded08577-console-config\") pod \"console-6fbf9dbdc9-b45gx\" (UID: \"d58a9997-5009-4986-ac9a-5313ded08577\") " pod="openshift-console/console-6fbf9dbdc9-b45gx" Jul 07 14:28:28.296369 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:28.296283 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d58a9997-5009-4986-ac9a-5313ded08577-oauth-serving-cert\") pod \"console-6fbf9dbdc9-b45gx\" (UID: \"d58a9997-5009-4986-ac9a-5313ded08577\") " pod="openshift-console/console-6fbf9dbdc9-b45gx" Jul 07 14:28:28.296369 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:28.296315 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d58a9997-5009-4986-ac9a-5313ded08577-service-ca\") pod \"console-6fbf9dbdc9-b45gx\" (UID: \"d58a9997-5009-4986-ac9a-5313ded08577\") " pod="openshift-console/console-6fbf9dbdc9-b45gx" Jul 07 14:28:28.296369 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:28.296348 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5gqlc\" (UniqueName: \"kubernetes.io/projected/d58a9997-5009-4986-ac9a-5313ded08577-kube-api-access-5gqlc\") pod \"console-6fbf9dbdc9-b45gx\" (UID: \"d58a9997-5009-4986-ac9a-5313ded08577\") " pod="openshift-console/console-6fbf9dbdc9-b45gx" Jul 07 14:28:28.296467 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:28.296376 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d58a9997-5009-4986-ac9a-5313ded08577-console-oauth-config\") pod \"console-6fbf9dbdc9-b45gx\" (UID: \"d58a9997-5009-4986-ac9a-5313ded08577\") " pod="openshift-console/console-6fbf9dbdc9-b45gx" Jul 07 14:28:28.296467 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:28.296433 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d58a9997-5009-4986-ac9a-5313ded08577-console-serving-cert\") pod \"console-6fbf9dbdc9-b45gx\" (UID: \"d58a9997-5009-4986-ac9a-5313ded08577\") " pod="openshift-console/console-6fbf9dbdc9-b45gx" Jul 07 14:28:28.296552 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:28.296479 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d58a9997-5009-4986-ac9a-5313ded08577-trusted-ca-bundle\") pod \"console-6fbf9dbdc9-b45gx\" (UID: \"d58a9997-5009-4986-ac9a-5313ded08577\") " pod="openshift-console/console-6fbf9dbdc9-b45gx" Jul 07 14:28:28.351231 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:28.351211 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6f79756669-sbqr9" Jul 07 14:28:28.351303 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:28.351254 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-6f79756669-sbqr9" Jul 07 14:28:28.355657 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:28.355639 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-6f79756669-sbqr9" Jul 07 14:28:28.396971 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:28.396945 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d58a9997-5009-4986-ac9a-5313ded08577-oauth-serving-cert\") pod \"console-6fbf9dbdc9-b45gx\" (UID: \"d58a9997-5009-4986-ac9a-5313ded08577\") " pod="openshift-console/console-6fbf9dbdc9-b45gx" Jul 07 14:28:28.397051 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:28.396985 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d58a9997-5009-4986-ac9a-5313ded08577-service-ca\") pod \"console-6fbf9dbdc9-b45gx\" (UID: \"d58a9997-5009-4986-ac9a-5313ded08577\") " pod="openshift-console/console-6fbf9dbdc9-b45gx" Jul 07 14:28:28.397051 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:28.397002 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5gqlc\" (UniqueName: \"kubernetes.io/projected/d58a9997-5009-4986-ac9a-5313ded08577-kube-api-access-5gqlc\") pod \"console-6fbf9dbdc9-b45gx\" (UID: \"d58a9997-5009-4986-ac9a-5313ded08577\") " pod="openshift-console/console-6fbf9dbdc9-b45gx" Jul 07 14:28:28.397132 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:28.397120 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d58a9997-5009-4986-ac9a-5313ded08577-console-oauth-config\") pod \"console-6fbf9dbdc9-b45gx\" (UID: \"d58a9997-5009-4986-ac9a-5313ded08577\") " pod="openshift-console/console-6fbf9dbdc9-b45gx" Jul 07 14:28:28.397176 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:28.397164 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d58a9997-5009-4986-ac9a-5313ded08577-console-serving-cert\") pod \"console-6fbf9dbdc9-b45gx\" (UID: \"d58a9997-5009-4986-ac9a-5313ded08577\") " pod="openshift-console/console-6fbf9dbdc9-b45gx" Jul 07 14:28:28.397224 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:28.397204 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d58a9997-5009-4986-ac9a-5313ded08577-trusted-ca-bundle\") pod \"console-6fbf9dbdc9-b45gx\" (UID: \"d58a9997-5009-4986-ac9a-5313ded08577\") " pod="openshift-console/console-6fbf9dbdc9-b45gx" Jul 07 14:28:28.397308 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:28.397289 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d58a9997-5009-4986-ac9a-5313ded08577-console-config\") pod \"console-6fbf9dbdc9-b45gx\" (UID: \"d58a9997-5009-4986-ac9a-5313ded08577\") " pod="openshift-console/console-6fbf9dbdc9-b45gx" Jul 07 14:28:28.397829 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:28.397805 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d58a9997-5009-4986-ac9a-5313ded08577-oauth-serving-cert\") pod \"console-6fbf9dbdc9-b45gx\" (UID: \"d58a9997-5009-4986-ac9a-5313ded08577\") " pod="openshift-console/console-6fbf9dbdc9-b45gx" Jul 07 14:28:28.397921 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:28.397812 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d58a9997-5009-4986-ac9a-5313ded08577-service-ca\") pod \"console-6fbf9dbdc9-b45gx\" (UID: \"d58a9997-5009-4986-ac9a-5313ded08577\") " pod="openshift-console/console-6fbf9dbdc9-b45gx" Jul 07 14:28:28.398370 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:28.398344 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d58a9997-5009-4986-ac9a-5313ded08577-trusted-ca-bundle\") pod \"console-6fbf9dbdc9-b45gx\" (UID: \"d58a9997-5009-4986-ac9a-5313ded08577\") " pod="openshift-console/console-6fbf9dbdc9-b45gx" Jul 07 14:28:28.398443 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:28.398424 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d58a9997-5009-4986-ac9a-5313ded08577-console-config\") pod \"console-6fbf9dbdc9-b45gx\" (UID: \"d58a9997-5009-4986-ac9a-5313ded08577\") " pod="openshift-console/console-6fbf9dbdc9-b45gx" Jul 07 14:28:28.399658 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:28.399641 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d58a9997-5009-4986-ac9a-5313ded08577-console-oauth-config\") pod \"console-6fbf9dbdc9-b45gx\" (UID: \"d58a9997-5009-4986-ac9a-5313ded08577\") " pod="openshift-console/console-6fbf9dbdc9-b45gx" Jul 07 14:28:28.399787 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:28.399770 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d58a9997-5009-4986-ac9a-5313ded08577-console-serving-cert\") pod \"console-6fbf9dbdc9-b45gx\" (UID: \"d58a9997-5009-4986-ac9a-5313ded08577\") " pod="openshift-console/console-6fbf9dbdc9-b45gx" Jul 07 14:28:28.404476 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:28.404456 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5gqlc\" (UniqueName: \"kubernetes.io/projected/d58a9997-5009-4986-ac9a-5313ded08577-kube-api-access-5gqlc\") pod \"console-6fbf9dbdc9-b45gx\" (UID: \"d58a9997-5009-4986-ac9a-5313ded08577\") " pod="openshift-console/console-6fbf9dbdc9-b45gx" Jul 07 14:28:28.509973 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:28.509906 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-6mwgs" event={"ID":"cc5feae9-bad0-4740-9bb5-db0bec7c280f","Type":"ContainerStarted","Data":"3336b232c851c37bcd4ad8ba367d9c27561ed70a5f78b29fb4a79ab607b5d53f"} Jul 07 14:28:28.513605 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:28.513585 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-6f79756669-sbqr9" Jul 07 14:28:28.547258 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:28.547239 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6fbf9dbdc9-b45gx" Jul 07 14:28:28.548864 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:28.548816 2557 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-image-registry/image-registry-687956bbfb-m9bjn" podUID="648e2b2f-08a5-4a83-b9aa-319ab2527ae4" containerName="registry" containerID="cri-o://28b8972dd748c52490c47687d88cfec1ba1c791207ba74bc8bb05ba48af16d27" gracePeriod=30 Jul 07 14:28:28.675966 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:28.675901 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6fbf9dbdc9-b45gx"] Jul 07 14:28:28.679634 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:28:28.679596 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd58a9997_5009_4986_ac9a_5313ded08577.slice/crio-b6c381957a31614fc37c308ddc29b3f345bac69593f47aa51c23712b1263a45f WatchSource:0}: Error finding container b6c381957a31614fc37c308ddc29b3f345bac69593f47aa51c23712b1263a45f: Status 404 returned error can't find the container with id b6c381957a31614fc37c308ddc29b3f345bac69593f47aa51c23712b1263a45f Jul 07 14:28:28.778689 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:28.778664 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-687956bbfb-m9bjn" Jul 07 14:28:28.901861 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:28.901833 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/648e2b2f-08a5-4a83-b9aa-319ab2527ae4-image-registry-private-configuration\") pod \"648e2b2f-08a5-4a83-b9aa-319ab2527ae4\" (UID: \"648e2b2f-08a5-4a83-b9aa-319ab2527ae4\") " Jul 07 14:28:28.901861 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:28.901865 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/648e2b2f-08a5-4a83-b9aa-319ab2527ae4-installation-pull-secrets\") pod \"648e2b2f-08a5-4a83-b9aa-319ab2527ae4\" (UID: \"648e2b2f-08a5-4a83-b9aa-319ab2527ae4\") " Jul 07 14:28:28.902075 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:28.901898 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/648e2b2f-08a5-4a83-b9aa-319ab2527ae4-ca-trust-extracted\") pod \"648e2b2f-08a5-4a83-b9aa-319ab2527ae4\" (UID: \"648e2b2f-08a5-4a83-b9aa-319ab2527ae4\") " Jul 07 14:28:28.902075 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:28.901915 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/648e2b2f-08a5-4a83-b9aa-319ab2527ae4-trusted-ca\") pod \"648e2b2f-08a5-4a83-b9aa-319ab2527ae4\" (UID: \"648e2b2f-08a5-4a83-b9aa-319ab2527ae4\") " Jul 07 14:28:28.902075 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:28.902050 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/648e2b2f-08a5-4a83-b9aa-319ab2527ae4-registry-tls\") pod \"648e2b2f-08a5-4a83-b9aa-319ab2527ae4\" (UID: \"648e2b2f-08a5-4a83-b9aa-319ab2527ae4\") " Jul 07 14:28:28.902231 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:28.902092 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/648e2b2f-08a5-4a83-b9aa-319ab2527ae4-bound-sa-token\") pod \"648e2b2f-08a5-4a83-b9aa-319ab2527ae4\" (UID: \"648e2b2f-08a5-4a83-b9aa-319ab2527ae4\") " Jul 07 14:28:28.902231 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:28.902132 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h4tnw\" (UniqueName: \"kubernetes.io/projected/648e2b2f-08a5-4a83-b9aa-319ab2527ae4-kube-api-access-h4tnw\") pod \"648e2b2f-08a5-4a83-b9aa-319ab2527ae4\" (UID: \"648e2b2f-08a5-4a83-b9aa-319ab2527ae4\") " Jul 07 14:28:28.902231 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:28.902182 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/648e2b2f-08a5-4a83-b9aa-319ab2527ae4-registry-certificates\") pod \"648e2b2f-08a5-4a83-b9aa-319ab2527ae4\" (UID: \"648e2b2f-08a5-4a83-b9aa-319ab2527ae4\") " Jul 07 14:28:28.902429 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:28.902342 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/648e2b2f-08a5-4a83-b9aa-319ab2527ae4-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "648e2b2f-08a5-4a83-b9aa-319ab2527ae4" (UID: "648e2b2f-08a5-4a83-b9aa-319ab2527ae4"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 14:28:28.902498 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:28.902474 2557 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/648e2b2f-08a5-4a83-b9aa-319ab2527ae4-trusted-ca\") on node \"ip-10-0-134-42.ec2.internal\" DevicePath \"\"" Jul 07 14:28:28.902814 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:28.902765 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/648e2b2f-08a5-4a83-b9aa-319ab2527ae4-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "648e2b2f-08a5-4a83-b9aa-319ab2527ae4" (UID: "648e2b2f-08a5-4a83-b9aa-319ab2527ae4"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 14:28:28.904457 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:28.904429 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/648e2b2f-08a5-4a83-b9aa-319ab2527ae4-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "648e2b2f-08a5-4a83-b9aa-319ab2527ae4" (UID: "648e2b2f-08a5-4a83-b9aa-319ab2527ae4"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 14:28:28.904567 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:28.904428 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/648e2b2f-08a5-4a83-b9aa-319ab2527ae4-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "648e2b2f-08a5-4a83-b9aa-319ab2527ae4" (UID: "648e2b2f-08a5-4a83-b9aa-319ab2527ae4"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 14:28:28.904567 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:28.904481 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/648e2b2f-08a5-4a83-b9aa-319ab2527ae4-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "648e2b2f-08a5-4a83-b9aa-319ab2527ae4" (UID: "648e2b2f-08a5-4a83-b9aa-319ab2527ae4"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 14:28:28.904649 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:28.904601 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/648e2b2f-08a5-4a83-b9aa-319ab2527ae4-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "648e2b2f-08a5-4a83-b9aa-319ab2527ae4" (UID: "648e2b2f-08a5-4a83-b9aa-319ab2527ae4"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 14:28:28.904728 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:28.904712 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/648e2b2f-08a5-4a83-b9aa-319ab2527ae4-kube-api-access-h4tnw" (OuterVolumeSpecName: "kube-api-access-h4tnw") pod "648e2b2f-08a5-4a83-b9aa-319ab2527ae4" (UID: "648e2b2f-08a5-4a83-b9aa-319ab2527ae4"). InnerVolumeSpecName "kube-api-access-h4tnw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 14:28:28.910400 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:28.910379 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/648e2b2f-08a5-4a83-b9aa-319ab2527ae4-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "648e2b2f-08a5-4a83-b9aa-319ab2527ae4" (UID: "648e2b2f-08a5-4a83-b9aa-319ab2527ae4"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 14:28:29.003415 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:29.003388 2557 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/648e2b2f-08a5-4a83-b9aa-319ab2527ae4-bound-sa-token\") on node \"ip-10-0-134-42.ec2.internal\" DevicePath \"\"" Jul 07 14:28:29.003415 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:29.003413 2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h4tnw\" (UniqueName: \"kubernetes.io/projected/648e2b2f-08a5-4a83-b9aa-319ab2527ae4-kube-api-access-h4tnw\") on node \"ip-10-0-134-42.ec2.internal\" DevicePath \"\"" Jul 07 14:28:29.003584 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:29.003422 2557 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/648e2b2f-08a5-4a83-b9aa-319ab2527ae4-registry-certificates\") on node \"ip-10-0-134-42.ec2.internal\" DevicePath \"\"" Jul 07 14:28:29.003584 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:29.003432 2557 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/648e2b2f-08a5-4a83-b9aa-319ab2527ae4-image-registry-private-configuration\") on node \"ip-10-0-134-42.ec2.internal\" DevicePath \"\"" Jul 07 14:28:29.003584 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:29.003441 2557 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/648e2b2f-08a5-4a83-b9aa-319ab2527ae4-installation-pull-secrets\") on node \"ip-10-0-134-42.ec2.internal\" DevicePath \"\"" Jul 07 14:28:29.003584 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:29.003448 2557 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/648e2b2f-08a5-4a83-b9aa-319ab2527ae4-ca-trust-extracted\") on node \"ip-10-0-134-42.ec2.internal\" DevicePath \"\"" Jul 07 14:28:29.003584 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:29.003456 2557 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/648e2b2f-08a5-4a83-b9aa-319ab2527ae4-registry-tls\") on node \"ip-10-0-134-42.ec2.internal\" DevicePath \"\"" Jul 07 14:28:29.515040 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:29.515006 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6fbf9dbdc9-b45gx" event={"ID":"d58a9997-5009-4986-ac9a-5313ded08577","Type":"ContainerStarted","Data":"992814ccc43322917993fa36ab4dcf9dee2346b8464bb70c311516afcfb58670"} Jul 07 14:28:29.515425 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:29.515061 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6fbf9dbdc9-b45gx" event={"ID":"d58a9997-5009-4986-ac9a-5313ded08577","Type":"ContainerStarted","Data":"b6c381957a31614fc37c308ddc29b3f345bac69593f47aa51c23712b1263a45f"} Jul 07 14:28:29.516195 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:29.516174 2557 generic.go:356] "Generic (PLEG): container finished" podID="648e2b2f-08a5-4a83-b9aa-319ab2527ae4" containerID="28b8972dd748c52490c47687d88cfec1ba1c791207ba74bc8bb05ba48af16d27" exitCode=0 Jul 07 14:28:29.516266 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:29.516211 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-687956bbfb-m9bjn" Jul 07 14:28:29.516322 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:29.516260 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-687956bbfb-m9bjn" event={"ID":"648e2b2f-08a5-4a83-b9aa-319ab2527ae4","Type":"ContainerDied","Data":"28b8972dd748c52490c47687d88cfec1ba1c791207ba74bc8bb05ba48af16d27"} Jul 07 14:28:29.516322 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:29.516312 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-687956bbfb-m9bjn" event={"ID":"648e2b2f-08a5-4a83-b9aa-319ab2527ae4","Type":"ContainerDied","Data":"3a70913a2798e9c42e0fdea44b58daf897ec4385e7134dd72e435a23a7859367"} Jul 07 14:28:29.516421 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:29.516340 2557 scope.go:117] "RemoveContainer" containerID="28b8972dd748c52490c47687d88cfec1ba1c791207ba74bc8bb05ba48af16d27" Jul 07 14:28:29.524544 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:29.524514 2557 scope.go:117] "RemoveContainer" containerID="28b8972dd748c52490c47687d88cfec1ba1c791207ba74bc8bb05ba48af16d27" Jul 07 14:28:29.524789 ip-10-0-134-42 kubenswrapper[2557]: E0707 14:28:29.524771 2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"28b8972dd748c52490c47687d88cfec1ba1c791207ba74bc8bb05ba48af16d27\": container with ID starting with 28b8972dd748c52490c47687d88cfec1ba1c791207ba74bc8bb05ba48af16d27 not found: ID does not exist" containerID="28b8972dd748c52490c47687d88cfec1ba1c791207ba74bc8bb05ba48af16d27" Jul 07 14:28:29.524854 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:29.524800 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"28b8972dd748c52490c47687d88cfec1ba1c791207ba74bc8bb05ba48af16d27"} err="failed to get container status \"28b8972dd748c52490c47687d88cfec1ba1c791207ba74bc8bb05ba48af16d27\": rpc error: code = NotFound desc = could not find container \"28b8972dd748c52490c47687d88cfec1ba1c791207ba74bc8bb05ba48af16d27\": container with ID starting with 28b8972dd748c52490c47687d88cfec1ba1c791207ba74bc8bb05ba48af16d27 not found: ID does not exist" Jul 07 14:28:29.533036 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:29.532999 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-6fbf9dbdc9-b45gx" podStartSLOduration=1.5329868119999999 podStartE2EDuration="1.532986812s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 14:28:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:28:29.53134172 +0000 UTC m=+146.240480099" watchObservedRunningTime="2026-07-07 14:28:29.532986812 +0000 UTC m=+146.242125191" Jul 07 14:28:29.545729 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:29.545708 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-687956bbfb-m9bjn"] Jul 07 14:28:29.550435 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:29.550413 2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-687956bbfb-m9bjn"] Jul 07 14:28:29.834129 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:29.834098 2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="648e2b2f-08a5-4a83-b9aa-319ab2527ae4" path="/var/lib/kubelet/pods/648e2b2f-08a5-4a83-b9aa-319ab2527ae4/volumes" Jul 07 14:28:30.106304 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:30.106231 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-d559d66d5-fpkmh_e3ead245-ab3d-41f4-9e62-c256b1968e9c/prometheus-operator-admission-webhook/0.log" Jul 07 14:28:30.434237 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:30.430482 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:28:30.531163 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:30.531120 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:28:31.506556 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:31.506458 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-6bffdf9778-2hh7m_7a17e389-0667-415a-b38d-216a19842db3/networking-console-plugin/0.log" Jul 07 14:28:31.526024 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:31.525998 2557 generic.go:356] "Generic (PLEG): container finished" podID="71300bd3-684d-46ed-b639-fcbc757ec917" containerID="9862f1deadb52d7d9b217cd19a657d4b7bdc6a08004c46ee07e964278e3b9c46" exitCode=0 Jul 07 14:28:31.526147 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:31.526069 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-xqdc5" event={"ID":"71300bd3-684d-46ed-b639-fcbc757ec917","Type":"ContainerDied","Data":"9862f1deadb52d7d9b217cd19a657d4b7bdc6a08004c46ee07e964278e3b9c46"} Jul 07 14:28:31.526413 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:31.526400 2557 scope.go:117] "RemoveContainer" containerID="9862f1deadb52d7d9b217cd19a657d4b7bdc6a08004c46ee07e964278e3b9c46" Jul 07 14:28:31.710698 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:31.710667 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-vd9g9_0fa80df4-c933-4758-b75f-3f70b7217c0d/console-operator/2.log" Jul 07 14:28:31.909160 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:31.909120 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-vd9g9_0fa80df4-c933-4758-b75f-3f70b7217c0d/console-operator/3.log" Jul 07 14:28:32.106451 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:32.106420 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-6f79756669-sbqr9_4e5725c5-2d41-4aa0-9270-6131a1365be7/console/0.log" Jul 07 14:28:32.507054 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:32.507019 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-7c58ff4df-55gs2_450aeb2e-26e4-4ebb-8006-eb9c43c6e2d1/router/0.log" Jul 07 14:28:32.531565 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:32.531534 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-xqdc5" event={"ID":"71300bd3-684d-46ed-b639-fcbc757ec917","Type":"ContainerStarted","Data":"8e7082b5bb14d2b12cb5c31f6aa1c3363c37275e599752d948753c0dd1fb712b"} Jul 07 14:28:32.640915 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:32.640880 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:28:32.744257 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:32.744223 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:28:32.906154 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:32.906120 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-8dtp8_8c3755dc-b805-4f6a-8c8e-6ec9077a8efb/serve-healthcheck-canary/0.log" Jul 07 14:28:34.152040 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:34.151993 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:28:34.251039 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:34.251007 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:28:36.317025 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:36.316987 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:28:36.377244 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:36.377214 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:28:37.794570 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:37.794508 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:28:37.857743 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:37.857700 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:28:38.547704 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:38.547671 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-6fbf9dbdc9-b45gx" Jul 07 14:28:38.547870 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:38.547734 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6fbf9dbdc9-b45gx" Jul 07 14:28:38.552081 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:38.552050 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-6fbf9dbdc9-b45gx" Jul 07 14:28:38.558470 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:38.558450 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-6fbf9dbdc9-b45gx" Jul 07 14:28:38.628050 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:38.628021 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6f79756669-sbqr9"] Jul 07 14:28:43.227437 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:43.227399 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:28:43.292725 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:43.292693 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:28:47.444194 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:47.444150 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:28:47.515442 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:47.515405 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:28:51.296902 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:51.296871 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:28:51.351711 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:28:51.351681 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:29:03.651340 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:29:03.651267 2557 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-6f79756669-sbqr9" podUID="4e5725c5-2d41-4aa0-9270-6131a1365be7" containerName="console" containerID="cri-o://4cb76d7736e6511d4a714df5f2b2bd769d2ee0d545c6414b93f7581612275c64" gracePeriod=15 Jul 07 14:29:03.891326 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:29:03.891301 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6f79756669-sbqr9" Jul 07 14:29:03.992438 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:29:03.992345 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-snx4z\" (UniqueName: \"kubernetes.io/projected/4e5725c5-2d41-4aa0-9270-6131a1365be7-kube-api-access-snx4z\") pod \"4e5725c5-2d41-4aa0-9270-6131a1365be7\" (UID: \"4e5725c5-2d41-4aa0-9270-6131a1365be7\") " Jul 07 14:29:03.992438 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:29:03.992386 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4e5725c5-2d41-4aa0-9270-6131a1365be7-service-ca\") pod \"4e5725c5-2d41-4aa0-9270-6131a1365be7\" (UID: \"4e5725c5-2d41-4aa0-9270-6131a1365be7\") " Jul 07 14:29:03.992438 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:29:03.992413 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4e5725c5-2d41-4aa0-9270-6131a1365be7-oauth-serving-cert\") pod \"4e5725c5-2d41-4aa0-9270-6131a1365be7\" (UID: \"4e5725c5-2d41-4aa0-9270-6131a1365be7\") " Jul 07 14:29:03.992743 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:29:03.992450 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4e5725c5-2d41-4aa0-9270-6131a1365be7-console-config\") pod \"4e5725c5-2d41-4aa0-9270-6131a1365be7\" (UID: \"4e5725c5-2d41-4aa0-9270-6131a1365be7\") " Jul 07 14:29:03.992743 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:29:03.992499 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4e5725c5-2d41-4aa0-9270-6131a1365be7-console-serving-cert\") pod \"4e5725c5-2d41-4aa0-9270-6131a1365be7\" (UID: \"4e5725c5-2d41-4aa0-9270-6131a1365be7\") " Jul 07 14:29:03.992743 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:29:03.992548 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4e5725c5-2d41-4aa0-9270-6131a1365be7-console-oauth-config\") pod \"4e5725c5-2d41-4aa0-9270-6131a1365be7\" (UID: \"4e5725c5-2d41-4aa0-9270-6131a1365be7\") " Jul 07 14:29:03.992882 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:29:03.992842 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4e5725c5-2d41-4aa0-9270-6131a1365be7-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "4e5725c5-2d41-4aa0-9270-6131a1365be7" (UID: "4e5725c5-2d41-4aa0-9270-6131a1365be7"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 14:29:03.992936 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:29:03.992884 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4e5725c5-2d41-4aa0-9270-6131a1365be7-console-config" (OuterVolumeSpecName: "console-config") pod "4e5725c5-2d41-4aa0-9270-6131a1365be7" (UID: "4e5725c5-2d41-4aa0-9270-6131a1365be7"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 14:29:03.992936 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:29:03.992900 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4e5725c5-2d41-4aa0-9270-6131a1365be7-service-ca" (OuterVolumeSpecName: "service-ca") pod "4e5725c5-2d41-4aa0-9270-6131a1365be7" (UID: "4e5725c5-2d41-4aa0-9270-6131a1365be7"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 14:29:03.994739 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:29:03.994707 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4e5725c5-2d41-4aa0-9270-6131a1365be7-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "4e5725c5-2d41-4aa0-9270-6131a1365be7" (UID: "4e5725c5-2d41-4aa0-9270-6131a1365be7"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 14:29:03.994836 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:29:03.994791 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4e5725c5-2d41-4aa0-9270-6131a1365be7-kube-api-access-snx4z" (OuterVolumeSpecName: "kube-api-access-snx4z") pod "4e5725c5-2d41-4aa0-9270-6131a1365be7" (UID: "4e5725c5-2d41-4aa0-9270-6131a1365be7"). InnerVolumeSpecName "kube-api-access-snx4z". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 14:29:03.994836 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:29:03.994810 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4e5725c5-2d41-4aa0-9270-6131a1365be7-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "4e5725c5-2d41-4aa0-9270-6131a1365be7" (UID: "4e5725c5-2d41-4aa0-9270-6131a1365be7"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 14:29:04.093206 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:29:04.093166 2557 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4e5725c5-2d41-4aa0-9270-6131a1365be7-console-serving-cert\") on node \"ip-10-0-134-42.ec2.internal\" DevicePath \"\"" Jul 07 14:29:04.093206 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:29:04.093198 2557 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4e5725c5-2d41-4aa0-9270-6131a1365be7-console-oauth-config\") on node \"ip-10-0-134-42.ec2.internal\" DevicePath \"\"" Jul 07 14:29:04.093206 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:29:04.093210 2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-snx4z\" (UniqueName: \"kubernetes.io/projected/4e5725c5-2d41-4aa0-9270-6131a1365be7-kube-api-access-snx4z\") on node \"ip-10-0-134-42.ec2.internal\" DevicePath \"\"" Jul 07 14:29:04.093431 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:29:04.093223 2557 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4e5725c5-2d41-4aa0-9270-6131a1365be7-service-ca\") on node \"ip-10-0-134-42.ec2.internal\" DevicePath \"\"" Jul 07 14:29:04.093431 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:29:04.093236 2557 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4e5725c5-2d41-4aa0-9270-6131a1365be7-oauth-serving-cert\") on node \"ip-10-0-134-42.ec2.internal\" DevicePath \"\"" Jul 07 14:29:04.093431 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:29:04.093248 2557 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4e5725c5-2d41-4aa0-9270-6131a1365be7-console-config\") on node \"ip-10-0-134-42.ec2.internal\" DevicePath \"\"" Jul 07 14:29:04.659364 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:29:04.659327 2557 generic.go:356] "Generic (PLEG): container finished" podID="4e5725c5-2d41-4aa0-9270-6131a1365be7" containerID="4cb76d7736e6511d4a714df5f2b2bd769d2ee0d545c6414b93f7581612275c64" exitCode=0 Jul 07 14:29:04.659804 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:29:04.659383 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6f79756669-sbqr9" Jul 07 14:29:04.659804 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:29:04.659408 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6f79756669-sbqr9" event={"ID":"4e5725c5-2d41-4aa0-9270-6131a1365be7","Type":"ContainerDied","Data":"4cb76d7736e6511d4a714df5f2b2bd769d2ee0d545c6414b93f7581612275c64"} Jul 07 14:29:04.659804 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:29:04.659458 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6f79756669-sbqr9" event={"ID":"4e5725c5-2d41-4aa0-9270-6131a1365be7","Type":"ContainerDied","Data":"44c46224ddc5575ccfc04a0570c6036b68409fa254deba7a0aa49268433eeb9e"} Jul 07 14:29:04.659804 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:29:04.659479 2557 scope.go:117] "RemoveContainer" containerID="4cb76d7736e6511d4a714df5f2b2bd769d2ee0d545c6414b93f7581612275c64" Jul 07 14:29:04.668165 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:29:04.668148 2557 scope.go:117] "RemoveContainer" containerID="4cb76d7736e6511d4a714df5f2b2bd769d2ee0d545c6414b93f7581612275c64" Jul 07 14:29:04.668402 ip-10-0-134-42 kubenswrapper[2557]: E0707 14:29:04.668385 2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4cb76d7736e6511d4a714df5f2b2bd769d2ee0d545c6414b93f7581612275c64\": container with ID starting with 4cb76d7736e6511d4a714df5f2b2bd769d2ee0d545c6414b93f7581612275c64 not found: ID does not exist" containerID="4cb76d7736e6511d4a714df5f2b2bd769d2ee0d545c6414b93f7581612275c64" Jul 07 14:29:04.668445 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:29:04.668409 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4cb76d7736e6511d4a714df5f2b2bd769d2ee0d545c6414b93f7581612275c64"} err="failed to get container status \"4cb76d7736e6511d4a714df5f2b2bd769d2ee0d545c6414b93f7581612275c64\": rpc error: code = NotFound desc = could not find container \"4cb76d7736e6511d4a714df5f2b2bd769d2ee0d545c6414b93f7581612275c64\": container with ID starting with 4cb76d7736e6511d4a714df5f2b2bd769d2ee0d545c6414b93f7581612275c64 not found: ID does not exist" Jul 07 14:29:04.677988 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:29:04.677959 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6f79756669-sbqr9"] Jul 07 14:29:04.682542 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:29:04.682505 2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-6f79756669-sbqr9"] Jul 07 14:29:05.833820 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:29:05.833788 2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4e5725c5-2d41-4aa0-9270-6131a1365be7" path="/var/lib/kubelet/pods/4e5725c5-2d41-4aa0-9270-6131a1365be7/volumes" Jul 07 14:29:10.931281 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:29:10.931244 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:29:33.966196 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:29:33.966157 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:29:34.259633 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:29:34.259546 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:29:39.018528 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:29:39.018479 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-564c4f7c89-g6czx"] Jul 07 14:29:39.018974 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:29:39.018811 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="648e2b2f-08a5-4a83-b9aa-319ab2527ae4" containerName="registry" Jul 07 14:29:39.018974 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:29:39.018823 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="648e2b2f-08a5-4a83-b9aa-319ab2527ae4" containerName="registry" Jul 07 14:29:39.018974 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:29:39.018849 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4e5725c5-2d41-4aa0-9270-6131a1365be7" containerName="console" Jul 07 14:29:39.018974 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:29:39.018854 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="4e5725c5-2d41-4aa0-9270-6131a1365be7" containerName="console" Jul 07 14:29:39.018974 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:29:39.018897 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="648e2b2f-08a5-4a83-b9aa-319ab2527ae4" containerName="registry" Jul 07 14:29:39.018974 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:29:39.018904 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="4e5725c5-2d41-4aa0-9270-6131a1365be7" containerName="console" Jul 07 14:29:39.022109 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:29:39.022085 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-564c4f7c89-g6czx" Jul 07 14:29:39.024397 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:29:39.024375 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\"" Jul 07 14:29:39.024921 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:29:39.024898 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\"" Jul 07 14:29:39.025473 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:29:39.024970 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\"" Jul 07 14:29:39.025473 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:29:39.024980 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-tj7d2\"" Jul 07 14:29:39.025473 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:29:39.025005 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\"" Jul 07 14:29:39.025473 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:29:39.025032 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\"" Jul 07 14:29:39.030951 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:29:39.030596 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\"" Jul 07 14:29:39.038831 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:29:39.038806 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-564c4f7c89-g6czx"] Jul 07 14:29:39.144718 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:29:39.144683 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9d863603-e1b9-49c8-93c0-dbff146ca2d7-serving-certs-ca-bundle\") pod \"telemeter-client-564c4f7c89-g6czx\" (UID: \"9d863603-e1b9-49c8-93c0-dbff146ca2d7\") " pod="openshift-monitoring/telemeter-client-564c4f7c89-g6czx" Jul 07 14:29:39.144892 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:29:39.144745 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9d863603-e1b9-49c8-93c0-dbff146ca2d7-telemeter-trusted-ca-bundle\") pod \"telemeter-client-564c4f7c89-g6czx\" (UID: \"9d863603-e1b9-49c8-93c0-dbff146ca2d7\") " pod="openshift-monitoring/telemeter-client-564c4f7c89-g6czx" Jul 07 14:29:39.144892 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:29:39.144779 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9d863603-e1b9-49c8-93c0-dbff146ca2d7-metrics-client-ca\") pod \"telemeter-client-564c4f7c89-g6czx\" (UID: \"9d863603-e1b9-49c8-93c0-dbff146ca2d7\") " pod="openshift-monitoring/telemeter-client-564c4f7c89-g6czx" Jul 07 14:29:39.144892 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:29:39.144806 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wxrcq\" (UniqueName: \"kubernetes.io/projected/9d863603-e1b9-49c8-93c0-dbff146ca2d7-kube-api-access-wxrcq\") pod \"telemeter-client-564c4f7c89-g6czx\" (UID: \"9d863603-e1b9-49c8-93c0-dbff146ca2d7\") " pod="openshift-monitoring/telemeter-client-564c4f7c89-g6czx" Jul 07 14:29:39.144892 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:29:39.144834 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/9d863603-e1b9-49c8-93c0-dbff146ca2d7-federate-client-tls\") pod \"telemeter-client-564c4f7c89-g6czx\" (UID: \"9d863603-e1b9-49c8-93c0-dbff146ca2d7\") " pod="openshift-monitoring/telemeter-client-564c4f7c89-g6czx" Jul 07 14:29:39.144892 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:29:39.144876 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/9d863603-e1b9-49c8-93c0-dbff146ca2d7-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-564c4f7c89-g6czx\" (UID: \"9d863603-e1b9-49c8-93c0-dbff146ca2d7\") " pod="openshift-monitoring/telemeter-client-564c4f7c89-g6czx" Jul 07 14:29:39.145046 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:29:39.144905 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/9d863603-e1b9-49c8-93c0-dbff146ca2d7-telemeter-client-tls\") pod \"telemeter-client-564c4f7c89-g6czx\" (UID: \"9d863603-e1b9-49c8-93c0-dbff146ca2d7\") " pod="openshift-monitoring/telemeter-client-564c4f7c89-g6czx" Jul 07 14:29:39.145046 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:29:39.144945 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/9d863603-e1b9-49c8-93c0-dbff146ca2d7-secret-telemeter-client\") pod \"telemeter-client-564c4f7c89-g6czx\" (UID: \"9d863603-e1b9-49c8-93c0-dbff146ca2d7\") " pod="openshift-monitoring/telemeter-client-564c4f7c89-g6czx" Jul 07 14:29:39.245689 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:29:39.245646 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9d863603-e1b9-49c8-93c0-dbff146ca2d7-metrics-client-ca\") pod \"telemeter-client-564c4f7c89-g6czx\" (UID: \"9d863603-e1b9-49c8-93c0-dbff146ca2d7\") " pod="openshift-monitoring/telemeter-client-564c4f7c89-g6czx" Jul 07 14:29:39.245689 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:29:39.245696 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wxrcq\" (UniqueName: \"kubernetes.io/projected/9d863603-e1b9-49c8-93c0-dbff146ca2d7-kube-api-access-wxrcq\") pod \"telemeter-client-564c4f7c89-g6czx\" (UID: \"9d863603-e1b9-49c8-93c0-dbff146ca2d7\") " pod="openshift-monitoring/telemeter-client-564c4f7c89-g6czx" Jul 07 14:29:39.245937 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:29:39.245726 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/9d863603-e1b9-49c8-93c0-dbff146ca2d7-federate-client-tls\") pod \"telemeter-client-564c4f7c89-g6czx\" (UID: \"9d863603-e1b9-49c8-93c0-dbff146ca2d7\") " pod="openshift-monitoring/telemeter-client-564c4f7c89-g6czx" Jul 07 14:29:39.245937 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:29:39.245754 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/9d863603-e1b9-49c8-93c0-dbff146ca2d7-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-564c4f7c89-g6czx\" (UID: \"9d863603-e1b9-49c8-93c0-dbff146ca2d7\") " pod="openshift-monitoring/telemeter-client-564c4f7c89-g6czx" Jul 07 14:29:39.245937 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:29:39.245789 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/9d863603-e1b9-49c8-93c0-dbff146ca2d7-telemeter-client-tls\") pod \"telemeter-client-564c4f7c89-g6czx\" (UID: \"9d863603-e1b9-49c8-93c0-dbff146ca2d7\") " pod="openshift-monitoring/telemeter-client-564c4f7c89-g6czx" Jul 07 14:29:39.245937 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:29:39.245825 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/9d863603-e1b9-49c8-93c0-dbff146ca2d7-secret-telemeter-client\") pod \"telemeter-client-564c4f7c89-g6czx\" (UID: \"9d863603-e1b9-49c8-93c0-dbff146ca2d7\") " pod="openshift-monitoring/telemeter-client-564c4f7c89-g6czx" Jul 07 14:29:39.245937 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:29:39.245861 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9d863603-e1b9-49c8-93c0-dbff146ca2d7-serving-certs-ca-bundle\") pod \"telemeter-client-564c4f7c89-g6czx\" (UID: \"9d863603-e1b9-49c8-93c0-dbff146ca2d7\") " pod="openshift-monitoring/telemeter-client-564c4f7c89-g6czx" Jul 07 14:29:39.246185 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:29:39.245933 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9d863603-e1b9-49c8-93c0-dbff146ca2d7-telemeter-trusted-ca-bundle\") pod \"telemeter-client-564c4f7c89-g6czx\" (UID: \"9d863603-e1b9-49c8-93c0-dbff146ca2d7\") " pod="openshift-monitoring/telemeter-client-564c4f7c89-g6czx" Jul 07 14:29:39.246585 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:29:39.246556 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9d863603-e1b9-49c8-93c0-dbff146ca2d7-metrics-client-ca\") pod \"telemeter-client-564c4f7c89-g6czx\" (UID: \"9d863603-e1b9-49c8-93c0-dbff146ca2d7\") " pod="openshift-monitoring/telemeter-client-564c4f7c89-g6czx" Jul 07 14:29:39.246729 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:29:39.246703 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9d863603-e1b9-49c8-93c0-dbff146ca2d7-serving-certs-ca-bundle\") pod \"telemeter-client-564c4f7c89-g6czx\" (UID: \"9d863603-e1b9-49c8-93c0-dbff146ca2d7\") " pod="openshift-monitoring/telemeter-client-564c4f7c89-g6czx" Jul 07 14:29:39.246894 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:29:39.246864 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9d863603-e1b9-49c8-93c0-dbff146ca2d7-telemeter-trusted-ca-bundle\") pod \"telemeter-client-564c4f7c89-g6czx\" (UID: \"9d863603-e1b9-49c8-93c0-dbff146ca2d7\") " pod="openshift-monitoring/telemeter-client-564c4f7c89-g6czx" Jul 07 14:29:39.248340 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:29:39.248315 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/9d863603-e1b9-49c8-93c0-dbff146ca2d7-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-564c4f7c89-g6czx\" (UID: \"9d863603-e1b9-49c8-93c0-dbff146ca2d7\") " pod="openshift-monitoring/telemeter-client-564c4f7c89-g6czx" Jul 07 14:29:39.248479 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:29:39.248460 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/9d863603-e1b9-49c8-93c0-dbff146ca2d7-secret-telemeter-client\") pod \"telemeter-client-564c4f7c89-g6czx\" (UID: \"9d863603-e1b9-49c8-93c0-dbff146ca2d7\") " pod="openshift-monitoring/telemeter-client-564c4f7c89-g6czx" Jul 07 14:29:39.248614 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:29:39.248596 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/9d863603-e1b9-49c8-93c0-dbff146ca2d7-federate-client-tls\") pod \"telemeter-client-564c4f7c89-g6czx\" (UID: \"9d863603-e1b9-49c8-93c0-dbff146ca2d7\") " pod="openshift-monitoring/telemeter-client-564c4f7c89-g6czx" Jul 07 14:29:39.248662 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:29:39.248616 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/9d863603-e1b9-49c8-93c0-dbff146ca2d7-telemeter-client-tls\") pod \"telemeter-client-564c4f7c89-g6czx\" (UID: \"9d863603-e1b9-49c8-93c0-dbff146ca2d7\") " pod="openshift-monitoring/telemeter-client-564c4f7c89-g6czx" Jul 07 14:29:39.254232 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:29:39.254210 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wxrcq\" (UniqueName: \"kubernetes.io/projected/9d863603-e1b9-49c8-93c0-dbff146ca2d7-kube-api-access-wxrcq\") pod \"telemeter-client-564c4f7c89-g6czx\" (UID: \"9d863603-e1b9-49c8-93c0-dbff146ca2d7\") " pod="openshift-monitoring/telemeter-client-564c4f7c89-g6czx" Jul 07 14:29:39.333871 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:29:39.333841 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-564c4f7c89-g6czx" Jul 07 14:29:39.462543 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:29:39.462496 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-564c4f7c89-g6czx"] Jul 07 14:29:39.465968 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:29:39.465941 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9d863603_e1b9_49c8_93c0_dbff146ca2d7.slice/crio-0197e51080861efdc5a8b6aca33992a12a82f6af4a88b14c4e05767dcbcacc29 WatchSource:0}: Error finding container 0197e51080861efdc5a8b6aca33992a12a82f6af4a88b14c4e05767dcbcacc29: Status 404 returned error can't find the container with id 0197e51080861efdc5a8b6aca33992a12a82f6af4a88b14c4e05767dcbcacc29 Jul 07 14:29:39.792129 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:29:39.792049 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-564c4f7c89-g6czx" event={"ID":"9d863603-e1b9-49c8-93c0-dbff146ca2d7","Type":"ContainerStarted","Data":"0197e51080861efdc5a8b6aca33992a12a82f6af4a88b14c4e05767dcbcacc29"} Jul 07 14:29:41.801567 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:29:41.801475 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-564c4f7c89-g6czx" event={"ID":"9d863603-e1b9-49c8-93c0-dbff146ca2d7","Type":"ContainerStarted","Data":"82baa88fe5db4f7bd4a28476dbd684433c691197bbf9a5c11bc969a72c614aaa"} Jul 07 14:29:43.812538 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:29:43.812489 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-564c4f7c89-g6czx" event={"ID":"9d863603-e1b9-49c8-93c0-dbff146ca2d7","Type":"ContainerStarted","Data":"a82b51d39e2c61e351adb6b9319d01501d9afd2b7efb357657df9515eae5895b"} Jul 07 14:29:43.812956 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:29:43.812584 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-564c4f7c89-g6czx" event={"ID":"9d863603-e1b9-49c8-93c0-dbff146ca2d7","Type":"ContainerStarted","Data":"12556cf8e3176b24e14ce397b91a9fc046b2d105097020e23e29069437f0142f"} Jul 07 14:29:43.836355 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:29:43.836305 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-564c4f7c89-g6czx" podStartSLOduration=2.435838676 podStartE2EDuration="5.836288503s" totalImagesPullingTime="3.400449827s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 14:29:38 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:29:43.834383367 +0000 UTC m=+220.543521757" watchObservedRunningTime="2026-07-07 14:29:43.836288503 +0000 UTC m=+220.545426930" Jul 07 14:29:44.642006 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:29:44.641970 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-8649cf795f-kn4kp"] Jul 07 14:29:44.647732 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:29:44.647705 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-8649cf795f-kn4kp" Jul 07 14:29:44.659609 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:29:44.659580 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-8649cf795f-kn4kp"] Jul 07 14:29:44.713833 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:29:44.713804 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:29:44.776835 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:29:44.776805 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:29:44.785831 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:29:44.785809 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/513779ad-5c00-4e1e-82c1-3651231a147f-service-ca\") pod \"console-8649cf795f-kn4kp\" (UID: \"513779ad-5c00-4e1e-82c1-3651231a147f\") " pod="openshift-console/console-8649cf795f-kn4kp" Jul 07 14:29:44.785905 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:29:44.785839 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/513779ad-5c00-4e1e-82c1-3651231a147f-console-config\") pod \"console-8649cf795f-kn4kp\" (UID: \"513779ad-5c00-4e1e-82c1-3651231a147f\") " pod="openshift-console/console-8649cf795f-kn4kp" Jul 07 14:29:44.785905 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:29:44.785870 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zhhkd\" (UniqueName: \"kubernetes.io/projected/513779ad-5c00-4e1e-82c1-3651231a147f-kube-api-access-zhhkd\") pod \"console-8649cf795f-kn4kp\" (UID: \"513779ad-5c00-4e1e-82c1-3651231a147f\") " pod="openshift-console/console-8649cf795f-kn4kp" Jul 07 14:29:44.785905 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:29:44.785897 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/513779ad-5c00-4e1e-82c1-3651231a147f-trusted-ca-bundle\") pod \"console-8649cf795f-kn4kp\" (UID: \"513779ad-5c00-4e1e-82c1-3651231a147f\") " pod="openshift-console/console-8649cf795f-kn4kp" Jul 07 14:29:44.786019 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:29:44.785953 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/513779ad-5c00-4e1e-82c1-3651231a147f-console-oauth-config\") pod \"console-8649cf795f-kn4kp\" (UID: \"513779ad-5c00-4e1e-82c1-3651231a147f\") " pod="openshift-console/console-8649cf795f-kn4kp" Jul 07 14:29:44.786019 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:29:44.785978 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/513779ad-5c00-4e1e-82c1-3651231a147f-oauth-serving-cert\") pod \"console-8649cf795f-kn4kp\" (UID: \"513779ad-5c00-4e1e-82c1-3651231a147f\") " pod="openshift-console/console-8649cf795f-kn4kp" Jul 07 14:29:44.786019 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:29:44.786014 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/513779ad-5c00-4e1e-82c1-3651231a147f-console-serving-cert\") pod \"console-8649cf795f-kn4kp\" (UID: \"513779ad-5c00-4e1e-82c1-3651231a147f\") " pod="openshift-console/console-8649cf795f-kn4kp" Jul 07 14:29:44.886323 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:29:44.886298 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/513779ad-5c00-4e1e-82c1-3651231a147f-console-oauth-config\") pod \"console-8649cf795f-kn4kp\" (UID: \"513779ad-5c00-4e1e-82c1-3651231a147f\") " pod="openshift-console/console-8649cf795f-kn4kp" Jul 07 14:29:44.886323 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:29:44.886325 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/513779ad-5c00-4e1e-82c1-3651231a147f-oauth-serving-cert\") pod \"console-8649cf795f-kn4kp\" (UID: \"513779ad-5c00-4e1e-82c1-3651231a147f\") " pod="openshift-console/console-8649cf795f-kn4kp" Jul 07 14:29:44.886794 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:29:44.886374 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/513779ad-5c00-4e1e-82c1-3651231a147f-console-serving-cert\") pod \"console-8649cf795f-kn4kp\" (UID: \"513779ad-5c00-4e1e-82c1-3651231a147f\") " pod="openshift-console/console-8649cf795f-kn4kp" Jul 07 14:29:44.886794 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:29:44.886627 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/513779ad-5c00-4e1e-82c1-3651231a147f-service-ca\") pod \"console-8649cf795f-kn4kp\" (UID: \"513779ad-5c00-4e1e-82c1-3651231a147f\") " pod="openshift-console/console-8649cf795f-kn4kp" Jul 07 14:29:44.886794 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:29:44.886715 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/513779ad-5c00-4e1e-82c1-3651231a147f-console-config\") pod \"console-8649cf795f-kn4kp\" (UID: \"513779ad-5c00-4e1e-82c1-3651231a147f\") " pod="openshift-console/console-8649cf795f-kn4kp" Jul 07 14:29:44.886948 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:29:44.886794 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zhhkd\" (UniqueName: \"kubernetes.io/projected/513779ad-5c00-4e1e-82c1-3651231a147f-kube-api-access-zhhkd\") pod \"console-8649cf795f-kn4kp\" (UID: \"513779ad-5c00-4e1e-82c1-3651231a147f\") " pod="openshift-console/console-8649cf795f-kn4kp" Jul 07 14:29:44.886948 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:29:44.886828 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/513779ad-5c00-4e1e-82c1-3651231a147f-trusted-ca-bundle\") pod \"console-8649cf795f-kn4kp\" (UID: \"513779ad-5c00-4e1e-82c1-3651231a147f\") " pod="openshift-console/console-8649cf795f-kn4kp" Jul 07 14:29:44.887165 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:29:44.887135 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/513779ad-5c00-4e1e-82c1-3651231a147f-oauth-serving-cert\") pod \"console-8649cf795f-kn4kp\" (UID: \"513779ad-5c00-4e1e-82c1-3651231a147f\") " pod="openshift-console/console-8649cf795f-kn4kp" Jul 07 14:29:44.887451 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:29:44.887424 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/513779ad-5c00-4e1e-82c1-3651231a147f-console-config\") pod \"console-8649cf795f-kn4kp\" (UID: \"513779ad-5c00-4e1e-82c1-3651231a147f\") " pod="openshift-console/console-8649cf795f-kn4kp" Jul 07 14:29:44.887775 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:29:44.887754 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/513779ad-5c00-4e1e-82c1-3651231a147f-service-ca\") pod \"console-8649cf795f-kn4kp\" (UID: \"513779ad-5c00-4e1e-82c1-3651231a147f\") " pod="openshift-console/console-8649cf795f-kn4kp" Jul 07 14:29:44.887821 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:29:44.887763 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/513779ad-5c00-4e1e-82c1-3651231a147f-trusted-ca-bundle\") pod \"console-8649cf795f-kn4kp\" (UID: \"513779ad-5c00-4e1e-82c1-3651231a147f\") " pod="openshift-console/console-8649cf795f-kn4kp" Jul 07 14:29:44.889020 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:29:44.888997 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/513779ad-5c00-4e1e-82c1-3651231a147f-console-oauth-config\") pod \"console-8649cf795f-kn4kp\" (UID: \"513779ad-5c00-4e1e-82c1-3651231a147f\") " pod="openshift-console/console-8649cf795f-kn4kp" Jul 07 14:29:44.889097 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:29:44.889065 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/513779ad-5c00-4e1e-82c1-3651231a147f-console-serving-cert\") pod \"console-8649cf795f-kn4kp\" (UID: \"513779ad-5c00-4e1e-82c1-3651231a147f\") " pod="openshift-console/console-8649cf795f-kn4kp" Jul 07 14:29:44.898949 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:29:44.898901 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zhhkd\" (UniqueName: \"kubernetes.io/projected/513779ad-5c00-4e1e-82c1-3651231a147f-kube-api-access-zhhkd\") pod \"console-8649cf795f-kn4kp\" (UID: \"513779ad-5c00-4e1e-82c1-3651231a147f\") " pod="openshift-console/console-8649cf795f-kn4kp" Jul 07 14:29:44.930274 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:29:44.930244 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:29:44.958189 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:29:44.958168 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-8649cf795f-kn4kp" Jul 07 14:29:44.992195 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:29:44.992173 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:29:45.092115 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:29:45.092084 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-8649cf795f-kn4kp"] Jul 07 14:29:45.095457 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:29:45.095425 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod513779ad_5c00_4e1e_82c1_3651231a147f.slice/crio-32e805d826aa94bd3fbdb462fe44abd0f45a7c379bfe503af7ab99cd4ec3c1d5 WatchSource:0}: Error finding container 32e805d826aa94bd3fbdb462fe44abd0f45a7c379bfe503af7ab99cd4ec3c1d5: Status 404 returned error can't find the container with id 32e805d826aa94bd3fbdb462fe44abd0f45a7c379bfe503af7ab99cd4ec3c1d5 Jul 07 14:29:45.451655 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:29:45.451617 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:29:45.519247 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:29:45.519210 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:29:45.822073 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:29:45.822037 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-8649cf795f-kn4kp" event={"ID":"513779ad-5c00-4e1e-82c1-3651231a147f","Type":"ContainerStarted","Data":"caf3a76b9460c83d538835d579e74c0dda5a95e10ff4adc2c5deba9acc4b8150"} Jul 07 14:29:45.822242 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:29:45.822093 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-8649cf795f-kn4kp" event={"ID":"513779ad-5c00-4e1e-82c1-3651231a147f","Type":"ContainerStarted","Data":"32e805d826aa94bd3fbdb462fe44abd0f45a7c379bfe503af7ab99cd4ec3c1d5"} Jul 07 14:29:45.839428 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:29:45.839380 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-8649cf795f-kn4kp" podStartSLOduration=1.8393645730000001 podStartE2EDuration="1.839364573s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 14:29:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:29:45.838463635 +0000 UTC m=+222.547602037" watchObservedRunningTime="2026-07-07 14:29:45.839364573 +0000 UTC m=+222.548502953" Jul 07 14:29:52.512377 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:29:52.512340 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:29:52.566787 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:29:52.566750 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:29:54.633230 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:29:54.633189 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:29:54.863128 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:29:54.863091 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:29:54.958787 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:29:54.958701 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-8649cf795f-kn4kp" Jul 07 14:29:54.958787 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:29:54.958781 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-8649cf795f-kn4kp" Jul 07 14:29:54.963243 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:29:54.963222 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-8649cf795f-kn4kp" Jul 07 14:29:55.864171 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:29:55.864143 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-8649cf795f-kn4kp" Jul 07 14:29:55.912029 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:29:55.911993 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6fbf9dbdc9-b45gx"] Jul 07 14:29:56.014595 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:29:56.014561 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:29:56.117838 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:29:56.117767 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:29:56.185706 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:29:56.185674 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:30:20.937620 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:30:20.937569 2557 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-6fbf9dbdc9-b45gx" podUID="d58a9997-5009-4986-ac9a-5313ded08577" containerName="console" containerID="cri-o://992814ccc43322917993fa36ab4dcf9dee2346b8464bb70c311516afcfb58670" gracePeriod=15 Jul 07 14:30:21.177623 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:30:21.177600 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6fbf9dbdc9-b45gx" Jul 07 14:30:21.364302 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:30:21.364273 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5gqlc\" (UniqueName: \"kubernetes.io/projected/d58a9997-5009-4986-ac9a-5313ded08577-kube-api-access-5gqlc\") pod \"d58a9997-5009-4986-ac9a-5313ded08577\" (UID: \"d58a9997-5009-4986-ac9a-5313ded08577\") " Jul 07 14:30:21.364421 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:30:21.364308 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d58a9997-5009-4986-ac9a-5313ded08577-oauth-serving-cert\") pod \"d58a9997-5009-4986-ac9a-5313ded08577\" (UID: \"d58a9997-5009-4986-ac9a-5313ded08577\") " Jul 07 14:30:21.364421 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:30:21.364336 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d58a9997-5009-4986-ac9a-5313ded08577-trusted-ca-bundle\") pod \"d58a9997-5009-4986-ac9a-5313ded08577\" (UID: \"d58a9997-5009-4986-ac9a-5313ded08577\") " Jul 07 14:30:21.364421 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:30:21.364365 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d58a9997-5009-4986-ac9a-5313ded08577-service-ca\") pod \"d58a9997-5009-4986-ac9a-5313ded08577\" (UID: \"d58a9997-5009-4986-ac9a-5313ded08577\") " Jul 07 14:30:21.364421 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:30:21.364408 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d58a9997-5009-4986-ac9a-5313ded08577-console-serving-cert\") pod \"d58a9997-5009-4986-ac9a-5313ded08577\" (UID: \"d58a9997-5009-4986-ac9a-5313ded08577\") " Jul 07 14:30:21.364646 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:30:21.364430 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d58a9997-5009-4986-ac9a-5313ded08577-console-config\") pod \"d58a9997-5009-4986-ac9a-5313ded08577\" (UID: \"d58a9997-5009-4986-ac9a-5313ded08577\") " Jul 07 14:30:21.364646 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:30:21.364447 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d58a9997-5009-4986-ac9a-5313ded08577-console-oauth-config\") pod \"d58a9997-5009-4986-ac9a-5313ded08577\" (UID: \"d58a9997-5009-4986-ac9a-5313ded08577\") " Jul 07 14:30:21.364850 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:30:21.364824 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d58a9997-5009-4986-ac9a-5313ded08577-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "d58a9997-5009-4986-ac9a-5313ded08577" (UID: "d58a9997-5009-4986-ac9a-5313ded08577"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 14:30:21.364980 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:30:21.364957 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d58a9997-5009-4986-ac9a-5313ded08577-console-config" (OuterVolumeSpecName: "console-config") pod "d58a9997-5009-4986-ac9a-5313ded08577" (UID: "d58a9997-5009-4986-ac9a-5313ded08577"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 14:30:21.365042 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:30:21.364980 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d58a9997-5009-4986-ac9a-5313ded08577-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "d58a9997-5009-4986-ac9a-5313ded08577" (UID: "d58a9997-5009-4986-ac9a-5313ded08577"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 14:30:21.365042 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:30:21.364974 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d58a9997-5009-4986-ac9a-5313ded08577-service-ca" (OuterVolumeSpecName: "service-ca") pod "d58a9997-5009-4986-ac9a-5313ded08577" (UID: "d58a9997-5009-4986-ac9a-5313ded08577"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 14:30:21.366554 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:30:21.366508 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d58a9997-5009-4986-ac9a-5313ded08577-kube-api-access-5gqlc" (OuterVolumeSpecName: "kube-api-access-5gqlc") pod "d58a9997-5009-4986-ac9a-5313ded08577" (UID: "d58a9997-5009-4986-ac9a-5313ded08577"). InnerVolumeSpecName "kube-api-access-5gqlc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 14:30:21.366554 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:30:21.366541 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d58a9997-5009-4986-ac9a-5313ded08577-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "d58a9997-5009-4986-ac9a-5313ded08577" (UID: "d58a9997-5009-4986-ac9a-5313ded08577"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 14:30:21.366688 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:30:21.366580 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d58a9997-5009-4986-ac9a-5313ded08577-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "d58a9997-5009-4986-ac9a-5313ded08577" (UID: "d58a9997-5009-4986-ac9a-5313ded08577"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 14:30:21.464954 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:30:21.464930 2557 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d58a9997-5009-4986-ac9a-5313ded08577-console-serving-cert\") on node \"ip-10-0-134-42.ec2.internal\" DevicePath \"\"" Jul 07 14:30:21.464954 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:30:21.464954 2557 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d58a9997-5009-4986-ac9a-5313ded08577-console-config\") on node \"ip-10-0-134-42.ec2.internal\" DevicePath \"\"" Jul 07 14:30:21.465092 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:30:21.464968 2557 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d58a9997-5009-4986-ac9a-5313ded08577-console-oauth-config\") on node \"ip-10-0-134-42.ec2.internal\" DevicePath \"\"" Jul 07 14:30:21.465092 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:30:21.464981 2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5gqlc\" (UniqueName: \"kubernetes.io/projected/d58a9997-5009-4986-ac9a-5313ded08577-kube-api-access-5gqlc\") on node \"ip-10-0-134-42.ec2.internal\" DevicePath \"\"" Jul 07 14:30:21.465092 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:30:21.464994 2557 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d58a9997-5009-4986-ac9a-5313ded08577-oauth-serving-cert\") on node \"ip-10-0-134-42.ec2.internal\" DevicePath \"\"" Jul 07 14:30:21.465092 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:30:21.465005 2557 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d58a9997-5009-4986-ac9a-5313ded08577-trusted-ca-bundle\") on node \"ip-10-0-134-42.ec2.internal\" DevicePath \"\"" Jul 07 14:30:21.465092 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:30:21.465019 2557 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d58a9997-5009-4986-ac9a-5313ded08577-service-ca\") on node \"ip-10-0-134-42.ec2.internal\" DevicePath \"\"" Jul 07 14:30:21.960207 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:30:21.960172 2557 generic.go:356] "Generic (PLEG): container finished" podID="d58a9997-5009-4986-ac9a-5313ded08577" containerID="992814ccc43322917993fa36ab4dcf9dee2346b8464bb70c311516afcfb58670" exitCode=0 Jul 07 14:30:21.960604 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:30:21.960224 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6fbf9dbdc9-b45gx" Jul 07 14:30:21.960604 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:30:21.960266 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6fbf9dbdc9-b45gx" event={"ID":"d58a9997-5009-4986-ac9a-5313ded08577","Type":"ContainerDied","Data":"992814ccc43322917993fa36ab4dcf9dee2346b8464bb70c311516afcfb58670"} Jul 07 14:30:21.960604 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:30:21.960331 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6fbf9dbdc9-b45gx" event={"ID":"d58a9997-5009-4986-ac9a-5313ded08577","Type":"ContainerDied","Data":"b6c381957a31614fc37c308ddc29b3f345bac69593f47aa51c23712b1263a45f"} Jul 07 14:30:21.960604 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:30:21.960362 2557 scope.go:117] "RemoveContainer" containerID="992814ccc43322917993fa36ab4dcf9dee2346b8464bb70c311516afcfb58670" Jul 07 14:30:21.968613 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:30:21.968592 2557 scope.go:117] "RemoveContainer" containerID="992814ccc43322917993fa36ab4dcf9dee2346b8464bb70c311516afcfb58670" Jul 07 14:30:21.968842 ip-10-0-134-42 kubenswrapper[2557]: E0707 14:30:21.968826 2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"992814ccc43322917993fa36ab4dcf9dee2346b8464bb70c311516afcfb58670\": container with ID starting with 992814ccc43322917993fa36ab4dcf9dee2346b8464bb70c311516afcfb58670 not found: ID does not exist" containerID="992814ccc43322917993fa36ab4dcf9dee2346b8464bb70c311516afcfb58670" Jul 07 14:30:21.968900 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:30:21.968850 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"992814ccc43322917993fa36ab4dcf9dee2346b8464bb70c311516afcfb58670"} err="failed to get container status \"992814ccc43322917993fa36ab4dcf9dee2346b8464bb70c311516afcfb58670\": rpc error: code = NotFound desc = could not find container \"992814ccc43322917993fa36ab4dcf9dee2346b8464bb70c311516afcfb58670\": container with ID starting with 992814ccc43322917993fa36ab4dcf9dee2346b8464bb70c311516afcfb58670 not found: ID does not exist" Jul 07 14:30:21.976377 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:30:21.976358 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6fbf9dbdc9-b45gx"] Jul 07 14:30:21.978621 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:30:21.978603 2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-6fbf9dbdc9-b45gx"] Jul 07 14:30:23.835514 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:30:23.835481 2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d58a9997-5009-4986-ac9a-5313ded08577" path="/var/lib/kubelet/pods/d58a9997-5009-4986-ac9a-5313ded08577/volumes" Jul 07 14:30:47.362901 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:30:47.362856 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:30:49.162845 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:30:49.162808 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:30:49.263734 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:30:49.263706 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:31:01.085894 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:31:01.085847 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:31:01.152868 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:31:01.152836 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:31:03.696565 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:31:03.696533 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-vd9g9_0fa80df4-c933-4758-b75f-3f70b7217c0d/console-operator/2.log" Jul 07 14:31:03.699322 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:31:03.699298 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-vd9g9_0fa80df4-c933-4758-b75f-3f70b7217c0d/console-operator/2.log" Jul 07 14:31:03.701840 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:31:03.701813 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-scfg4_90d2dce1-be42-4595-b9c4-74a024eef0e9/ovn-acl-logging/0.log" Jul 07 14:31:03.705548 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:31:03.705505 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-scfg4_90d2dce1-be42-4595-b9c4-74a024eef0e9/ovn-acl-logging/0.log" Jul 07 14:31:03.710169 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:31:03.710144 2557 kubelet.go:1648] "Image garbage collection succeeded" Jul 07 14:31:05.122638 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:31:05.122610 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-7d4b64bf94-ptm9z"] Jul 07 14:31:05.124119 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:31:05.123017 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d58a9997-5009-4986-ac9a-5313ded08577" containerName="console" Jul 07 14:31:05.124119 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:31:05.123033 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="d58a9997-5009-4986-ac9a-5313ded08577" containerName="console" Jul 07 14:31:05.124119 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:31:05.123113 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="d58a9997-5009-4986-ac9a-5313ded08577" containerName="console" Jul 07 14:31:05.125415 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:31:05.125388 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7d4b64bf94-ptm9z" Jul 07 14:31:05.137317 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:31:05.137294 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7d4b64bf94-ptm9z"] Jul 07 14:31:05.164482 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:31:05.164458 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/44e65409-bca6-4333-8aa6-c6a0ca6ea8ba-console-config\") pod \"console-7d4b64bf94-ptm9z\" (UID: \"44e65409-bca6-4333-8aa6-c6a0ca6ea8ba\") " pod="openshift-console/console-7d4b64bf94-ptm9z" Jul 07 14:31:05.164659 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:31:05.164488 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l6qr2\" (UniqueName: \"kubernetes.io/projected/44e65409-bca6-4333-8aa6-c6a0ca6ea8ba-kube-api-access-l6qr2\") pod \"console-7d4b64bf94-ptm9z\" (UID: \"44e65409-bca6-4333-8aa6-c6a0ca6ea8ba\") " pod="openshift-console/console-7d4b64bf94-ptm9z" Jul 07 14:31:05.164659 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:31:05.164561 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/44e65409-bca6-4333-8aa6-c6a0ca6ea8ba-console-serving-cert\") pod \"console-7d4b64bf94-ptm9z\" (UID: \"44e65409-bca6-4333-8aa6-c6a0ca6ea8ba\") " pod="openshift-console/console-7d4b64bf94-ptm9z" Jul 07 14:31:05.164659 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:31:05.164636 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/44e65409-bca6-4333-8aa6-c6a0ca6ea8ba-oauth-serving-cert\") pod \"console-7d4b64bf94-ptm9z\" (UID: \"44e65409-bca6-4333-8aa6-c6a0ca6ea8ba\") " pod="openshift-console/console-7d4b64bf94-ptm9z" Jul 07 14:31:05.164812 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:31:05.164664 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/44e65409-bca6-4333-8aa6-c6a0ca6ea8ba-console-oauth-config\") pod \"console-7d4b64bf94-ptm9z\" (UID: \"44e65409-bca6-4333-8aa6-c6a0ca6ea8ba\") " pod="openshift-console/console-7d4b64bf94-ptm9z" Jul 07 14:31:05.164812 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:31:05.164704 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/44e65409-bca6-4333-8aa6-c6a0ca6ea8ba-trusted-ca-bundle\") pod \"console-7d4b64bf94-ptm9z\" (UID: \"44e65409-bca6-4333-8aa6-c6a0ca6ea8ba\") " pod="openshift-console/console-7d4b64bf94-ptm9z" Jul 07 14:31:05.164812 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:31:05.164747 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/44e65409-bca6-4333-8aa6-c6a0ca6ea8ba-service-ca\") pod \"console-7d4b64bf94-ptm9z\" (UID: \"44e65409-bca6-4333-8aa6-c6a0ca6ea8ba\") " pod="openshift-console/console-7d4b64bf94-ptm9z" Jul 07 14:31:05.214144 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:31:05.214116 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:31:05.265224 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:31:05.265203 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/44e65409-bca6-4333-8aa6-c6a0ca6ea8ba-trusted-ca-bundle\") pod \"console-7d4b64bf94-ptm9z\" (UID: \"44e65409-bca6-4333-8aa6-c6a0ca6ea8ba\") " pod="openshift-console/console-7d4b64bf94-ptm9z" Jul 07 14:31:05.265336 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:31:05.265232 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/44e65409-bca6-4333-8aa6-c6a0ca6ea8ba-service-ca\") pod \"console-7d4b64bf94-ptm9z\" (UID: \"44e65409-bca6-4333-8aa6-c6a0ca6ea8ba\") " pod="openshift-console/console-7d4b64bf94-ptm9z" Jul 07 14:31:05.265336 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:31:05.265275 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/44e65409-bca6-4333-8aa6-c6a0ca6ea8ba-console-config\") pod \"console-7d4b64bf94-ptm9z\" (UID: \"44e65409-bca6-4333-8aa6-c6a0ca6ea8ba\") " pod="openshift-console/console-7d4b64bf94-ptm9z" Jul 07 14:31:05.265336 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:31:05.265293 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l6qr2\" (UniqueName: \"kubernetes.io/projected/44e65409-bca6-4333-8aa6-c6a0ca6ea8ba-kube-api-access-l6qr2\") pod \"console-7d4b64bf94-ptm9z\" (UID: \"44e65409-bca6-4333-8aa6-c6a0ca6ea8ba\") " pod="openshift-console/console-7d4b64bf94-ptm9z" Jul 07 14:31:05.265336 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:31:05.265327 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/44e65409-bca6-4333-8aa6-c6a0ca6ea8ba-console-serving-cert\") pod \"console-7d4b64bf94-ptm9z\" (UID: \"44e65409-bca6-4333-8aa6-c6a0ca6ea8ba\") " pod="openshift-console/console-7d4b64bf94-ptm9z" Jul 07 14:31:05.265588 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:31:05.265358 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/44e65409-bca6-4333-8aa6-c6a0ca6ea8ba-oauth-serving-cert\") pod \"console-7d4b64bf94-ptm9z\" (UID: \"44e65409-bca6-4333-8aa6-c6a0ca6ea8ba\") " pod="openshift-console/console-7d4b64bf94-ptm9z" Jul 07 14:31:05.265588 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:31:05.265384 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/44e65409-bca6-4333-8aa6-c6a0ca6ea8ba-console-oauth-config\") pod \"console-7d4b64bf94-ptm9z\" (UID: \"44e65409-bca6-4333-8aa6-c6a0ca6ea8ba\") " pod="openshift-console/console-7d4b64bf94-ptm9z" Jul 07 14:31:05.266018 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:31:05.265992 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/44e65409-bca6-4333-8aa6-c6a0ca6ea8ba-service-ca\") pod \"console-7d4b64bf94-ptm9z\" (UID: \"44e65409-bca6-4333-8aa6-c6a0ca6ea8ba\") " pod="openshift-console/console-7d4b64bf94-ptm9z" Jul 07 14:31:05.266121 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:31:05.266103 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/44e65409-bca6-4333-8aa6-c6a0ca6ea8ba-console-config\") pod \"console-7d4b64bf94-ptm9z\" (UID: \"44e65409-bca6-4333-8aa6-c6a0ca6ea8ba\") " pod="openshift-console/console-7d4b64bf94-ptm9z" Jul 07 14:31:05.266178 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:31:05.266152 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/44e65409-bca6-4333-8aa6-c6a0ca6ea8ba-oauth-serving-cert\") pod \"console-7d4b64bf94-ptm9z\" (UID: \"44e65409-bca6-4333-8aa6-c6a0ca6ea8ba\") " pod="openshift-console/console-7d4b64bf94-ptm9z" Jul 07 14:31:05.266178 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:31:05.266168 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/44e65409-bca6-4333-8aa6-c6a0ca6ea8ba-trusted-ca-bundle\") pod \"console-7d4b64bf94-ptm9z\" (UID: \"44e65409-bca6-4333-8aa6-c6a0ca6ea8ba\") " pod="openshift-console/console-7d4b64bf94-ptm9z" Jul 07 14:31:05.267784 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:31:05.267761 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/44e65409-bca6-4333-8aa6-c6a0ca6ea8ba-console-oauth-config\") pod \"console-7d4b64bf94-ptm9z\" (UID: \"44e65409-bca6-4333-8aa6-c6a0ca6ea8ba\") " pod="openshift-console/console-7d4b64bf94-ptm9z" Jul 07 14:31:05.267857 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:31:05.267814 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/44e65409-bca6-4333-8aa6-c6a0ca6ea8ba-console-serving-cert\") pod \"console-7d4b64bf94-ptm9z\" (UID: \"44e65409-bca6-4333-8aa6-c6a0ca6ea8ba\") " pod="openshift-console/console-7d4b64bf94-ptm9z" Jul 07 14:31:05.271632 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:31:05.271604 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:31:05.272890 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:31:05.272867 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-l6qr2\" (UniqueName: \"kubernetes.io/projected/44e65409-bca6-4333-8aa6-c6a0ca6ea8ba-kube-api-access-l6qr2\") pod \"console-7d4b64bf94-ptm9z\" (UID: \"44e65409-bca6-4333-8aa6-c6a0ca6ea8ba\") " pod="openshift-console/console-7d4b64bf94-ptm9z" Jul 07 14:31:05.437733 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:31:05.437669 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7d4b64bf94-ptm9z" Jul 07 14:31:05.567435 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:31:05.567403 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7d4b64bf94-ptm9z"] Jul 07 14:31:05.570513 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:31:05.570485 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod44e65409_bca6_4333_8aa6_c6a0ca6ea8ba.slice/crio-d221531079cff0dc6e4002a1d651c6f599f2d1ad081011d4057fa4272fd905c1 WatchSource:0}: Error finding container d221531079cff0dc6e4002a1d651c6f599f2d1ad081011d4057fa4272fd905c1: Status 404 returned error can't find the container with id d221531079cff0dc6e4002a1d651c6f599f2d1ad081011d4057fa4272fd905c1 Jul 07 14:31:06.130811 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:31:06.130773 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7d4b64bf94-ptm9z" event={"ID":"44e65409-bca6-4333-8aa6-c6a0ca6ea8ba","Type":"ContainerStarted","Data":"d2effd5275dee91c258f91409b9553b8240867a47ab5b59d61943da9778e20f1"} Jul 07 14:31:06.131294 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:31:06.130823 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7d4b64bf94-ptm9z" event={"ID":"44e65409-bca6-4333-8aa6-c6a0ca6ea8ba","Type":"ContainerStarted","Data":"d221531079cff0dc6e4002a1d651c6f599f2d1ad081011d4057fa4272fd905c1"} Jul 07 14:31:06.149098 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:31:06.149050 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-7d4b64bf94-ptm9z" podStartSLOduration=1.149037211 podStartE2EDuration="1.149037211s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 14:31:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:31:06.14765411 +0000 UTC m=+302.856792489" watchObservedRunningTime="2026-07-07 14:31:06.149037211 +0000 UTC m=+302.858175590" Jul 07 14:31:06.326790 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:31:06.326757 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:31:06.427272 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:31:06.427200 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:31:15.438381 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:31:15.438348 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-7d4b64bf94-ptm9z" Jul 07 14:31:15.438847 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:31:15.438397 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7d4b64bf94-ptm9z" Jul 07 14:31:15.443196 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:31:15.443173 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-7d4b64bf94-ptm9z" Jul 07 14:31:16.173648 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:31:16.173621 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-7d4b64bf94-ptm9z" Jul 07 14:31:16.223363 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:31:16.223327 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-8649cf795f-kn4kp"] Jul 07 14:31:16.370277 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:31:16.370245 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:31:16.463716 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:31:16.463630 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:31:31.748744 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:31:31.748683 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:31:32.836041 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:31:32.836007 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540kngpn"] Jul 07 14:31:32.838915 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:31:32.838896 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540kngpn" Jul 07 14:31:32.840887 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:31:32.840869 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 07 14:31:32.840989 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:31:32.840882 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-kqlhz\"" Jul 07 14:31:32.841291 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:31:32.841278 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 07 14:31:32.848807 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:31:32.848785 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540kngpn"] Jul 07 14:31:32.986472 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:31:32.986436 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c3f29dc0-d2ba-47bc-b1b8-70c711c12c52-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540kngpn\" (UID: \"c3f29dc0-d2ba-47bc-b1b8-70c711c12c52\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540kngpn" Jul 07 14:31:32.986725 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:31:32.986492 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c3f29dc0-d2ba-47bc-b1b8-70c711c12c52-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540kngpn\" (UID: \"c3f29dc0-d2ba-47bc-b1b8-70c711c12c52\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540kngpn" Jul 07 14:31:32.986725 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:31:32.986558 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wwldz\" (UniqueName: \"kubernetes.io/projected/c3f29dc0-d2ba-47bc-b1b8-70c711c12c52-kube-api-access-wwldz\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540kngpn\" (UID: \"c3f29dc0-d2ba-47bc-b1b8-70c711c12c52\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540kngpn" Jul 07 14:31:33.087403 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:31:33.087328 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c3f29dc0-d2ba-47bc-b1b8-70c711c12c52-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540kngpn\" (UID: \"c3f29dc0-d2ba-47bc-b1b8-70c711c12c52\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540kngpn" Jul 07 14:31:33.087403 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:31:33.087376 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c3f29dc0-d2ba-47bc-b1b8-70c711c12c52-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540kngpn\" (UID: \"c3f29dc0-d2ba-47bc-b1b8-70c711c12c52\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540kngpn" Jul 07 14:31:33.087403 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:31:33.087401 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wwldz\" (UniqueName: \"kubernetes.io/projected/c3f29dc0-d2ba-47bc-b1b8-70c711c12c52-kube-api-access-wwldz\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540kngpn\" (UID: \"c3f29dc0-d2ba-47bc-b1b8-70c711c12c52\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540kngpn" Jul 07 14:31:33.087716 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:31:33.087697 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c3f29dc0-d2ba-47bc-b1b8-70c711c12c52-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540kngpn\" (UID: \"c3f29dc0-d2ba-47bc-b1b8-70c711c12c52\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540kngpn" Jul 07 14:31:33.087788 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:31:33.087768 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c3f29dc0-d2ba-47bc-b1b8-70c711c12c52-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540kngpn\" (UID: \"c3f29dc0-d2ba-47bc-b1b8-70c711c12c52\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540kngpn" Jul 07 14:31:33.096913 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:31:33.096892 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wwldz\" (UniqueName: \"kubernetes.io/projected/c3f29dc0-d2ba-47bc-b1b8-70c711c12c52-kube-api-access-wwldz\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540kngpn\" (UID: \"c3f29dc0-d2ba-47bc-b1b8-70c711c12c52\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540kngpn" Jul 07 14:31:33.148772 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:31:33.148747 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540kngpn" Jul 07 14:31:33.201626 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:31:33.201557 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:31:33.277000 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:31:33.276974 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540kngpn"] Jul 07 14:31:33.279476 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:31:33.279448 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc3f29dc0_d2ba_47bc_b1b8_70c711c12c52.slice/crio-09a6a490aae240e75f2e3eb49c1cf51e121ee106cd33f5f7ff3c8e2f6c7c02c2 WatchSource:0}: Error finding container 09a6a490aae240e75f2e3eb49c1cf51e121ee106cd33f5f7ff3c8e2f6c7c02c2: Status 404 returned error can't find the container with id 09a6a490aae240e75f2e3eb49c1cf51e121ee106cd33f5f7ff3c8e2f6c7c02c2 Jul 07 14:31:33.281326 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:31:33.281312 2557 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 07 14:31:34.241831 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:31:34.241791 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540kngpn" event={"ID":"c3f29dc0-d2ba-47bc-b1b8-70c711c12c52","Type":"ContainerStarted","Data":"09a6a490aae240e75f2e3eb49c1cf51e121ee106cd33f5f7ff3c8e2f6c7c02c2"} Jul 07 14:31:41.244687 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:31:41.244608 2557 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-8649cf795f-kn4kp" podUID="513779ad-5c00-4e1e-82c1-3651231a147f" containerName="console" containerID="cri-o://caf3a76b9460c83d538835d579e74c0dda5a95e10ff4adc2c5deba9acc4b8150" gracePeriod=15 Jul 07 14:31:42.787105 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:31:42.787083 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-8649cf795f-kn4kp" Jul 07 14:31:42.964329 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:31:42.964231 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/513779ad-5c00-4e1e-82c1-3651231a147f-oauth-serving-cert\") pod \"513779ad-5c00-4e1e-82c1-3651231a147f\" (UID: \"513779ad-5c00-4e1e-82c1-3651231a147f\") " Jul 07 14:31:42.964329 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:31:42.964333 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/513779ad-5c00-4e1e-82c1-3651231a147f-console-oauth-config\") pod \"513779ad-5c00-4e1e-82c1-3651231a147f\" (UID: \"513779ad-5c00-4e1e-82c1-3651231a147f\") " Jul 07 14:31:42.964553 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:31:42.964359 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/513779ad-5c00-4e1e-82c1-3651231a147f-console-serving-cert\") pod \"513779ad-5c00-4e1e-82c1-3651231a147f\" (UID: \"513779ad-5c00-4e1e-82c1-3651231a147f\") " Jul 07 14:31:42.964553 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:31:42.964389 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/513779ad-5c00-4e1e-82c1-3651231a147f-trusted-ca-bundle\") pod \"513779ad-5c00-4e1e-82c1-3651231a147f\" (UID: \"513779ad-5c00-4e1e-82c1-3651231a147f\") " Jul 07 14:31:42.964553 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:31:42.964442 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zhhkd\" (UniqueName: \"kubernetes.io/projected/513779ad-5c00-4e1e-82c1-3651231a147f-kube-api-access-zhhkd\") pod \"513779ad-5c00-4e1e-82c1-3651231a147f\" (UID: \"513779ad-5c00-4e1e-82c1-3651231a147f\") " Jul 07 14:31:42.964553 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:31:42.964488 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/513779ad-5c00-4e1e-82c1-3651231a147f-service-ca\") pod \"513779ad-5c00-4e1e-82c1-3651231a147f\" (UID: \"513779ad-5c00-4e1e-82c1-3651231a147f\") " Jul 07 14:31:42.964553 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:31:42.964553 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/513779ad-5c00-4e1e-82c1-3651231a147f-console-config\") pod \"513779ad-5c00-4e1e-82c1-3651231a147f\" (UID: \"513779ad-5c00-4e1e-82c1-3651231a147f\") " Jul 07 14:31:42.964806 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:31:42.964635 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/513779ad-5c00-4e1e-82c1-3651231a147f-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "513779ad-5c00-4e1e-82c1-3651231a147f" (UID: "513779ad-5c00-4e1e-82c1-3651231a147f"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 14:31:42.964895 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:31:42.964873 2557 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/513779ad-5c00-4e1e-82c1-3651231a147f-oauth-serving-cert\") on node \"ip-10-0-134-42.ec2.internal\" DevicePath \"\"" Jul 07 14:31:42.964949 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:31:42.964919 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/513779ad-5c00-4e1e-82c1-3651231a147f-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "513779ad-5c00-4e1e-82c1-3651231a147f" (UID: "513779ad-5c00-4e1e-82c1-3651231a147f"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 14:31:42.964995 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:31:42.964928 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/513779ad-5c00-4e1e-82c1-3651231a147f-console-config" (OuterVolumeSpecName: "console-config") pod "513779ad-5c00-4e1e-82c1-3651231a147f" (UID: "513779ad-5c00-4e1e-82c1-3651231a147f"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 14:31:42.964995 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:31:42.964938 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/513779ad-5c00-4e1e-82c1-3651231a147f-service-ca" (OuterVolumeSpecName: "service-ca") pod "513779ad-5c00-4e1e-82c1-3651231a147f" (UID: "513779ad-5c00-4e1e-82c1-3651231a147f"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 14:31:42.966618 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:31:42.966590 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/513779ad-5c00-4e1e-82c1-3651231a147f-kube-api-access-zhhkd" (OuterVolumeSpecName: "kube-api-access-zhhkd") pod "513779ad-5c00-4e1e-82c1-3651231a147f" (UID: "513779ad-5c00-4e1e-82c1-3651231a147f"). InnerVolumeSpecName "kube-api-access-zhhkd". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 14:31:42.966725 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:31:42.966629 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/513779ad-5c00-4e1e-82c1-3651231a147f-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "513779ad-5c00-4e1e-82c1-3651231a147f" (UID: "513779ad-5c00-4e1e-82c1-3651231a147f"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 14:31:42.966725 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:31:42.966657 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/513779ad-5c00-4e1e-82c1-3651231a147f-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "513779ad-5c00-4e1e-82c1-3651231a147f" (UID: "513779ad-5c00-4e1e-82c1-3651231a147f"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 14:31:43.066031 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:31:43.066000 2557 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/513779ad-5c00-4e1e-82c1-3651231a147f-console-oauth-config\") on node \"ip-10-0-134-42.ec2.internal\" DevicePath \"\"" Jul 07 14:31:43.066031 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:31:43.066025 2557 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/513779ad-5c00-4e1e-82c1-3651231a147f-console-serving-cert\") on node \"ip-10-0-134-42.ec2.internal\" DevicePath \"\"" Jul 07 14:31:43.066031 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:31:43.066034 2557 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/513779ad-5c00-4e1e-82c1-3651231a147f-trusted-ca-bundle\") on node \"ip-10-0-134-42.ec2.internal\" DevicePath \"\"" Jul 07 14:31:43.066306 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:31:43.066045 2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zhhkd\" (UniqueName: \"kubernetes.io/projected/513779ad-5c00-4e1e-82c1-3651231a147f-kube-api-access-zhhkd\") on node \"ip-10-0-134-42.ec2.internal\" DevicePath \"\"" Jul 07 14:31:43.066306 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:31:43.066058 2557 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/513779ad-5c00-4e1e-82c1-3651231a147f-service-ca\") on node \"ip-10-0-134-42.ec2.internal\" DevicePath \"\"" Jul 07 14:31:43.066306 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:31:43.066068 2557 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/513779ad-5c00-4e1e-82c1-3651231a147f-console-config\") on node \"ip-10-0-134-42.ec2.internal\" DevicePath \"\"" Jul 07 14:31:43.279732 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:31:43.279639 2557 generic.go:356] "Generic (PLEG): container finished" podID="513779ad-5c00-4e1e-82c1-3651231a147f" containerID="caf3a76b9460c83d538835d579e74c0dda5a95e10ff4adc2c5deba9acc4b8150" exitCode=0 Jul 07 14:31:43.279732 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:31:43.279683 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-8649cf795f-kn4kp" Jul 07 14:31:43.279926 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:31:43.279727 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-8649cf795f-kn4kp" event={"ID":"513779ad-5c00-4e1e-82c1-3651231a147f","Type":"ContainerDied","Data":"caf3a76b9460c83d538835d579e74c0dda5a95e10ff4adc2c5deba9acc4b8150"} Jul 07 14:31:43.279926 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:31:43.279786 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-8649cf795f-kn4kp" event={"ID":"513779ad-5c00-4e1e-82c1-3651231a147f","Type":"ContainerDied","Data":"32e805d826aa94bd3fbdb462fe44abd0f45a7c379bfe503af7ab99cd4ec3c1d5"} Jul 07 14:31:43.279926 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:31:43.279807 2557 scope.go:117] "RemoveContainer" containerID="caf3a76b9460c83d538835d579e74c0dda5a95e10ff4adc2c5deba9acc4b8150" Jul 07 14:31:43.281634 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:31:43.281608 2557 generic.go:356] "Generic (PLEG): container finished" podID="c3f29dc0-d2ba-47bc-b1b8-70c711c12c52" containerID="254972ed2e775050e514aa246629d07b0b8d056fea5915270aaf662091654546" exitCode=0 Jul 07 14:31:43.281750 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:31:43.281720 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540kngpn" event={"ID":"c3f29dc0-d2ba-47bc-b1b8-70c711c12c52","Type":"ContainerDied","Data":"254972ed2e775050e514aa246629d07b0b8d056fea5915270aaf662091654546"} Jul 07 14:31:43.288663 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:31:43.288625 2557 scope.go:117] "RemoveContainer" containerID="caf3a76b9460c83d538835d579e74c0dda5a95e10ff4adc2c5deba9acc4b8150" Jul 07 14:31:43.289036 ip-10-0-134-42 kubenswrapper[2557]: E0707 14:31:43.288992 2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"caf3a76b9460c83d538835d579e74c0dda5a95e10ff4adc2c5deba9acc4b8150\": container with ID starting with caf3a76b9460c83d538835d579e74c0dda5a95e10ff4adc2c5deba9acc4b8150 not found: ID does not exist" containerID="caf3a76b9460c83d538835d579e74c0dda5a95e10ff4adc2c5deba9acc4b8150" Jul 07 14:31:43.289129 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:31:43.289028 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"caf3a76b9460c83d538835d579e74c0dda5a95e10ff4adc2c5deba9acc4b8150"} err="failed to get container status \"caf3a76b9460c83d538835d579e74c0dda5a95e10ff4adc2c5deba9acc4b8150\": rpc error: code = NotFound desc = could not find container \"caf3a76b9460c83d538835d579e74c0dda5a95e10ff4adc2c5deba9acc4b8150\": container with ID starting with caf3a76b9460c83d538835d579e74c0dda5a95e10ff4adc2c5deba9acc4b8150 not found: ID does not exist" Jul 07 14:31:43.310744 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:31:43.310717 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-8649cf795f-kn4kp"] Jul 07 14:31:43.311967 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:31:43.311950 2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-8649cf795f-kn4kp"] Jul 07 14:31:43.834862 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:31:43.834835 2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="513779ad-5c00-4e1e-82c1-3651231a147f" path="/var/lib/kubelet/pods/513779ad-5c00-4e1e-82c1-3651231a147f/volumes" Jul 07 14:31:46.299605 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:31:46.299566 2557 generic.go:356] "Generic (PLEG): container finished" podID="c3f29dc0-d2ba-47bc-b1b8-70c711c12c52" containerID="706427e56e6b195d60a2bda298e515b736c922bd9692914ac92b657eb439347b" exitCode=0 Jul 07 14:31:46.300051 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:31:46.299653 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540kngpn" event={"ID":"c3f29dc0-d2ba-47bc-b1b8-70c711c12c52","Type":"ContainerDied","Data":"706427e56e6b195d60a2bda298e515b736c922bd9692914ac92b657eb439347b"} Jul 07 14:31:57.359980 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:31:57.359952 2557 generic.go:356] "Generic (PLEG): container finished" podID="c3f29dc0-d2ba-47bc-b1b8-70c711c12c52" containerID="7271c299f4a90c574f05d9c34e2cdd617cd4dc089b0b8a082bf45c77f00d7b58" exitCode=0 Jul 07 14:31:57.360277 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:31:57.360010 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540kngpn" event={"ID":"c3f29dc0-d2ba-47bc-b1b8-70c711c12c52","Type":"ContainerDied","Data":"7271c299f4a90c574f05d9c34e2cdd617cd4dc089b0b8a082bf45c77f00d7b58"} Jul 07 14:31:58.497245 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:31:58.497223 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540kngpn" Jul 07 14:31:58.588278 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:31:58.588251 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c3f29dc0-d2ba-47bc-b1b8-70c711c12c52-bundle\") pod \"c3f29dc0-d2ba-47bc-b1b8-70c711c12c52\" (UID: \"c3f29dc0-d2ba-47bc-b1b8-70c711c12c52\") " Jul 07 14:31:58.588396 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:31:58.588335 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wwldz\" (UniqueName: \"kubernetes.io/projected/c3f29dc0-d2ba-47bc-b1b8-70c711c12c52-kube-api-access-wwldz\") pod \"c3f29dc0-d2ba-47bc-b1b8-70c711c12c52\" (UID: \"c3f29dc0-d2ba-47bc-b1b8-70c711c12c52\") " Jul 07 14:31:58.588396 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:31:58.588353 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c3f29dc0-d2ba-47bc-b1b8-70c711c12c52-util\") pod \"c3f29dc0-d2ba-47bc-b1b8-70c711c12c52\" (UID: \"c3f29dc0-d2ba-47bc-b1b8-70c711c12c52\") " Jul 07 14:31:58.588938 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:31:58.588914 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c3f29dc0-d2ba-47bc-b1b8-70c711c12c52-bundle" (OuterVolumeSpecName: "bundle") pod "c3f29dc0-d2ba-47bc-b1b8-70c711c12c52" (UID: "c3f29dc0-d2ba-47bc-b1b8-70c711c12c52"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 14:31:58.590342 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:31:58.590317 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c3f29dc0-d2ba-47bc-b1b8-70c711c12c52-kube-api-access-wwldz" (OuterVolumeSpecName: "kube-api-access-wwldz") pod "c3f29dc0-d2ba-47bc-b1b8-70c711c12c52" (UID: "c3f29dc0-d2ba-47bc-b1b8-70c711c12c52"). InnerVolumeSpecName "kube-api-access-wwldz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 14:31:58.592337 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:31:58.592315 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c3f29dc0-d2ba-47bc-b1b8-70c711c12c52-util" (OuterVolumeSpecName: "util") pod "c3f29dc0-d2ba-47bc-b1b8-70c711c12c52" (UID: "c3f29dc0-d2ba-47bc-b1b8-70c711c12c52"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 14:31:58.689225 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:31:58.689177 2557 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c3f29dc0-d2ba-47bc-b1b8-70c711c12c52-bundle\") on node \"ip-10-0-134-42.ec2.internal\" DevicePath \"\"" Jul 07 14:31:58.689225 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:31:58.689197 2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wwldz\" (UniqueName: \"kubernetes.io/projected/c3f29dc0-d2ba-47bc-b1b8-70c711c12c52-kube-api-access-wwldz\") on node \"ip-10-0-134-42.ec2.internal\" DevicePath \"\"" Jul 07 14:31:58.689225 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:31:58.689205 2557 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c3f29dc0-d2ba-47bc-b1b8-70c711c12c52-util\") on node \"ip-10-0-134-42.ec2.internal\" DevicePath \"\"" Jul 07 14:31:59.368759 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:31:59.368723 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540kngpn" event={"ID":"c3f29dc0-d2ba-47bc-b1b8-70c711c12c52","Type":"ContainerDied","Data":"09a6a490aae240e75f2e3eb49c1cf51e121ee106cd33f5f7ff3c8e2f6c7c02c2"} Jul 07 14:31:59.368927 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:31:59.368771 2557 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="09a6a490aae240e75f2e3eb49c1cf51e121ee106cd33f5f7ff3c8e2f6c7c02c2" Jul 07 14:31:59.368927 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:31:59.368737 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540kngpn" Jul 07 14:32:03.967266 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:32:03.967227 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:32:04.944583 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:32:04.944548 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:32:05.950626 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:32:05.950279 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-wpdd4"] Jul 07 14:32:05.951024 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:32:05.950692 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c3f29dc0-d2ba-47bc-b1b8-70c711c12c52" containerName="util" Jul 07 14:32:05.951024 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:32:05.950706 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="c3f29dc0-d2ba-47bc-b1b8-70c711c12c52" containerName="util" Jul 07 14:32:05.951024 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:32:05.950724 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c3f29dc0-d2ba-47bc-b1b8-70c711c12c52" containerName="pull" Jul 07 14:32:05.951024 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:32:05.950730 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="c3f29dc0-d2ba-47bc-b1b8-70c711c12c52" containerName="pull" Jul 07 14:32:05.951024 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:32:05.950736 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c3f29dc0-d2ba-47bc-b1b8-70c711c12c52" containerName="extract" Jul 07 14:32:05.951024 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:32:05.950741 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="c3f29dc0-d2ba-47bc-b1b8-70c711c12c52" containerName="extract" Jul 07 14:32:05.951024 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:32:05.950755 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="513779ad-5c00-4e1e-82c1-3651231a147f" containerName="console" Jul 07 14:32:05.951024 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:32:05.950759 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="513779ad-5c00-4e1e-82c1-3651231a147f" containerName="console" Jul 07 14:32:05.951024 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:32:05.950814 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="c3f29dc0-d2ba-47bc-b1b8-70c711c12c52" containerName="extract" Jul 07 14:32:05.951024 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:32:05.950825 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="513779ad-5c00-4e1e-82c1-3651231a147f" containerName="console" Jul 07 14:32:05.953332 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:32:05.953316 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-wpdd4" Jul 07 14:32:05.955152 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:32:05.955130 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\"" Jul 07 14:32:05.955410 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:32:05.955395 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-sngdk\"" Jul 07 14:32:05.955835 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:32:05.955816 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\"" Jul 07 14:32:05.967995 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:32:05.967976 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-wpdd4"] Jul 07 14:32:06.047691 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:32:06.047631 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5e244834-f046-4033-9680-8f0ae20415e0-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-wpdd4\" (UID: \"5e244834-f046-4033-9680-8f0ae20415e0\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-wpdd4" Jul 07 14:32:06.047691 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:32:06.047692 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d76rm\" (UniqueName: \"kubernetes.io/projected/5e244834-f046-4033-9680-8f0ae20415e0-kube-api-access-d76rm\") pod \"cert-manager-operator-controller-manager-55c444bf86-wpdd4\" (UID: \"5e244834-f046-4033-9680-8f0ae20415e0\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-wpdd4" Jul 07 14:32:06.149189 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:32:06.149141 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5e244834-f046-4033-9680-8f0ae20415e0-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-wpdd4\" (UID: \"5e244834-f046-4033-9680-8f0ae20415e0\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-wpdd4" Jul 07 14:32:06.149189 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:32:06.149198 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d76rm\" (UniqueName: \"kubernetes.io/projected/5e244834-f046-4033-9680-8f0ae20415e0-kube-api-access-d76rm\") pod \"cert-manager-operator-controller-manager-55c444bf86-wpdd4\" (UID: \"5e244834-f046-4033-9680-8f0ae20415e0\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-wpdd4" Jul 07 14:32:06.149573 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:32:06.149554 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5e244834-f046-4033-9680-8f0ae20415e0-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-wpdd4\" (UID: \"5e244834-f046-4033-9680-8f0ae20415e0\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-wpdd4" Jul 07 14:32:06.157426 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:32:06.157402 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d76rm\" (UniqueName: \"kubernetes.io/projected/5e244834-f046-4033-9680-8f0ae20415e0-kube-api-access-d76rm\") pod \"cert-manager-operator-controller-manager-55c444bf86-wpdd4\" (UID: \"5e244834-f046-4033-9680-8f0ae20415e0\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-wpdd4" Jul 07 14:32:06.262170 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:32:06.262083 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-wpdd4" Jul 07 14:32:06.393873 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:32:06.393838 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-wpdd4"] Jul 07 14:32:06.396980 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:32:06.396957 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5e244834_f046_4033_9680_8f0ae20415e0.slice/crio-12eb8ed95b6666c8b7b8bf6170d67163e0aa2fd544567b5572b006615e85f014 WatchSource:0}: Error finding container 12eb8ed95b6666c8b7b8bf6170d67163e0aa2fd544567b5572b006615e85f014: Status 404 returned error can't find the container with id 12eb8ed95b6666c8b7b8bf6170d67163e0aa2fd544567b5572b006615e85f014 Jul 07 14:32:07.402511 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:32:07.402459 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-wpdd4" event={"ID":"5e244834-f046-4033-9680-8f0ae20415e0","Type":"ContainerStarted","Data":"12eb8ed95b6666c8b7b8bf6170d67163e0aa2fd544567b5572b006615e85f014"} Jul 07 14:32:07.583839 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:32:07.583796 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:32:07.683058 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:32:07.682979 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:32:08.408623 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:32:08.408585 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-wpdd4" event={"ID":"5e244834-f046-4033-9680-8f0ae20415e0","Type":"ContainerStarted","Data":"adf5fe9f2597e609654f5d6fa52cab316a8a0e91518032f7b0ba97d0aef59c6b"} Jul 07 14:32:08.440229 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:32:08.440175 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-wpdd4" podStartSLOduration=1.599405422 podStartE2EDuration="3.440159927s" totalImagesPullingTime="1.840754505s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 14:32:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:32:08.440016625 +0000 UTC m=+365.149155040" watchObservedRunningTime="2026-07-07 14:32:08.440159927 +0000 UTC m=+365.149298306" Jul 07 14:32:08.873177 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:32:08.873144 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:32:09.442803 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:32:09.442770 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:32:09.542634 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:32:09.542592 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:32:09.823147 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:32:09.823117 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fkbb9t"] Jul 07 14:32:09.825707 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:32:09.825690 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fkbb9t" Jul 07 14:32:09.827647 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:32:09.827621 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 07 14:32:09.827747 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:32:09.827662 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-kqlhz\"" Jul 07 14:32:09.827868 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:32:09.827853 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 07 14:32:09.839303 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:32:09.839282 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fkbb9t"] Jul 07 14:32:09.881767 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:32:09.881739 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9l4gt\" (UniqueName: \"kubernetes.io/projected/da22481d-e70b-43d3-90a9-115cda863e5a-kube-api-access-9l4gt\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fkbb9t\" (UID: \"da22481d-e70b-43d3-90a9-115cda863e5a\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fkbb9t" Jul 07 14:32:09.881898 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:32:09.881779 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/da22481d-e70b-43d3-90a9-115cda863e5a-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fkbb9t\" (UID: \"da22481d-e70b-43d3-90a9-115cda863e5a\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fkbb9t" Jul 07 14:32:09.881898 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:32:09.881804 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/da22481d-e70b-43d3-90a9-115cda863e5a-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fkbb9t\" (UID: \"da22481d-e70b-43d3-90a9-115cda863e5a\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fkbb9t" Jul 07 14:32:09.983228 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:32:09.983171 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9l4gt\" (UniqueName: \"kubernetes.io/projected/da22481d-e70b-43d3-90a9-115cda863e5a-kube-api-access-9l4gt\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fkbb9t\" (UID: \"da22481d-e70b-43d3-90a9-115cda863e5a\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fkbb9t" Jul 07 14:32:09.983410 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:32:09.983254 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/da22481d-e70b-43d3-90a9-115cda863e5a-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fkbb9t\" (UID: \"da22481d-e70b-43d3-90a9-115cda863e5a\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fkbb9t" Jul 07 14:32:09.983410 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:32:09.983300 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/da22481d-e70b-43d3-90a9-115cda863e5a-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fkbb9t\" (UID: \"da22481d-e70b-43d3-90a9-115cda863e5a\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fkbb9t" Jul 07 14:32:09.983666 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:32:09.983645 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/da22481d-e70b-43d3-90a9-115cda863e5a-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fkbb9t\" (UID: \"da22481d-e70b-43d3-90a9-115cda863e5a\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fkbb9t" Jul 07 14:32:09.983742 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:32:09.983674 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/da22481d-e70b-43d3-90a9-115cda863e5a-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fkbb9t\" (UID: \"da22481d-e70b-43d3-90a9-115cda863e5a\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fkbb9t" Jul 07 14:32:09.992788 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:32:09.992767 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9l4gt\" (UniqueName: \"kubernetes.io/projected/da22481d-e70b-43d3-90a9-115cda863e5a-kube-api-access-9l4gt\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fkbb9t\" (UID: \"da22481d-e70b-43d3-90a9-115cda863e5a\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fkbb9t" Jul 07 14:32:10.134994 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:32:10.134910 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fkbb9t" Jul 07 14:32:10.267208 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:32:10.267174 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fkbb9t"] Jul 07 14:32:10.270205 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:32:10.270179 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podda22481d_e70b_43d3_90a9_115cda863e5a.slice/crio-da54e93ddbf6585609f6a78f78b5d23cfb530678690d4da2b37388c79c3f3829 WatchSource:0}: Error finding container da54e93ddbf6585609f6a78f78b5d23cfb530678690d4da2b37388c79c3f3829: Status 404 returned error can't find the container with id da54e93ddbf6585609f6a78f78b5d23cfb530678690d4da2b37388c79c3f3829 Jul 07 14:32:10.419010 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:32:10.418933 2557 generic.go:356] "Generic (PLEG): container finished" podID="da22481d-e70b-43d3-90a9-115cda863e5a" containerID="e8f8ebe4dadbbbeeb5975c1af9400db31ebba1b406deab6ab28f6c9fa067490c" exitCode=0 Jul 07 14:32:10.419162 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:32:10.419029 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fkbb9t" event={"ID":"da22481d-e70b-43d3-90a9-115cda863e5a","Type":"ContainerDied","Data":"e8f8ebe4dadbbbeeb5975c1af9400db31ebba1b406deab6ab28f6c9fa067490c"} Jul 07 14:32:10.419162 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:32:10.419086 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fkbb9t" event={"ID":"da22481d-e70b-43d3-90a9-115cda863e5a","Type":"ContainerStarted","Data":"da54e93ddbf6585609f6a78f78b5d23cfb530678690d4da2b37388c79c3f3829"} Jul 07 14:32:11.544054 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:32:11.544020 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:32:11.921512 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:32:11.921478 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-rjnq2"] Jul 07 14:32:11.927480 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:32:11.927454 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-rjnq2" Jul 07 14:32:11.929781 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:32:11.929757 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-vvg9q\"" Jul 07 14:32:11.929895 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:32:11.929757 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Jul 07 14:32:11.929975 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:32:11.929954 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Jul 07 14:32:11.940750 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:32:11.940718 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-rjnq2"] Jul 07 14:32:11.980807 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:32:11.980766 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:32:12.001568 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:32:12.001541 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d9c9a960-6781-4581-8bf4-1b6158cbe857-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-rjnq2\" (UID: \"d9c9a960-6781-4581-8bf4-1b6158cbe857\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-rjnq2" Jul 07 14:32:12.001745 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:32:12.001722 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7rncb\" (UniqueName: \"kubernetes.io/projected/d9c9a960-6781-4581-8bf4-1b6158cbe857-kube-api-access-7rncb\") pod \"cert-manager-cainjector-559cdd8c87-rjnq2\" (UID: \"d9c9a960-6781-4581-8bf4-1b6158cbe857\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-rjnq2" Jul 07 14:32:12.103228 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:32:12.103190 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d9c9a960-6781-4581-8bf4-1b6158cbe857-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-rjnq2\" (UID: \"d9c9a960-6781-4581-8bf4-1b6158cbe857\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-rjnq2" Jul 07 14:32:12.103402 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:32:12.103306 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7rncb\" (UniqueName: \"kubernetes.io/projected/d9c9a960-6781-4581-8bf4-1b6158cbe857-kube-api-access-7rncb\") pod \"cert-manager-cainjector-559cdd8c87-rjnq2\" (UID: \"d9c9a960-6781-4581-8bf4-1b6158cbe857\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-rjnq2" Jul 07 14:32:12.111557 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:32:12.111508 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d9c9a960-6781-4581-8bf4-1b6158cbe857-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-rjnq2\" (UID: \"d9c9a960-6781-4581-8bf4-1b6158cbe857\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-rjnq2" Jul 07 14:32:12.111688 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:32:12.111588 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7rncb\" (UniqueName: \"kubernetes.io/projected/d9c9a960-6781-4581-8bf4-1b6158cbe857-kube-api-access-7rncb\") pod \"cert-manager-cainjector-559cdd8c87-rjnq2\" (UID: \"d9c9a960-6781-4581-8bf4-1b6158cbe857\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-rjnq2" Jul 07 14:32:12.246718 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:32:12.246630 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-rjnq2" Jul 07 14:32:12.419911 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:32:12.419889 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-rjnq2"] Jul 07 14:32:12.422632 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:32:12.422602 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd9c9a960_6781_4581_8bf4_1b6158cbe857.slice/crio-fc4bdb59117e658eb4e407694e8c8052c16b4f6eb31b377280c308b33dbd733f WatchSource:0}: Error finding container fc4bdb59117e658eb4e407694e8c8052c16b4f6eb31b377280c308b33dbd733f: Status 404 returned error can't find the container with id fc4bdb59117e658eb4e407694e8c8052c16b4f6eb31b377280c308b33dbd733f Jul 07 14:32:12.430262 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:32:12.430233 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fkbb9t" event={"ID":"da22481d-e70b-43d3-90a9-115cda863e5a","Type":"ContainerStarted","Data":"f6f40ccaedc16233ea304acff67a762c63eb66e8e03d29df0f41e98ddb77f30f"} Jul 07 14:32:12.431461 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:32:12.431435 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-rjnq2" event={"ID":"d9c9a960-6781-4581-8bf4-1b6158cbe857","Type":"ContainerStarted","Data":"fc4bdb59117e658eb4e407694e8c8052c16b4f6eb31b377280c308b33dbd733f"} Jul 07 14:32:13.439573 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:32:13.439472 2557 generic.go:356] "Generic (PLEG): container finished" podID="da22481d-e70b-43d3-90a9-115cda863e5a" containerID="f6f40ccaedc16233ea304acff67a762c63eb66e8e03d29df0f41e98ddb77f30f" exitCode=0 Jul 07 14:32:13.439573 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:32:13.439544 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fkbb9t" event={"ID":"da22481d-e70b-43d3-90a9-115cda863e5a","Type":"ContainerDied","Data":"f6f40ccaedc16233ea304acff67a762c63eb66e8e03d29df0f41e98ddb77f30f"} Jul 07 14:32:14.369910 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:32:14.369870 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:32:14.446671 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:32:14.446634 2557 generic.go:356] "Generic (PLEG): container finished" podID="da22481d-e70b-43d3-90a9-115cda863e5a" containerID="512c65a5727807d01fb43538e80f5e8b870dd4859d40f0912727ff47252478f4" exitCode=0 Jul 07 14:32:14.447087 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:32:14.446687 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fkbb9t" event={"ID":"da22481d-e70b-43d3-90a9-115cda863e5a","Type":"ContainerDied","Data":"512c65a5727807d01fb43538e80f5e8b870dd4859d40f0912727ff47252478f4"} Jul 07 14:32:14.469866 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:32:14.469830 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:32:15.454119 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:32:15.453029 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-rjnq2" event={"ID":"d9c9a960-6781-4581-8bf4-1b6158cbe857","Type":"ContainerStarted","Data":"dc2caed213cfd80651d1605827188ab41abca5e3806979ec90a0a4cfbc68595a"} Jul 07 14:32:15.471075 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:32:15.471025 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-rjnq2" podStartSLOduration=1.963120291 podStartE2EDuration="4.47101018s" totalImagesPullingTime="2.507889889s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 14:32:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:32:15.468846237 +0000 UTC m=+372.177984617" watchObservedRunningTime="2026-07-07 14:32:15.47101018 +0000 UTC m=+372.180148560" Jul 07 14:32:15.583297 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:32:15.583272 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fkbb9t" Jul 07 14:32:15.632899 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:32:15.632873 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/da22481d-e70b-43d3-90a9-115cda863e5a-util\") pod \"da22481d-e70b-43d3-90a9-115cda863e5a\" (UID: \"da22481d-e70b-43d3-90a9-115cda863e5a\") " Jul 07 14:32:15.633062 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:32:15.632923 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9l4gt\" (UniqueName: \"kubernetes.io/projected/da22481d-e70b-43d3-90a9-115cda863e5a-kube-api-access-9l4gt\") pod \"da22481d-e70b-43d3-90a9-115cda863e5a\" (UID: \"da22481d-e70b-43d3-90a9-115cda863e5a\") " Jul 07 14:32:15.633062 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:32:15.632955 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/da22481d-e70b-43d3-90a9-115cda863e5a-bundle\") pod \"da22481d-e70b-43d3-90a9-115cda863e5a\" (UID: \"da22481d-e70b-43d3-90a9-115cda863e5a\") " Jul 07 14:32:15.633356 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:32:15.633329 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/da22481d-e70b-43d3-90a9-115cda863e5a-bundle" (OuterVolumeSpecName: "bundle") pod "da22481d-e70b-43d3-90a9-115cda863e5a" (UID: "da22481d-e70b-43d3-90a9-115cda863e5a"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 14:32:15.634939 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:32:15.634914 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/da22481d-e70b-43d3-90a9-115cda863e5a-kube-api-access-9l4gt" (OuterVolumeSpecName: "kube-api-access-9l4gt") pod "da22481d-e70b-43d3-90a9-115cda863e5a" (UID: "da22481d-e70b-43d3-90a9-115cda863e5a"). InnerVolumeSpecName "kube-api-access-9l4gt". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 14:32:15.664678 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:32:15.664651 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/da22481d-e70b-43d3-90a9-115cda863e5a-util" (OuterVolumeSpecName: "util") pod "da22481d-e70b-43d3-90a9-115cda863e5a" (UID: "da22481d-e70b-43d3-90a9-115cda863e5a"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 14:32:15.733939 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:32:15.733865 2557 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/da22481d-e70b-43d3-90a9-115cda863e5a-util\") on node \"ip-10-0-134-42.ec2.internal\" DevicePath \"\"" Jul 07 14:32:15.733939 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:32:15.733891 2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9l4gt\" (UniqueName: \"kubernetes.io/projected/da22481d-e70b-43d3-90a9-115cda863e5a-kube-api-access-9l4gt\") on node \"ip-10-0-134-42.ec2.internal\" DevicePath \"\"" Jul 07 14:32:15.733939 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:32:15.733901 2557 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/da22481d-e70b-43d3-90a9-115cda863e5a-bundle\") on node \"ip-10-0-134-42.ec2.internal\" DevicePath \"\"" Jul 07 14:32:16.458853 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:32:16.458825 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fkbb9t" Jul 07 14:32:16.458853 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:32:16.458838 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fkbb9t" event={"ID":"da22481d-e70b-43d3-90a9-115cda863e5a","Type":"ContainerDied","Data":"da54e93ddbf6585609f6a78f78b5d23cfb530678690d4da2b37388c79c3f3829"} Jul 07 14:32:16.459406 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:32:16.458885 2557 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="da54e93ddbf6585609f6a78f78b5d23cfb530678690d4da2b37388c79c3f3829" Jul 07 14:32:19.140199 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:32:19.140167 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:32:20.414776 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:32:20.414742 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:32:20.504648 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:32:20.504615 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:32:21.776702 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:32:21.776669 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-cfvfb"] Jul 07 14:32:21.777065 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:32:21.776966 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="da22481d-e70b-43d3-90a9-115cda863e5a" containerName="pull" Jul 07 14:32:21.777065 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:32:21.776977 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="da22481d-e70b-43d3-90a9-115cda863e5a" containerName="pull" Jul 07 14:32:21.777065 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:32:21.776989 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="da22481d-e70b-43d3-90a9-115cda863e5a" containerName="util" Jul 07 14:32:21.777065 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:32:21.776994 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="da22481d-e70b-43d3-90a9-115cda863e5a" containerName="util" Jul 07 14:32:21.777065 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:32:21.777002 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="da22481d-e70b-43d3-90a9-115cda863e5a" containerName="extract" Jul 07 14:32:21.777065 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:32:21.777008 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="da22481d-e70b-43d3-90a9-115cda863e5a" containerName="extract" Jul 07 14:32:21.777065 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:32:21.777060 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="da22481d-e70b-43d3-90a9-115cda863e5a" containerName="extract" Jul 07 14:32:21.779042 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:32:21.779025 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-cfvfb" Jul 07 14:32:21.780991 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:32:21.780972 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-hls4q\"" Jul 07 14:32:21.791247 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:32:21.791224 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-cfvfb"] Jul 07 14:32:21.880566 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:32:21.880513 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/04520229-760d-49ae-9388-6dbd4b425f75-bound-sa-token\") pod \"cert-manager-55b4d954d5-cfvfb\" (UID: \"04520229-760d-49ae-9388-6dbd4b425f75\") " pod="cert-manager/cert-manager-55b4d954d5-cfvfb" Jul 07 14:32:21.880752 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:32:21.880576 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ttskp\" (UniqueName: \"kubernetes.io/projected/04520229-760d-49ae-9388-6dbd4b425f75-kube-api-access-ttskp\") pod \"cert-manager-55b4d954d5-cfvfb\" (UID: \"04520229-760d-49ae-9388-6dbd4b425f75\") " pod="cert-manager/cert-manager-55b4d954d5-cfvfb" Jul 07 14:32:21.981077 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:32:21.981045 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/04520229-760d-49ae-9388-6dbd4b425f75-bound-sa-token\") pod \"cert-manager-55b4d954d5-cfvfb\" (UID: \"04520229-760d-49ae-9388-6dbd4b425f75\") " pod="cert-manager/cert-manager-55b4d954d5-cfvfb" Jul 07 14:32:21.981217 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:32:21.981081 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ttskp\" (UniqueName: \"kubernetes.io/projected/04520229-760d-49ae-9388-6dbd4b425f75-kube-api-access-ttskp\") pod \"cert-manager-55b4d954d5-cfvfb\" (UID: \"04520229-760d-49ae-9388-6dbd4b425f75\") " pod="cert-manager/cert-manager-55b4d954d5-cfvfb" Jul 07 14:32:21.988895 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:32:21.988862 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/04520229-760d-49ae-9388-6dbd4b425f75-bound-sa-token\") pod \"cert-manager-55b4d954d5-cfvfb\" (UID: \"04520229-760d-49ae-9388-6dbd4b425f75\") " pod="cert-manager/cert-manager-55b4d954d5-cfvfb" Jul 07 14:32:21.988975 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:32:21.988952 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ttskp\" (UniqueName: \"kubernetes.io/projected/04520229-760d-49ae-9388-6dbd4b425f75-kube-api-access-ttskp\") pod \"cert-manager-55b4d954d5-cfvfb\" (UID: \"04520229-760d-49ae-9388-6dbd4b425f75\") " pod="cert-manager/cert-manager-55b4d954d5-cfvfb" Jul 07 14:32:22.088147 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:32:22.088127 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-cfvfb" Jul 07 14:32:22.213777 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:32:22.213748 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-cfvfb"] Jul 07 14:32:22.216650 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:32:22.216624 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod04520229_760d_49ae_9388_6dbd4b425f75.slice/crio-09040aa70fbad186e6f4d1fa9484d9cfbac7dd71b30eaa9b900a607e04ef388f WatchSource:0}: Error finding container 09040aa70fbad186e6f4d1fa9484d9cfbac7dd71b30eaa9b900a607e04ef388f: Status 404 returned error can't find the container with id 09040aa70fbad186e6f4d1fa9484d9cfbac7dd71b30eaa9b900a607e04ef388f Jul 07 14:32:22.485115 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:32:22.485022 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-cfvfb" event={"ID":"04520229-760d-49ae-9388-6dbd4b425f75","Type":"ContainerStarted","Data":"f2a545db3e99959c40cd271f79d4737456c5c901b288701ed44c952467d027fe"} Jul 07 14:32:22.485115 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:32:22.485064 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-cfvfb" event={"ID":"04520229-760d-49ae-9388-6dbd4b425f75","Type":"ContainerStarted","Data":"09040aa70fbad186e6f4d1fa9484d9cfbac7dd71b30eaa9b900a607e04ef388f"} Jul 07 14:32:22.501849 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:32:22.501798 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-cfvfb" podStartSLOduration=1.5017817089999999 podStartE2EDuration="1.501781709s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 14:32:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:32:22.499894972 +0000 UTC m=+379.209033353" watchObservedRunningTime="2026-07-07 14:32:22.501781709 +0000 UTC m=+379.210920090" Jul 07 14:32:22.615755 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:32:22.615720 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-79j49"] Jul 07 14:32:22.617992 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:32:22.617970 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-79j49" Jul 07 14:32:22.619966 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:32:22.619945 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 07 14:32:22.619966 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:32:22.619970 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 07 14:32:22.620185 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:32:22.620172 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-wttsz\"" Jul 07 14:32:22.630260 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:32:22.630240 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-79j49"] Jul 07 14:32:22.687299 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:32:22.687268 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ts5cj\" (UniqueName: \"kubernetes.io/projected/14c8a9df-2859-42c0-9e42-2036662ba764-kube-api-access-ts5cj\") pod \"openshift-lws-operator-fd8ccff4c-79j49\" (UID: \"14c8a9df-2859-42c0-9e42-2036662ba764\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-79j49" Jul 07 14:32:22.687442 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:32:22.687309 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/14c8a9df-2859-42c0-9e42-2036662ba764-tmp\") pod \"openshift-lws-operator-fd8ccff4c-79j49\" (UID: \"14c8a9df-2859-42c0-9e42-2036662ba764\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-79j49" Jul 07 14:32:22.788005 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:32:22.787925 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ts5cj\" (UniqueName: \"kubernetes.io/projected/14c8a9df-2859-42c0-9e42-2036662ba764-kube-api-access-ts5cj\") pod \"openshift-lws-operator-fd8ccff4c-79j49\" (UID: \"14c8a9df-2859-42c0-9e42-2036662ba764\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-79j49" Jul 07 14:32:22.788005 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:32:22.787972 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/14c8a9df-2859-42c0-9e42-2036662ba764-tmp\") pod \"openshift-lws-operator-fd8ccff4c-79j49\" (UID: \"14c8a9df-2859-42c0-9e42-2036662ba764\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-79j49" Jul 07 14:32:22.788344 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:32:22.788330 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/14c8a9df-2859-42c0-9e42-2036662ba764-tmp\") pod \"openshift-lws-operator-fd8ccff4c-79j49\" (UID: \"14c8a9df-2859-42c0-9e42-2036662ba764\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-79j49" Jul 07 14:32:22.797968 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:32:22.797944 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ts5cj\" (UniqueName: \"kubernetes.io/projected/14c8a9df-2859-42c0-9e42-2036662ba764-kube-api-access-ts5cj\") pod \"openshift-lws-operator-fd8ccff4c-79j49\" (UID: \"14c8a9df-2859-42c0-9e42-2036662ba764\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-79j49" Jul 07 14:32:22.926442 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:32:22.926420 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-79j49" Jul 07 14:32:23.050406 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:32:23.050335 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-79j49"] Jul 07 14:32:23.053781 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:32:23.053758 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod14c8a9df_2859_42c0_9e42_2036662ba764.slice/crio-77971ecd5273032635d580f32d5576d3c70d6f2a60c30fc268124bc9f945309f WatchSource:0}: Error finding container 77971ecd5273032635d580f32d5576d3c70d6f2a60c30fc268124bc9f945309f: Status 404 returned error can't find the container with id 77971ecd5273032635d580f32d5576d3c70d6f2a60c30fc268124bc9f945309f Jul 07 14:32:23.489774 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:32:23.489743 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-79j49" event={"ID":"14c8a9df-2859-42c0-9e42-2036662ba764","Type":"ContainerStarted","Data":"77971ecd5273032635d580f32d5576d3c70d6f2a60c30fc268124bc9f945309f"} Jul 07 14:32:25.500916 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:32:25.500876 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-79j49" event={"ID":"14c8a9df-2859-42c0-9e42-2036662ba764","Type":"ContainerStarted","Data":"ab08a0969c7c9b54cf4ecad973cf0b517089d6596bf6f278db863404857f23c0"} Jul 07 14:32:25.517162 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:32:25.517106 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-79j49" podStartSLOduration=1.72337329 podStartE2EDuration="3.517086717s" totalImagesPullingTime="1.793713427s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 14:32:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:32:25.515545489 +0000 UTC m=+382.224683870" watchObservedRunningTime="2026-07-07 14:32:25.517086717 +0000 UTC m=+382.226225098" Jul 07 14:32:26.161323 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:32:26.161283 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:32:26.221620 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:32:26.221593 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:32:28.481425 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:32:28.481392 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:32:30.476333 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:32:30.476293 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afm298r"] Jul 07 14:32:30.492556 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:32:30.492507 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afm298r"] Jul 07 14:32:30.492704 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:32:30.492637 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afm298r" Jul 07 14:32:30.494877 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:32:30.494855 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 07 14:32:30.495381 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:32:30.495365 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 07 14:32:30.495469 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:32:30.495368 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-kqlhz\"" Jul 07 14:32:30.649535 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:32:30.649483 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7edff8a3-0efd-4e39-a3cb-a945fc83bfbd-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afm298r\" (UID: \"7edff8a3-0efd-4e39-a3cb-a945fc83bfbd\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afm298r" Jul 07 14:32:30.649690 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:32:30.649549 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7edff8a3-0efd-4e39-a3cb-a945fc83bfbd-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afm298r\" (UID: \"7edff8a3-0efd-4e39-a3cb-a945fc83bfbd\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afm298r" Jul 07 14:32:30.649690 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:32:30.649607 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hgjtk\" (UniqueName: \"kubernetes.io/projected/7edff8a3-0efd-4e39-a3cb-a945fc83bfbd-kube-api-access-hgjtk\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afm298r\" (UID: \"7edff8a3-0efd-4e39-a3cb-a945fc83bfbd\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afm298r" Jul 07 14:32:30.750887 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:32:30.750818 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7edff8a3-0efd-4e39-a3cb-a945fc83bfbd-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afm298r\" (UID: \"7edff8a3-0efd-4e39-a3cb-a945fc83bfbd\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afm298r" Jul 07 14:32:30.750991 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:32:30.750885 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7edff8a3-0efd-4e39-a3cb-a945fc83bfbd-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afm298r\" (UID: \"7edff8a3-0efd-4e39-a3cb-a945fc83bfbd\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afm298r" Jul 07 14:32:30.750991 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:32:30.750915 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hgjtk\" (UniqueName: \"kubernetes.io/projected/7edff8a3-0efd-4e39-a3cb-a945fc83bfbd-kube-api-access-hgjtk\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afm298r\" (UID: \"7edff8a3-0efd-4e39-a3cb-a945fc83bfbd\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afm298r" Jul 07 14:32:30.751180 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:32:30.751161 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7edff8a3-0efd-4e39-a3cb-a945fc83bfbd-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afm298r\" (UID: \"7edff8a3-0efd-4e39-a3cb-a945fc83bfbd\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afm298r" Jul 07 14:32:30.751245 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:32:30.751199 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7edff8a3-0efd-4e39-a3cb-a945fc83bfbd-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afm298r\" (UID: \"7edff8a3-0efd-4e39-a3cb-a945fc83bfbd\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afm298r" Jul 07 14:32:30.760775 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:32:30.760752 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hgjtk\" (UniqueName: \"kubernetes.io/projected/7edff8a3-0efd-4e39-a3cb-a945fc83bfbd-kube-api-access-hgjtk\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afm298r\" (UID: \"7edff8a3-0efd-4e39-a3cb-a945fc83bfbd\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afm298r" Jul 07 14:32:30.802711 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:32:30.802689 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afm298r" Jul 07 14:32:30.949143 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:32:30.949112 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afm298r"] Jul 07 14:32:30.951795 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:32:30.951765 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7edff8a3_0efd_4e39_a3cb_a945fc83bfbd.slice/crio-c596f49f99204e8025f56e1ee0a2519a2779b00268d728db25eb97152af0ed48 WatchSource:0}: Error finding container c596f49f99204e8025f56e1ee0a2519a2779b00268d728db25eb97152af0ed48: Status 404 returned error can't find the container with id c596f49f99204e8025f56e1ee0a2519a2779b00268d728db25eb97152af0ed48 Jul 07 14:32:31.528848 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:32:31.528817 2557 generic.go:356] "Generic (PLEG): container finished" podID="7edff8a3-0efd-4e39-a3cb-a945fc83bfbd" containerID="7dbbd4e11eea49e2ca8c6114a86dd5930ec422efec52124ac1db507813fa694e" exitCode=0 Jul 07 14:32:31.529221 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:32:31.528883 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afm298r" event={"ID":"7edff8a3-0efd-4e39-a3cb-a945fc83bfbd","Type":"ContainerDied","Data":"7dbbd4e11eea49e2ca8c6114a86dd5930ec422efec52124ac1db507813fa694e"} Jul 07 14:32:31.529221 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:32:31.528923 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afm298r" event={"ID":"7edff8a3-0efd-4e39-a3cb-a945fc83bfbd","Type":"ContainerStarted","Data":"c596f49f99204e8025f56e1ee0a2519a2779b00268d728db25eb97152af0ed48"} Jul 07 14:32:31.848969 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:32:31.848936 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:32:31.965200 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:32:31.965163 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:32:32.535369 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:32:32.535282 2557 generic.go:356] "Generic (PLEG): container finished" podID="7edff8a3-0efd-4e39-a3cb-a945fc83bfbd" containerID="11a6a9c2224658109160f73d4681755b206a17fec3a77161dd032c39972d1106" exitCode=0 Jul 07 14:32:32.535744 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:32:32.535371 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afm298r" event={"ID":"7edff8a3-0efd-4e39-a3cb-a945fc83bfbd","Type":"ContainerDied","Data":"11a6a9c2224658109160f73d4681755b206a17fec3a77161dd032c39972d1106"} Jul 07 14:32:33.543272 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:32:33.543231 2557 generic.go:356] "Generic (PLEG): container finished" podID="7edff8a3-0efd-4e39-a3cb-a945fc83bfbd" containerID="f4050608341790aaf9601af7c72dab072f67b6144eaada0a26d23b099eedd778" exitCode=0 Jul 07 14:32:33.543751 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:32:33.543316 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afm298r" event={"ID":"7edff8a3-0efd-4e39-a3cb-a945fc83bfbd","Type":"ContainerDied","Data":"f4050608341790aaf9601af7c72dab072f67b6144eaada0a26d23b099eedd778"} Jul 07 14:32:34.675200 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:32:34.675172 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afm298r" Jul 07 14:32:34.778825 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:32:34.778799 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7edff8a3-0efd-4e39-a3cb-a945fc83bfbd-bundle\") pod \"7edff8a3-0efd-4e39-a3cb-a945fc83bfbd\" (UID: \"7edff8a3-0efd-4e39-a3cb-a945fc83bfbd\") " Jul 07 14:32:34.778957 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:32:34.778851 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7edff8a3-0efd-4e39-a3cb-a945fc83bfbd-util\") pod \"7edff8a3-0efd-4e39-a3cb-a945fc83bfbd\" (UID: \"7edff8a3-0efd-4e39-a3cb-a945fc83bfbd\") " Jul 07 14:32:34.778957 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:32:34.778872 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hgjtk\" (UniqueName: \"kubernetes.io/projected/7edff8a3-0efd-4e39-a3cb-a945fc83bfbd-kube-api-access-hgjtk\") pod \"7edff8a3-0efd-4e39-a3cb-a945fc83bfbd\" (UID: \"7edff8a3-0efd-4e39-a3cb-a945fc83bfbd\") " Jul 07 14:32:34.779513 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:32:34.779489 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7edff8a3-0efd-4e39-a3cb-a945fc83bfbd-bundle" (OuterVolumeSpecName: "bundle") pod "7edff8a3-0efd-4e39-a3cb-a945fc83bfbd" (UID: "7edff8a3-0efd-4e39-a3cb-a945fc83bfbd"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 14:32:34.780868 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:32:34.780844 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7edff8a3-0efd-4e39-a3cb-a945fc83bfbd-kube-api-access-hgjtk" (OuterVolumeSpecName: "kube-api-access-hgjtk") pod "7edff8a3-0efd-4e39-a3cb-a945fc83bfbd" (UID: "7edff8a3-0efd-4e39-a3cb-a945fc83bfbd"). InnerVolumeSpecName "kube-api-access-hgjtk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 14:32:34.784288 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:32:34.784268 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7edff8a3-0efd-4e39-a3cb-a945fc83bfbd-util" (OuterVolumeSpecName: "util") pod "7edff8a3-0efd-4e39-a3cb-a945fc83bfbd" (UID: "7edff8a3-0efd-4e39-a3cb-a945fc83bfbd"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 14:32:34.879879 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:32:34.879846 2557 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7edff8a3-0efd-4e39-a3cb-a945fc83bfbd-bundle\") on node \"ip-10-0-134-42.ec2.internal\" DevicePath \"\"" Jul 07 14:32:34.879879 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:32:34.879874 2557 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7edff8a3-0efd-4e39-a3cb-a945fc83bfbd-util\") on node \"ip-10-0-134-42.ec2.internal\" DevicePath \"\"" Jul 07 14:32:34.879879 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:32:34.879882 2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hgjtk\" (UniqueName: \"kubernetes.io/projected/7edff8a3-0efd-4e39-a3cb-a945fc83bfbd-kube-api-access-hgjtk\") on node \"ip-10-0-134-42.ec2.internal\" DevicePath \"\"" Jul 07 14:32:35.553645 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:32:35.553620 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afm298r" Jul 07 14:32:35.553912 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:32:35.553644 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afm298r" event={"ID":"7edff8a3-0efd-4e39-a3cb-a945fc83bfbd","Type":"ContainerDied","Data":"c596f49f99204e8025f56e1ee0a2519a2779b00268d728db25eb97152af0ed48"} Jul 07 14:32:35.553912 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:32:35.553688 2557 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c596f49f99204e8025f56e1ee0a2519a2779b00268d728db25eb97152af0ed48" Jul 07 14:32:42.588627 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:32:42.588590 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:32:42.700047 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:32:42.700012 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:32:43.229004 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:32:43.228967 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:32:43.412255 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:32:43.412227 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:32:43.512576 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:32:43.512482 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:32:43.598688 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:32:43.598657 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7868f6cbb6-t7m9p"] Jul 07 14:32:43.599031 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:32:43.598955 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7edff8a3-0efd-4e39-a3cb-a945fc83bfbd" containerName="pull" Jul 07 14:32:43.599031 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:32:43.598966 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="7edff8a3-0efd-4e39-a3cb-a945fc83bfbd" containerName="pull" Jul 07 14:32:43.599031 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:32:43.598977 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7edff8a3-0efd-4e39-a3cb-a945fc83bfbd" containerName="util" Jul 07 14:32:43.599031 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:32:43.598982 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="7edff8a3-0efd-4e39-a3cb-a945fc83bfbd" containerName="util" Jul 07 14:32:43.599031 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:32:43.598991 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7edff8a3-0efd-4e39-a3cb-a945fc83bfbd" containerName="extract" Jul 07 14:32:43.599031 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:32:43.598997 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="7edff8a3-0efd-4e39-a3cb-a945fc83bfbd" containerName="extract" Jul 07 14:32:43.599247 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:32:43.599061 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="7edff8a3-0efd-4e39-a3cb-a945fc83bfbd" containerName="extract" Jul 07 14:32:43.601996 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:32:43.601979 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-7868f6cbb6-t7m9p" Jul 07 14:32:43.603951 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:32:43.603928 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 07 14:32:43.605380 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:32:43.605364 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 07 14:32:43.606339 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:32:43.606325 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-g2dhx\"" Jul 07 14:32:43.606600 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:32:43.606586 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 07 14:32:43.606651 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:32:43.606638 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 07 14:32:43.638918 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:32:43.638738 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7868f6cbb6-t7m9p"] Jul 07 14:32:43.748667 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:32:43.748633 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/344eede0-1be4-4e5e-87b9-81aae9469040-webhook-cert\") pod \"opendatahub-operator-controller-manager-7868f6cbb6-t7m9p\" (UID: \"344eede0-1be4-4e5e-87b9-81aae9469040\") " pod="opendatahub/opendatahub-operator-controller-manager-7868f6cbb6-t7m9p" Jul 07 14:32:43.748667 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:32:43.748677 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/344eede0-1be4-4e5e-87b9-81aae9469040-tmp\") pod \"opendatahub-operator-controller-manager-7868f6cbb6-t7m9p\" (UID: \"344eede0-1be4-4e5e-87b9-81aae9469040\") " pod="opendatahub/opendatahub-operator-controller-manager-7868f6cbb6-t7m9p" Jul 07 14:32:43.748882 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:32:43.748720 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/344eede0-1be4-4e5e-87b9-81aae9469040-manifests\") pod \"opendatahub-operator-controller-manager-7868f6cbb6-t7m9p\" (UID: \"344eede0-1be4-4e5e-87b9-81aae9469040\") " pod="opendatahub/opendatahub-operator-controller-manager-7868f6cbb6-t7m9p" Jul 07 14:32:43.748882 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:32:43.748807 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p4r6s\" (UniqueName: \"kubernetes.io/projected/344eede0-1be4-4e5e-87b9-81aae9469040-kube-api-access-p4r6s\") pod \"opendatahub-operator-controller-manager-7868f6cbb6-t7m9p\" (UID: \"344eede0-1be4-4e5e-87b9-81aae9469040\") " pod="opendatahub/opendatahub-operator-controller-manager-7868f6cbb6-t7m9p" Jul 07 14:32:43.748882 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:32:43.748841 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/344eede0-1be4-4e5e-87b9-81aae9469040-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7868f6cbb6-t7m9p\" (UID: \"344eede0-1be4-4e5e-87b9-81aae9469040\") " pod="opendatahub/opendatahub-operator-controller-manager-7868f6cbb6-t7m9p" Jul 07 14:32:43.793670 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:32:43.793587 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:32:43.849604 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:32:43.849574 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p4r6s\" (UniqueName: \"kubernetes.io/projected/344eede0-1be4-4e5e-87b9-81aae9469040-kube-api-access-p4r6s\") pod \"opendatahub-operator-controller-manager-7868f6cbb6-t7m9p\" (UID: \"344eede0-1be4-4e5e-87b9-81aae9469040\") " pod="opendatahub/opendatahub-operator-controller-manager-7868f6cbb6-t7m9p" Jul 07 14:32:43.849748 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:32:43.849609 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/344eede0-1be4-4e5e-87b9-81aae9469040-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7868f6cbb6-t7m9p\" (UID: \"344eede0-1be4-4e5e-87b9-81aae9469040\") " pod="opendatahub/opendatahub-operator-controller-manager-7868f6cbb6-t7m9p" Jul 07 14:32:43.849748 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:32:43.849731 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/344eede0-1be4-4e5e-87b9-81aae9469040-webhook-cert\") pod \"opendatahub-operator-controller-manager-7868f6cbb6-t7m9p\" (UID: \"344eede0-1be4-4e5e-87b9-81aae9469040\") " pod="opendatahub/opendatahub-operator-controller-manager-7868f6cbb6-t7m9p" Jul 07 14:32:43.849834 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:32:43.849763 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/344eede0-1be4-4e5e-87b9-81aae9469040-tmp\") pod \"opendatahub-operator-controller-manager-7868f6cbb6-t7m9p\" (UID: \"344eede0-1be4-4e5e-87b9-81aae9469040\") " pod="opendatahub/opendatahub-operator-controller-manager-7868f6cbb6-t7m9p" Jul 07 14:32:43.849834 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:32:43.849813 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/344eede0-1be4-4e5e-87b9-81aae9469040-manifests\") pod \"opendatahub-operator-controller-manager-7868f6cbb6-t7m9p\" (UID: \"344eede0-1be4-4e5e-87b9-81aae9469040\") " pod="opendatahub/opendatahub-operator-controller-manager-7868f6cbb6-t7m9p" Jul 07 14:32:43.850176 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:32:43.850155 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/344eede0-1be4-4e5e-87b9-81aae9469040-manifests\") pod \"opendatahub-operator-controller-manager-7868f6cbb6-t7m9p\" (UID: \"344eede0-1be4-4e5e-87b9-81aae9469040\") " pod="opendatahub/opendatahub-operator-controller-manager-7868f6cbb6-t7m9p" Jul 07 14:32:43.851871 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:32:43.851839 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/344eede0-1be4-4e5e-87b9-81aae9469040-tmp\") pod \"opendatahub-operator-controller-manager-7868f6cbb6-t7m9p\" (UID: \"344eede0-1be4-4e5e-87b9-81aae9469040\") " pod="opendatahub/opendatahub-operator-controller-manager-7868f6cbb6-t7m9p" Jul 07 14:32:43.852111 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:32:43.852090 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/344eede0-1be4-4e5e-87b9-81aae9469040-webhook-cert\") pod \"opendatahub-operator-controller-manager-7868f6cbb6-t7m9p\" (UID: \"344eede0-1be4-4e5e-87b9-81aae9469040\") " pod="opendatahub/opendatahub-operator-controller-manager-7868f6cbb6-t7m9p" Jul 07 14:32:43.852179 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:32:43.852124 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/344eede0-1be4-4e5e-87b9-81aae9469040-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7868f6cbb6-t7m9p\" (UID: \"344eede0-1be4-4e5e-87b9-81aae9469040\") " pod="opendatahub/opendatahub-operator-controller-manager-7868f6cbb6-t7m9p" Jul 07 14:32:43.858398 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:32:43.858377 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p4r6s\" (UniqueName: \"kubernetes.io/projected/344eede0-1be4-4e5e-87b9-81aae9469040-kube-api-access-p4r6s\") pod \"opendatahub-operator-controller-manager-7868f6cbb6-t7m9p\" (UID: \"344eede0-1be4-4e5e-87b9-81aae9469040\") " pod="opendatahub/opendatahub-operator-controller-manager-7868f6cbb6-t7m9p" Jul 07 14:32:43.894023 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:32:43.893999 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:32:43.911860 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:32:43.911837 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-7868f6cbb6-t7m9p" Jul 07 14:32:44.016786 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:32:44.016737 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:32:44.066024 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:32:44.065964 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7868f6cbb6-t7m9p"] Jul 07 14:32:44.068158 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:32:44.068131 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod344eede0_1be4_4e5e_87b9_81aae9469040.slice/crio-729411523fd307829685e78fb7d16729faae16954639422ef1df9047a9467dfc WatchSource:0}: Error finding container 729411523fd307829685e78fb7d16729faae16954639422ef1df9047a9467dfc: Status 404 returned error can't find the container with id 729411523fd307829685e78fb7d16729faae16954639422ef1df9047a9467dfc Jul 07 14:32:44.141345 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:32:44.141316 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:32:44.244810 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:32:44.244783 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:32:44.341766 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:32:44.341724 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:32:44.598043 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:32:44.597963 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7868f6cbb6-t7m9p" event={"ID":"344eede0-1be4-4e5e-87b9-81aae9469040","Type":"ContainerStarted","Data":"729411523fd307829685e78fb7d16729faae16954639422ef1df9047a9467dfc"} Jul 07 14:32:48.620192 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:32:48.620156 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7868f6cbb6-t7m9p" event={"ID":"344eede0-1be4-4e5e-87b9-81aae9469040","Type":"ContainerStarted","Data":"6b028df97d25366d3d65278c45d8885bb704997040acb47abf1fbe912f254ade"} Jul 07 14:32:49.626134 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:32:49.626097 2557 generic.go:356] "Generic (PLEG): container finished" podID="344eede0-1be4-4e5e-87b9-81aae9469040" containerID="6b028df97d25366d3d65278c45d8885bb704997040acb47abf1fbe912f254ade" exitCode=0 Jul 07 14:32:49.626627 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:32:49.626153 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7868f6cbb6-t7m9p" event={"ID":"344eede0-1be4-4e5e-87b9-81aae9469040","Type":"ContainerDied","Data":"6b028df97d25366d3d65278c45d8885bb704997040acb47abf1fbe912f254ade"} Jul 07 14:32:51.636461 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:32:51.636419 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7868f6cbb6-t7m9p" event={"ID":"344eede0-1be4-4e5e-87b9-81aae9469040","Type":"ContainerStarted","Data":"426284b43dc021cab707a642a4ee3186073d30cae8f13964b635ec5eae73002b"} Jul 07 14:32:51.636850 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:32:51.636646 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-7868f6cbb6-t7m9p" Jul 07 14:32:51.657844 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:32:51.657802 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-7868f6cbb6-t7m9p" podStartSLOduration=2.952266881 podStartE2EDuration="8.657786696s" totalImagesPullingTime="4.632504553s" totalInitContainerRuntime="1.073015262s" isStatefulPod=false podCreationTimestamp="2026-07-07 14:32:43 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:32:51.656498862 +0000 UTC m=+408.365637264" watchObservedRunningTime="2026-07-07 14:32:51.657786696 +0000 UTC m=+408.366925076" Jul 07 14:32:54.749859 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:32:54.749819 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:32:54.850745 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:32:54.850709 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:33:02.643681 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:33:02.643649 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-7868f6cbb6-t7m9p" Jul 07 14:33:04.364770 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:33:04.364730 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:33:04.485233 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:33:04.485192 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:33:04.961133 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:33:04.961084 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:33:05.080597 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:33:05.080570 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-7646845f6c-hcxf5"] Jul 07 14:33:05.085965 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:33:05.085941 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-7646845f6c-hcxf5" Jul 07 14:33:05.095179 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:33:05.095160 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-mz56n\"" Jul 07 14:33:05.095316 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:33:05.095290 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Jul 07 14:33:05.095463 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:33:05.095441 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Jul 07 14:33:05.104600 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:33:05.104577 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:33:05.119766 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:33:05.119742 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-7646845f6c-hcxf5"] Jul 07 14:33:05.126551 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:33:05.126530 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1da3f1a9-9803-42bc-b609-8fdd84a03972-tls-certs\") pod \"kube-auth-proxy-7646845f6c-hcxf5\" (UID: \"1da3f1a9-9803-42bc-b609-8fdd84a03972\") " pod="openshift-ingress/kube-auth-proxy-7646845f6c-hcxf5" Jul 07 14:33:05.126653 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:33:05.126565 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rf9wd\" (UniqueName: \"kubernetes.io/projected/1da3f1a9-9803-42bc-b609-8fdd84a03972-kube-api-access-rf9wd\") pod \"kube-auth-proxy-7646845f6c-hcxf5\" (UID: \"1da3f1a9-9803-42bc-b609-8fdd84a03972\") " pod="openshift-ingress/kube-auth-proxy-7646845f6c-hcxf5" Jul 07 14:33:05.126653 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:33:05.126583 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1da3f1a9-9803-42bc-b609-8fdd84a03972-tmp\") pod \"kube-auth-proxy-7646845f6c-hcxf5\" (UID: \"1da3f1a9-9803-42bc-b609-8fdd84a03972\") " pod="openshift-ingress/kube-auth-proxy-7646845f6c-hcxf5" Jul 07 14:33:05.227345 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:33:05.227270 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1da3f1a9-9803-42bc-b609-8fdd84a03972-tls-certs\") pod \"kube-auth-proxy-7646845f6c-hcxf5\" (UID: \"1da3f1a9-9803-42bc-b609-8fdd84a03972\") " pod="openshift-ingress/kube-auth-proxy-7646845f6c-hcxf5" Jul 07 14:33:05.227345 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:33:05.227316 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rf9wd\" (UniqueName: \"kubernetes.io/projected/1da3f1a9-9803-42bc-b609-8fdd84a03972-kube-api-access-rf9wd\") pod \"kube-auth-proxy-7646845f6c-hcxf5\" (UID: \"1da3f1a9-9803-42bc-b609-8fdd84a03972\") " pod="openshift-ingress/kube-auth-proxy-7646845f6c-hcxf5" Jul 07 14:33:05.227345 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:33:05.227335 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1da3f1a9-9803-42bc-b609-8fdd84a03972-tmp\") pod \"kube-auth-proxy-7646845f6c-hcxf5\" (UID: \"1da3f1a9-9803-42bc-b609-8fdd84a03972\") " pod="openshift-ingress/kube-auth-proxy-7646845f6c-hcxf5" Jul 07 14:33:05.229842 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:33:05.229817 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1da3f1a9-9803-42bc-b609-8fdd84a03972-tmp\") pod \"kube-auth-proxy-7646845f6c-hcxf5\" (UID: \"1da3f1a9-9803-42bc-b609-8fdd84a03972\") " pod="openshift-ingress/kube-auth-proxy-7646845f6c-hcxf5" Jul 07 14:33:05.229969 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:33:05.229838 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:33:05.229969 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:33:05.229946 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1da3f1a9-9803-42bc-b609-8fdd84a03972-tls-certs\") pod \"kube-auth-proxy-7646845f6c-hcxf5\" (UID: \"1da3f1a9-9803-42bc-b609-8fdd84a03972\") " pod="openshift-ingress/kube-auth-proxy-7646845f6c-hcxf5" Jul 07 14:33:05.235801 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:33:05.235778 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rf9wd\" (UniqueName: \"kubernetes.io/projected/1da3f1a9-9803-42bc-b609-8fdd84a03972-kube-api-access-rf9wd\") pod \"kube-auth-proxy-7646845f6c-hcxf5\" (UID: \"1da3f1a9-9803-42bc-b609-8fdd84a03972\") " pod="openshift-ingress/kube-auth-proxy-7646845f6c-hcxf5" Jul 07 14:33:05.315555 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:33:05.315527 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:33:05.396955 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:33:05.396918 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-7646845f6c-hcxf5" Jul 07 14:33:05.471587 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:33:05.471507 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:33:05.534964 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:33:05.534929 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-7646845f6c-hcxf5"] Jul 07 14:33:05.537725 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:33:05.537695 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1da3f1a9_9803_42bc_b609_8fdd84a03972.slice/crio-0db356fe0bda0ebe0ba84cf86024fa8179bc47c565703f5e2c6f5120e1a2680c WatchSource:0}: Error finding container 0db356fe0bda0ebe0ba84cf86024fa8179bc47c565703f5e2c6f5120e1a2680c: Status 404 returned error can't find the container with id 0db356fe0bda0ebe0ba84cf86024fa8179bc47c565703f5e2c6f5120e1a2680c Jul 07 14:33:05.597600 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:33:05.597555 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:33:05.713412 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:33:05.713383 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-7646845f6c-hcxf5" event={"ID":"1da3f1a9-9803-42bc-b609-8fdd84a03972","Type":"ContainerStarted","Data":"0db356fe0bda0ebe0ba84cf86024fa8179bc47c565703f5e2c6f5120e1a2680c"} Jul 07 14:33:06.039366 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:33:06.039313 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:33:06.327821 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:33:06.327795 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:33:06.606019 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:33:06.605913 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:33:06.679350 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:33:06.679311 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:33:07.643056 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:33:07.643000 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:33:08.728791 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:33:08.728755 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-7646845f6c-hcxf5" event={"ID":"1da3f1a9-9803-42bc-b609-8fdd84a03972","Type":"ContainerStarted","Data":"08ed25fee45d27cdc8349e2097992d5e4ab014f8f82a72fbc1306a506801b93b"} Jul 07 14:33:08.749309 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:33:08.749252 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-7646845f6c-hcxf5" podStartSLOduration=0.988941214 podStartE2EDuration="3.74923255s" totalImagesPullingTime="2.760291336s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 14:33:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:33:08.746341968 +0000 UTC m=+425.455480350" watchObservedRunningTime="2026-07-07 14:33:08.74923255 +0000 UTC m=+425.458370931" Jul 07 14:33:12.319419 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:33:12.319384 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:33:12.446337 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:33:12.446305 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:33:12.591471 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:33:12.591441 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-vffwj"] Jul 07 14:33:12.594966 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:33:12.594947 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-vffwj" Jul 07 14:33:12.596881 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:33:12.596856 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Jul 07 14:33:12.596995 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:33:12.596956 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Jul 07 14:33:12.597243 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:33:12.597221 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 07 14:33:12.597356 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:33:12.597232 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Jul 07 14:33:12.597771 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:33:12.597754 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-m8nrh\"" Jul 07 14:33:12.609500 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:33:12.609475 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-vffwj"] Jul 07 14:33:12.642313 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:33:12.642287 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:33:12.686966 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:33:12.686940 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/d4731d32-8190-4426-b1f0-d66f3f00488c-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-vffwj\" (UID: \"d4731d32-8190-4426-b1f0-d66f3f00488c\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-vffwj" Jul 07 14:33:12.687055 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:33:12.686990 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/d4731d32-8190-4426-b1f0-d66f3f00488c-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-vffwj\" (UID: \"d4731d32-8190-4426-b1f0-d66f3f00488c\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-vffwj" Jul 07 14:33:12.687103 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:33:12.687060 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/d4731d32-8190-4426-b1f0-d66f3f00488c-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-vffwj\" (UID: \"d4731d32-8190-4426-b1f0-d66f3f00488c\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-vffwj" Jul 07 14:33:12.687146 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:33:12.687107 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9vqbj\" (UniqueName: \"kubernetes.io/projected/d4731d32-8190-4426-b1f0-d66f3f00488c-kube-api-access-9vqbj\") pod \"istiod-openshift-gateway-94bb8fbfd-vffwj\" (UID: \"d4731d32-8190-4426-b1f0-d66f3f00488c\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-vffwj" Jul 07 14:33:12.687186 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:33:12.687147 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/d4731d32-8190-4426-b1f0-d66f3f00488c-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-vffwj\" (UID: \"d4731d32-8190-4426-b1f0-d66f3f00488c\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-vffwj" Jul 07 14:33:12.687186 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:33:12.687172 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/d4731d32-8190-4426-b1f0-d66f3f00488c-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-vffwj\" (UID: \"d4731d32-8190-4426-b1f0-d66f3f00488c\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-vffwj" Jul 07 14:33:12.687258 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:33:12.687191 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d4731d32-8190-4426-b1f0-d66f3f00488c-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-vffwj\" (UID: \"d4731d32-8190-4426-b1f0-d66f3f00488c\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-vffwj" Jul 07 14:33:12.788307 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:33:12.788277 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/d4731d32-8190-4426-b1f0-d66f3f00488c-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-vffwj\" (UID: \"d4731d32-8190-4426-b1f0-d66f3f00488c\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-vffwj" Jul 07 14:33:12.788307 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:33:12.788313 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/d4731d32-8190-4426-b1f0-d66f3f00488c-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-vffwj\" (UID: \"d4731d32-8190-4426-b1f0-d66f3f00488c\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-vffwj" Jul 07 14:33:12.788591 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:33:12.788330 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d4731d32-8190-4426-b1f0-d66f3f00488c-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-vffwj\" (UID: \"d4731d32-8190-4426-b1f0-d66f3f00488c\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-vffwj" Jul 07 14:33:12.788591 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:33:12.788368 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/d4731d32-8190-4426-b1f0-d66f3f00488c-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-vffwj\" (UID: \"d4731d32-8190-4426-b1f0-d66f3f00488c\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-vffwj" Jul 07 14:33:12.788591 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:33:12.788416 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/d4731d32-8190-4426-b1f0-d66f3f00488c-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-vffwj\" (UID: \"d4731d32-8190-4426-b1f0-d66f3f00488c\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-vffwj" Jul 07 14:33:12.788591 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:33:12.788453 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/d4731d32-8190-4426-b1f0-d66f3f00488c-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-vffwj\" (UID: \"d4731d32-8190-4426-b1f0-d66f3f00488c\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-vffwj" Jul 07 14:33:12.788591 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:33:12.788496 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9vqbj\" (UniqueName: \"kubernetes.io/projected/d4731d32-8190-4426-b1f0-d66f3f00488c-kube-api-access-9vqbj\") pod \"istiod-openshift-gateway-94bb8fbfd-vffwj\" (UID: \"d4731d32-8190-4426-b1f0-d66f3f00488c\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-vffwj" Jul 07 14:33:12.789601 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:33:12.789573 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/d4731d32-8190-4426-b1f0-d66f3f00488c-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-vffwj\" (UID: \"d4731d32-8190-4426-b1f0-d66f3f00488c\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-vffwj" Jul 07 14:33:12.791115 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:33:12.791088 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/d4731d32-8190-4426-b1f0-d66f3f00488c-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-vffwj\" (UID: \"d4731d32-8190-4426-b1f0-d66f3f00488c\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-vffwj" Jul 07 14:33:12.791215 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:33:12.791187 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/d4731d32-8190-4426-b1f0-d66f3f00488c-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-vffwj\" (UID: \"d4731d32-8190-4426-b1f0-d66f3f00488c\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-vffwj" Jul 07 14:33:12.791371 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:33:12.791355 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/d4731d32-8190-4426-b1f0-d66f3f00488c-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-vffwj\" (UID: \"d4731d32-8190-4426-b1f0-d66f3f00488c\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-vffwj" Jul 07 14:33:12.791412 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:33:12.791379 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/d4731d32-8190-4426-b1f0-d66f3f00488c-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-vffwj\" (UID: \"d4731d32-8190-4426-b1f0-d66f3f00488c\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-vffwj" Jul 07 14:33:12.797162 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:33:12.797138 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d4731d32-8190-4426-b1f0-d66f3f00488c-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-vffwj\" (UID: \"d4731d32-8190-4426-b1f0-d66f3f00488c\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-vffwj" Jul 07 14:33:12.797308 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:33:12.797291 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9vqbj\" (UniqueName: \"kubernetes.io/projected/d4731d32-8190-4426-b1f0-d66f3f00488c-kube-api-access-9vqbj\") pod \"istiod-openshift-gateway-94bb8fbfd-vffwj\" (UID: \"d4731d32-8190-4426-b1f0-d66f3f00488c\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-vffwj" Jul 07 14:33:12.905398 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:33:12.905323 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-vffwj" Jul 07 14:33:13.076288 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:33:13.076266 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-vffwj"] Jul 07 14:33:13.077775 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:33:13.077752 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd4731d32_8190_4426_b1f0_d66f3f00488c.slice/crio-54e423470a4586fad68f33a6a857bc4b370b214154214913d4ac3adc026d3b9a WatchSource:0}: Error finding container 54e423470a4586fad68f33a6a857bc4b370b214154214913d4ac3adc026d3b9a: Status 404 returned error can't find the container with id 54e423470a4586fad68f33a6a857bc4b370b214154214913d4ac3adc026d3b9a Jul 07 14:33:13.210810 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:33:13.210709 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:33:13.307293 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:33:13.307263 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:33:13.753134 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:33:13.753100 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-vffwj" event={"ID":"d4731d32-8190-4426-b1f0-d66f3f00488c","Type":"ContainerStarted","Data":"54e423470a4586fad68f33a6a857bc4b370b214154214913d4ac3adc026d3b9a"} Jul 07 14:33:15.226450 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:33:15.226408 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:33:15.797088 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:33:15.797040 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 07 14:33:15.797212 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:33:15.797131 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 07 14:33:15.836966 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:33:15.836899 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:33:16.009200 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:33:16.009169 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:33:16.161734 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:33:16.161696 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:33:16.267875 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:33:16.267838 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:33:16.364095 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:33:16.364058 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:33:16.771386 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:33:16.771352 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-vffwj" event={"ID":"d4731d32-8190-4426-b1f0-d66f3f00488c","Type":"ContainerStarted","Data":"787a5ccee2d4ebc815f5bb014c4b4896fd5fb2194a457046c35cbc6ce073ef3d"} Jul 07 14:33:16.771588 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:33:16.771543 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-vffwj" Jul 07 14:33:16.773101 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:33:16.773068 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-vffwj" Jul 07 14:33:16.794490 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:33:16.794450 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-vffwj" podStartSLOduration=2.077397332 podStartE2EDuration="4.79443792s" totalImagesPullingTime="2.717040588s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 14:33:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:33:16.792629569 +0000 UTC m=+433.501767948" watchObservedRunningTime="2026-07-07 14:33:16.79443792 +0000 UTC m=+433.503576299" Jul 07 14:33:37.252720 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:33:37.252624 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:33:37.354172 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:33:37.354134 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:34:05.923907 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:34:05.923856 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:34:06.018917 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:34:06.018883 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:34:06.052967 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:34:06.052939 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-hfmbf"] Jul 07 14:34:06.055385 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:34:06.055368 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-hfmbf" Jul 07 14:34:06.057615 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:34:06.057597 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-catalog-dockercfg-bng6f\"" Jul 07 14:34:06.057713 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:34:06.057620 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 07 14:34:06.058082 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:34:06.058068 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 07 14:34:06.080365 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:34:06.080344 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-hfmbf"] Jul 07 14:34:06.118936 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:34:06.118907 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:34:06.123117 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:34:06.123099 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q5bdz\" (UniqueName: \"kubernetes.io/projected/b022b5dc-5a2a-4d7e-80a3-d596c82f9d34-kube-api-access-q5bdz\") pod \"kuadrant-operator-catalog-hfmbf\" (UID: \"b022b5dc-5a2a-4d7e-80a3-d596c82f9d34\") " pod="kuadrant-system/kuadrant-operator-catalog-hfmbf" Jul 07 14:34:06.224004 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:34:06.223950 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q5bdz\" (UniqueName: \"kubernetes.io/projected/b022b5dc-5a2a-4d7e-80a3-d596c82f9d34-kube-api-access-q5bdz\") pod \"kuadrant-operator-catalog-hfmbf\" (UID: \"b022b5dc-5a2a-4d7e-80a3-d596c82f9d34\") " pod="kuadrant-system/kuadrant-operator-catalog-hfmbf" Jul 07 14:34:06.231611 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:34:06.231591 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q5bdz\" (UniqueName: \"kubernetes.io/projected/b022b5dc-5a2a-4d7e-80a3-d596c82f9d34-kube-api-access-q5bdz\") pod \"kuadrant-operator-catalog-hfmbf\" (UID: \"b022b5dc-5a2a-4d7e-80a3-d596c82f9d34\") " pod="kuadrant-system/kuadrant-operator-catalog-hfmbf" Jul 07 14:34:06.365292 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:34:06.365264 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-hfmbf" Jul 07 14:34:06.413487 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:34:06.413439 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-hfmbf"] Jul 07 14:34:06.492256 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:34:06.492188 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-hfmbf"] Jul 07 14:34:06.495968 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:34:06.495945 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb022b5dc_5a2a_4d7e_80a3_d596c82f9d34.slice/crio-2b706a33a044e832c78452734c4be30d4809b89517aea59f9416dedcf0421561 WatchSource:0}: Error finding container 2b706a33a044e832c78452734c4be30d4809b89517aea59f9416dedcf0421561: Status 404 returned error can't find the container with id 2b706a33a044e832c78452734c4be30d4809b89517aea59f9416dedcf0421561 Jul 07 14:34:06.622123 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:34:06.622089 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-ngld5"] Jul 07 14:34:06.624959 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:34:06.624942 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-ngld5" Jul 07 14:34:06.627373 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:34:06.627337 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jgn29\" (UniqueName: \"kubernetes.io/projected/35e57567-53ab-4fe8-a9d9-55c156e703c7-kube-api-access-jgn29\") pod \"kuadrant-operator-catalog-ngld5\" (UID: \"35e57567-53ab-4fe8-a9d9-55c156e703c7\") " pod="kuadrant-system/kuadrant-operator-catalog-ngld5" Jul 07 14:34:06.634142 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:34:06.634114 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-ngld5"] Jul 07 14:34:06.728558 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:34:06.728533 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jgn29\" (UniqueName: \"kubernetes.io/projected/35e57567-53ab-4fe8-a9d9-55c156e703c7-kube-api-access-jgn29\") pod \"kuadrant-operator-catalog-ngld5\" (UID: \"35e57567-53ab-4fe8-a9d9-55c156e703c7\") " pod="kuadrant-system/kuadrant-operator-catalog-ngld5" Jul 07 14:34:06.743379 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:34:06.743311 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jgn29\" (UniqueName: \"kubernetes.io/projected/35e57567-53ab-4fe8-a9d9-55c156e703c7-kube-api-access-jgn29\") pod \"kuadrant-operator-catalog-ngld5\" (UID: \"35e57567-53ab-4fe8-a9d9-55c156e703c7\") " pod="kuadrant-system/kuadrant-operator-catalog-ngld5" Jul 07 14:34:06.863190 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:34:06.863155 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:34:06.934395 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:34:06.934370 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-ngld5" Jul 07 14:34:07.006637 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:34:07.006563 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-hfmbf" event={"ID":"b022b5dc-5a2a-4d7e-80a3-d596c82f9d34","Type":"ContainerStarted","Data":"2b706a33a044e832c78452734c4be30d4809b89517aea59f9416dedcf0421561"} Jul 07 14:34:07.069922 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:34:07.069897 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-ngld5"] Jul 07 14:34:07.071756 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:34:07.071725 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:34:07.071910 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:34:07.071887 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod35e57567_53ab_4fe8_a9d9_55c156e703c7.slice/crio-0c52973ae4dc919e5431d2d82a8489cff9746e0d15477348a936f53b68066855 WatchSource:0}: Error finding container 0c52973ae4dc919e5431d2d82a8489cff9746e0d15477348a936f53b68066855: Status 404 returned error can't find the container with id 0c52973ae4dc919e5431d2d82a8489cff9746e0d15477348a936f53b68066855 Jul 07 14:34:08.014316 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:34:08.014234 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-ngld5" event={"ID":"35e57567-53ab-4fe8-a9d9-55c156e703c7","Type":"ContainerStarted","Data":"0c52973ae4dc919e5431d2d82a8489cff9746e0d15477348a936f53b68066855"} Jul 07 14:34:09.020164 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:34:09.020129 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-hfmbf" event={"ID":"b022b5dc-5a2a-4d7e-80a3-d596c82f9d34","Type":"ContainerStarted","Data":"10a2e128a81709be3ee19e527c02a887fb2e4a5a65bee6debd973d1436ebe71f"} Jul 07 14:34:09.020164 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:34:09.020148 2557 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-catalog-hfmbf" podUID="b022b5dc-5a2a-4d7e-80a3-d596c82f9d34" containerName="registry-server" containerID="cri-o://10a2e128a81709be3ee19e527c02a887fb2e4a5a65bee6debd973d1436ebe71f" gracePeriod=2 Jul 07 14:34:09.021822 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:34:09.021791 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-ngld5" event={"ID":"35e57567-53ab-4fe8-a9d9-55c156e703c7","Type":"ContainerStarted","Data":"cb0baaebe8ff2f440f6e9eff4fbba4aefcca1fb13bdc56fdfb768da0c42958e6"} Jul 07 14:34:09.038152 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:34:09.038113 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-hfmbf" podStartSLOduration=0.660105103 podStartE2EDuration="3.038099624s" totalImagesPullingTime="2.377994521s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 14:34:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:34:09.036270646 +0000 UTC m=+485.745409045" watchObservedRunningTime="2026-07-07 14:34:09.038099624 +0000 UTC m=+485.747238003" Jul 07 14:34:09.054840 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:34:09.054800 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-ngld5" podStartSLOduration=1.249977583 podStartE2EDuration="3.054785803s" totalImagesPullingTime="1.80480822s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 14:34:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:34:09.053290329 +0000 UTC m=+485.762428711" watchObservedRunningTime="2026-07-07 14:34:09.054785803 +0000 UTC m=+485.763924182" Jul 07 14:34:09.260309 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:34:09.260285 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-hfmbf_b022b5dc-5a2a-4d7e-80a3-d596c82f9d34/registry-server/0.log" Jul 07 14:34:09.261614 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:34:09.261597 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-hfmbf" Jul 07 14:34:09.353335 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:34:09.353305 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q5bdz\" (UniqueName: \"kubernetes.io/projected/b022b5dc-5a2a-4d7e-80a3-d596c82f9d34-kube-api-access-q5bdz\") pod \"b022b5dc-5a2a-4d7e-80a3-d596c82f9d34\" (UID: \"b022b5dc-5a2a-4d7e-80a3-d596c82f9d34\") " Jul 07 14:34:09.355427 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:34:09.355404 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b022b5dc-5a2a-4d7e-80a3-d596c82f9d34-kube-api-access-q5bdz" (OuterVolumeSpecName: "kube-api-access-q5bdz") pod "b022b5dc-5a2a-4d7e-80a3-d596c82f9d34" (UID: "b022b5dc-5a2a-4d7e-80a3-d596c82f9d34"). InnerVolumeSpecName "kube-api-access-q5bdz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 14:34:09.454067 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:34:09.453986 2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q5bdz\" (UniqueName: \"kubernetes.io/projected/b022b5dc-5a2a-4d7e-80a3-d596c82f9d34-kube-api-access-q5bdz\") on node \"ip-10-0-134-42.ec2.internal\" DevicePath \"\"" Jul 07 14:34:10.027381 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:34:10.027358 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-hfmbf_b022b5dc-5a2a-4d7e-80a3-d596c82f9d34/registry-server/0.log" Jul 07 14:34:10.027739 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:34:10.027398 2557 generic.go:356] "Generic (PLEG): container finished" podID="b022b5dc-5a2a-4d7e-80a3-d596c82f9d34" containerID="10a2e128a81709be3ee19e527c02a887fb2e4a5a65bee6debd973d1436ebe71f" exitCode=1 Jul 07 14:34:10.027739 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:34:10.027428 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-hfmbf" event={"ID":"b022b5dc-5a2a-4d7e-80a3-d596c82f9d34","Type":"ContainerDied","Data":"10a2e128a81709be3ee19e527c02a887fb2e4a5a65bee6debd973d1436ebe71f"} Jul 07 14:34:10.027739 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:34:10.027455 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-hfmbf" Jul 07 14:34:10.027739 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:34:10.027474 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-hfmbf" event={"ID":"b022b5dc-5a2a-4d7e-80a3-d596c82f9d34","Type":"ContainerDied","Data":"2b706a33a044e832c78452734c4be30d4809b89517aea59f9416dedcf0421561"} Jul 07 14:34:10.027739 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:34:10.027493 2557 scope.go:117] "RemoveContainer" containerID="10a2e128a81709be3ee19e527c02a887fb2e4a5a65bee6debd973d1436ebe71f" Jul 07 14:34:10.036274 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:34:10.036258 2557 scope.go:117] "RemoveContainer" containerID="10a2e128a81709be3ee19e527c02a887fb2e4a5a65bee6debd973d1436ebe71f" Jul 07 14:34:10.036540 ip-10-0-134-42 kubenswrapper[2557]: E0707 14:34:10.036502 2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"10a2e128a81709be3ee19e527c02a887fb2e4a5a65bee6debd973d1436ebe71f\": container with ID starting with 10a2e128a81709be3ee19e527c02a887fb2e4a5a65bee6debd973d1436ebe71f not found: ID does not exist" containerID="10a2e128a81709be3ee19e527c02a887fb2e4a5a65bee6debd973d1436ebe71f" Jul 07 14:34:10.036588 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:34:10.036546 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"10a2e128a81709be3ee19e527c02a887fb2e4a5a65bee6debd973d1436ebe71f"} err="failed to get container status \"10a2e128a81709be3ee19e527c02a887fb2e4a5a65bee6debd973d1436ebe71f\": rpc error: code = NotFound desc = could not find container \"10a2e128a81709be3ee19e527c02a887fb2e4a5a65bee6debd973d1436ebe71f\": container with ID starting with 10a2e128a81709be3ee19e527c02a887fb2e4a5a65bee6debd973d1436ebe71f not found: ID does not exist" Jul 07 14:34:10.041450 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:34:10.041426 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-hfmbf"] Jul 07 14:34:10.044861 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:34:10.044839 2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-hfmbf"] Jul 07 14:34:11.835744 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:34:11.835714 2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b022b5dc-5a2a-4d7e-80a3-d596c82f9d34" path="/var/lib/kubelet/pods/b022b5dc-5a2a-4d7e-80a3-d596c82f9d34/volumes" Jul 07 14:34:16.935287 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:34:16.935254 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-ngld5" Jul 07 14:34:16.935805 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:34:16.935300 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-ngld5" Jul 07 14:34:16.957570 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:34:16.957545 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-ngld5" Jul 07 14:34:17.081781 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:34:17.081760 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-ngld5" Jul 07 14:34:21.468965 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:34:21.468931 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73pr78p"] Jul 07 14:34:21.469394 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:34:21.469271 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b022b5dc-5a2a-4d7e-80a3-d596c82f9d34" containerName="registry-server" Jul 07 14:34:21.469394 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:34:21.469283 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="b022b5dc-5a2a-4d7e-80a3-d596c82f9d34" containerName="registry-server" Jul 07 14:34:21.469394 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:34:21.469341 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="b022b5dc-5a2a-4d7e-80a3-d596c82f9d34" containerName="registry-server" Jul 07 14:34:21.472727 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:34:21.472704 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73pr78p" Jul 07 14:34:21.474859 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:34:21.474839 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-gbxt7\"" Jul 07 14:34:21.483776 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:34:21.483753 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73pr78p"] Jul 07 14:34:21.552168 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:34:21.552139 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wmcn5\" (UniqueName: \"kubernetes.io/projected/2870ed2a-a2da-44d8-9a7b-51b5b550469b-kube-api-access-wmcn5\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73pr78p\" (UID: \"2870ed2a-a2da-44d8-9a7b-51b5b550469b\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73pr78p" Jul 07 14:34:21.552270 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:34:21.552187 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2870ed2a-a2da-44d8-9a7b-51b5b550469b-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73pr78p\" (UID: \"2870ed2a-a2da-44d8-9a7b-51b5b550469b\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73pr78p" Jul 07 14:34:21.552270 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:34:21.552204 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2870ed2a-a2da-44d8-9a7b-51b5b550469b-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73pr78p\" (UID: \"2870ed2a-a2da-44d8-9a7b-51b5b550469b\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73pr78p" Jul 07 14:34:21.653252 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:34:21.653230 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wmcn5\" (UniqueName: \"kubernetes.io/projected/2870ed2a-a2da-44d8-9a7b-51b5b550469b-kube-api-access-wmcn5\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73pr78p\" (UID: \"2870ed2a-a2da-44d8-9a7b-51b5b550469b\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73pr78p" Jul 07 14:34:21.653350 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:34:21.653273 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2870ed2a-a2da-44d8-9a7b-51b5b550469b-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73pr78p\" (UID: \"2870ed2a-a2da-44d8-9a7b-51b5b550469b\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73pr78p" Jul 07 14:34:21.653350 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:34:21.653291 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2870ed2a-a2da-44d8-9a7b-51b5b550469b-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73pr78p\" (UID: \"2870ed2a-a2da-44d8-9a7b-51b5b550469b\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73pr78p" Jul 07 14:34:21.653677 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:34:21.653659 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2870ed2a-a2da-44d8-9a7b-51b5b550469b-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73pr78p\" (UID: \"2870ed2a-a2da-44d8-9a7b-51b5b550469b\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73pr78p" Jul 07 14:34:21.653725 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:34:21.653673 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2870ed2a-a2da-44d8-9a7b-51b5b550469b-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73pr78p\" (UID: \"2870ed2a-a2da-44d8-9a7b-51b5b550469b\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73pr78p" Jul 07 14:34:21.662318 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:34:21.662299 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wmcn5\" (UniqueName: \"kubernetes.io/projected/2870ed2a-a2da-44d8-9a7b-51b5b550469b-kube-api-access-wmcn5\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73pr78p\" (UID: \"2870ed2a-a2da-44d8-9a7b-51b5b550469b\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73pr78p" Jul 07 14:34:21.783310 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:34:21.783249 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73pr78p" Jul 07 14:34:21.916382 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:34:21.916351 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73pr78p"] Jul 07 14:34:21.919554 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:34:21.919511 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2870ed2a_a2da_44d8_9a7b_51b5b550469b.slice/crio-ebc460235f1eb17a0325c77e3e65f3a062b2ecd1288722847ed9db6f89ec22aa WatchSource:0}: Error finding container ebc460235f1eb17a0325c77e3e65f3a062b2ecd1288722847ed9db6f89ec22aa: Status 404 returned error can't find the container with id ebc460235f1eb17a0325c77e3e65f3a062b2ecd1288722847ed9db6f89ec22aa Jul 07 14:34:22.072718 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:34:22.072689 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12m9zw"] Jul 07 14:34:22.075173 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:34:22.075155 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12m9zw" Jul 07 14:34:22.086374 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:34:22.086354 2557 generic.go:356] "Generic (PLEG): container finished" podID="2870ed2a-a2da-44d8-9a7b-51b5b550469b" containerID="2ceffef52f0d1e28b7e1216530783b0c9d9d68871e1a29186a1321517848a123" exitCode=0 Jul 07 14:34:22.086478 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:34:22.086404 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73pr78p" event={"ID":"2870ed2a-a2da-44d8-9a7b-51b5b550469b","Type":"ContainerDied","Data":"2ceffef52f0d1e28b7e1216530783b0c9d9d68871e1a29186a1321517848a123"} Jul 07 14:34:22.086478 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:34:22.086445 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73pr78p" event={"ID":"2870ed2a-a2da-44d8-9a7b-51b5b550469b","Type":"ContainerStarted","Data":"ebc460235f1eb17a0325c77e3e65f3a062b2ecd1288722847ed9db6f89ec22aa"} Jul 07 14:34:22.091279 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:34:22.091260 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12m9zw"] Jul 07 14:34:22.157442 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:34:22.157420 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/740e7355-ef13-48ab-a537-87418f967bb4-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12m9zw\" (UID: \"740e7355-ef13-48ab-a537-87418f967bb4\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12m9zw" Jul 07 14:34:22.157585 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:34:22.157469 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8z64m\" (UniqueName: \"kubernetes.io/projected/740e7355-ef13-48ab-a537-87418f967bb4-kube-api-access-8z64m\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12m9zw\" (UID: \"740e7355-ef13-48ab-a537-87418f967bb4\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12m9zw" Jul 07 14:34:22.157585 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:34:22.157505 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/740e7355-ef13-48ab-a537-87418f967bb4-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12m9zw\" (UID: \"740e7355-ef13-48ab-a537-87418f967bb4\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12m9zw" Jul 07 14:34:22.158187 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:34:22.158161 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:34:22.258431 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:34:22.258406 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/740e7355-ef13-48ab-a537-87418f967bb4-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12m9zw\" (UID: \"740e7355-ef13-48ab-a537-87418f967bb4\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12m9zw" Jul 07 14:34:22.258550 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:34:22.258438 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8z64m\" (UniqueName: \"kubernetes.io/projected/740e7355-ef13-48ab-a537-87418f967bb4-kube-api-access-8z64m\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12m9zw\" (UID: \"740e7355-ef13-48ab-a537-87418f967bb4\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12m9zw" Jul 07 14:34:22.258550 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:34:22.258460 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/740e7355-ef13-48ab-a537-87418f967bb4-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12m9zw\" (UID: \"740e7355-ef13-48ab-a537-87418f967bb4\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12m9zw" Jul 07 14:34:22.258803 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:34:22.258787 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/740e7355-ef13-48ab-a537-87418f967bb4-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12m9zw\" (UID: \"740e7355-ef13-48ab-a537-87418f967bb4\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12m9zw" Jul 07 14:34:22.258839 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:34:22.258787 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/740e7355-ef13-48ab-a537-87418f967bb4-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12m9zw\" (UID: \"740e7355-ef13-48ab-a537-87418f967bb4\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12m9zw" Jul 07 14:34:22.267045 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:34:22.267021 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8z64m\" (UniqueName: \"kubernetes.io/projected/740e7355-ef13-48ab-a537-87418f967bb4-kube-api-access-8z64m\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12m9zw\" (UID: \"740e7355-ef13-48ab-a537-87418f967bb4\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12m9zw" Jul 07 14:34:22.383759 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:34:22.383695 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12m9zw" Jul 07 14:34:22.474980 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:34:22.474940 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0kmp97"] Jul 07 14:34:22.479784 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:34:22.478752 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0kmp97" Jul 07 14:34:22.487112 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:34:22.487087 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0kmp97"] Jul 07 14:34:22.514695 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:34:22.514670 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12m9zw"] Jul 07 14:34:22.516230 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:34:22.516203 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod740e7355_ef13_48ab_a537_87418f967bb4.slice/crio-a359cb299b2c1198d7d6d12e67a846d3fec063270d519ffc7150ac1f46d09048 WatchSource:0}: Error finding container a359cb299b2c1198d7d6d12e67a846d3fec063270d519ffc7150ac1f46d09048: Status 404 returned error can't find the container with id a359cb299b2c1198d7d6d12e67a846d3fec063270d519ffc7150ac1f46d09048 Jul 07 14:34:22.560604 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:34:22.560576 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ac44a2b1-1db5-4e2b-a3a4-7f08168b3902-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0kmp97\" (UID: \"ac44a2b1-1db5-4e2b-a3a4-7f08168b3902\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0kmp97" Jul 07 14:34:22.560706 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:34:22.560660 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ac44a2b1-1db5-4e2b-a3a4-7f08168b3902-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0kmp97\" (UID: \"ac44a2b1-1db5-4e2b-a3a4-7f08168b3902\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0kmp97" Jul 07 14:34:22.560706 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:34:22.560692 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hgb5v\" (UniqueName: \"kubernetes.io/projected/ac44a2b1-1db5-4e2b-a3a4-7f08168b3902-kube-api-access-hgb5v\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0kmp97\" (UID: \"ac44a2b1-1db5-4e2b-a3a4-7f08168b3902\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0kmp97" Jul 07 14:34:22.661376 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:34:22.661307 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ac44a2b1-1db5-4e2b-a3a4-7f08168b3902-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0kmp97\" (UID: \"ac44a2b1-1db5-4e2b-a3a4-7f08168b3902\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0kmp97" Jul 07 14:34:22.661376 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:34:22.661365 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ac44a2b1-1db5-4e2b-a3a4-7f08168b3902-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0kmp97\" (UID: \"ac44a2b1-1db5-4e2b-a3a4-7f08168b3902\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0kmp97" Jul 07 14:34:22.661591 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:34:22.661391 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hgb5v\" (UniqueName: \"kubernetes.io/projected/ac44a2b1-1db5-4e2b-a3a4-7f08168b3902-kube-api-access-hgb5v\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0kmp97\" (UID: \"ac44a2b1-1db5-4e2b-a3a4-7f08168b3902\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0kmp97" Jul 07 14:34:22.661748 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:34:22.661724 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ac44a2b1-1db5-4e2b-a3a4-7f08168b3902-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0kmp97\" (UID: \"ac44a2b1-1db5-4e2b-a3a4-7f08168b3902\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0kmp97" Jul 07 14:34:22.661809 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:34:22.661744 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ac44a2b1-1db5-4e2b-a3a4-7f08168b3902-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0kmp97\" (UID: \"ac44a2b1-1db5-4e2b-a3a4-7f08168b3902\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0kmp97" Jul 07 14:34:22.670663 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:34:22.670644 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hgb5v\" (UniqueName: \"kubernetes.io/projected/ac44a2b1-1db5-4e2b-a3a4-7f08168b3902-kube-api-access-hgb5v\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0kmp97\" (UID: \"ac44a2b1-1db5-4e2b-a3a4-7f08168b3902\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0kmp97" Jul 07 14:34:22.791158 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:34:22.791132 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0kmp97" Jul 07 14:34:22.876814 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:34:22.876786 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759ps8sv"] Jul 07 14:34:22.880331 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:34:22.880311 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759ps8sv" Jul 07 14:34:22.892383 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:34:22.892355 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759ps8sv"] Jul 07 14:34:22.932165 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:34:22.932144 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0kmp97"] Jul 07 14:34:22.938506 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:34:22.938475 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podac44a2b1_1db5_4e2b_a3a4_7f08168b3902.slice/crio-e409f08b90b38ff8a0e27e97d2409b353d06eebf53a09303526a85141d1503d7 WatchSource:0}: Error finding container e409f08b90b38ff8a0e27e97d2409b353d06eebf53a09303526a85141d1503d7: Status 404 returned error can't find the container with id e409f08b90b38ff8a0e27e97d2409b353d06eebf53a09303526a85141d1503d7 Jul 07 14:34:22.963460 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:34:22.963438 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rcxss\" (UniqueName: \"kubernetes.io/projected/dd0c1d7e-44ea-4e2f-a1a4-d8b7877d2069-kube-api-access-rcxss\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759ps8sv\" (UID: \"dd0c1d7e-44ea-4e2f-a1a4-d8b7877d2069\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759ps8sv" Jul 07 14:34:22.963595 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:34:22.963578 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/dd0c1d7e-44ea-4e2f-a1a4-d8b7877d2069-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759ps8sv\" (UID: \"dd0c1d7e-44ea-4e2f-a1a4-d8b7877d2069\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759ps8sv" Jul 07 14:34:22.963639 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:34:22.963609 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/dd0c1d7e-44ea-4e2f-a1a4-d8b7877d2069-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759ps8sv\" (UID: \"dd0c1d7e-44ea-4e2f-a1a4-d8b7877d2069\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759ps8sv" Jul 07 14:34:23.064844 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:34:23.064813 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rcxss\" (UniqueName: \"kubernetes.io/projected/dd0c1d7e-44ea-4e2f-a1a4-d8b7877d2069-kube-api-access-rcxss\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759ps8sv\" (UID: \"dd0c1d7e-44ea-4e2f-a1a4-d8b7877d2069\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759ps8sv" Jul 07 14:34:23.064981 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:34:23.064893 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/dd0c1d7e-44ea-4e2f-a1a4-d8b7877d2069-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759ps8sv\" (UID: \"dd0c1d7e-44ea-4e2f-a1a4-d8b7877d2069\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759ps8sv" Jul 07 14:34:23.064981 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:34:23.064918 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/dd0c1d7e-44ea-4e2f-a1a4-d8b7877d2069-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759ps8sv\" (UID: \"dd0c1d7e-44ea-4e2f-a1a4-d8b7877d2069\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759ps8sv" Jul 07 14:34:23.065323 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:34:23.065303 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/dd0c1d7e-44ea-4e2f-a1a4-d8b7877d2069-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759ps8sv\" (UID: \"dd0c1d7e-44ea-4e2f-a1a4-d8b7877d2069\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759ps8sv" Jul 07 14:34:23.065323 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:34:23.065311 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/dd0c1d7e-44ea-4e2f-a1a4-d8b7877d2069-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759ps8sv\" (UID: \"dd0c1d7e-44ea-4e2f-a1a4-d8b7877d2069\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759ps8sv" Jul 07 14:34:23.073259 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:34:23.073233 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rcxss\" (UniqueName: \"kubernetes.io/projected/dd0c1d7e-44ea-4e2f-a1a4-d8b7877d2069-kube-api-access-rcxss\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759ps8sv\" (UID: \"dd0c1d7e-44ea-4e2f-a1a4-d8b7877d2069\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759ps8sv" Jul 07 14:34:23.093135 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:34:23.093112 2557 generic.go:356] "Generic (PLEG): container finished" podID="2870ed2a-a2da-44d8-9a7b-51b5b550469b" containerID="da738eccace82ccb6c4e1d23fc9790df2150aba0e8a298e61b8e14cd6c3b61d8" exitCode=0 Jul 07 14:34:23.093232 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:34:23.093175 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73pr78p" event={"ID":"2870ed2a-a2da-44d8-9a7b-51b5b550469b","Type":"ContainerDied","Data":"da738eccace82ccb6c4e1d23fc9790df2150aba0e8a298e61b8e14cd6c3b61d8"} Jul 07 14:34:23.094708 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:34:23.094631 2557 generic.go:356] "Generic (PLEG): container finished" podID="ac44a2b1-1db5-4e2b-a3a4-7f08168b3902" containerID="75f33dba9a526bf0e257ca6dbdec4b5f33201b137a05d7dd76b25fa5a2066390" exitCode=0 Jul 07 14:34:23.094787 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:34:23.094761 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0kmp97" event={"ID":"ac44a2b1-1db5-4e2b-a3a4-7f08168b3902","Type":"ContainerDied","Data":"75f33dba9a526bf0e257ca6dbdec4b5f33201b137a05d7dd76b25fa5a2066390"} Jul 07 14:34:23.094840 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:34:23.094809 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0kmp97" event={"ID":"ac44a2b1-1db5-4e2b-a3a4-7f08168b3902","Type":"ContainerStarted","Data":"e409f08b90b38ff8a0e27e97d2409b353d06eebf53a09303526a85141d1503d7"} Jul 07 14:34:23.096362 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:34:23.096343 2557 generic.go:356] "Generic (PLEG): container finished" podID="740e7355-ef13-48ab-a537-87418f967bb4" containerID="48e8ac7ef65831181541d8e89c605034f467936d25cf5efbce863fc3ec947bc2" exitCode=0 Jul 07 14:34:23.096447 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:34:23.096395 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12m9zw" event={"ID":"740e7355-ef13-48ab-a537-87418f967bb4","Type":"ContainerDied","Data":"48e8ac7ef65831181541d8e89c605034f467936d25cf5efbce863fc3ec947bc2"} Jul 07 14:34:23.096447 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:34:23.096427 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12m9zw" event={"ID":"740e7355-ef13-48ab-a537-87418f967bb4","Type":"ContainerStarted","Data":"a359cb299b2c1198d7d6d12e67a846d3fec063270d519ffc7150ac1f46d09048"} Jul 07 14:34:23.193642 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:34:23.193577 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759ps8sv" Jul 07 14:34:23.341897 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:34:23.341871 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759ps8sv"] Jul 07 14:34:23.342145 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:34:23.342116 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddd0c1d7e_44ea_4e2f_a1a4_d8b7877d2069.slice/crio-7beac85658fadac647fed9a6c4758ce080aa135de98b6620c9998f90bfd16fcb WatchSource:0}: Error finding container 7beac85658fadac647fed9a6c4758ce080aa135de98b6620c9998f90bfd16fcb: Status 404 returned error can't find the container with id 7beac85658fadac647fed9a6c4758ce080aa135de98b6620c9998f90bfd16fcb Jul 07 14:34:24.104298 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:34:24.104266 2557 generic.go:356] "Generic (PLEG): container finished" podID="2870ed2a-a2da-44d8-9a7b-51b5b550469b" containerID="30e47f3cb8ffd344f2cd88866f4fab3d19eb0a1ba6244c8bf77c5ad720e28c26" exitCode=0 Jul 07 14:34:24.104700 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:34:24.104343 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73pr78p" event={"ID":"2870ed2a-a2da-44d8-9a7b-51b5b550469b","Type":"ContainerDied","Data":"30e47f3cb8ffd344f2cd88866f4fab3d19eb0a1ba6244c8bf77c5ad720e28c26"} Jul 07 14:34:24.105680 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:34:24.105651 2557 generic.go:356] "Generic (PLEG): container finished" podID="dd0c1d7e-44ea-4e2f-a1a4-d8b7877d2069" containerID="e9ab568f60a6e2d8714df0a3ce65c32b7e14a3aba27e5a21884d7d90059adabb" exitCode=0 Jul 07 14:34:24.105820 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:34:24.105730 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759ps8sv" event={"ID":"dd0c1d7e-44ea-4e2f-a1a4-d8b7877d2069","Type":"ContainerDied","Data":"e9ab568f60a6e2d8714df0a3ce65c32b7e14a3aba27e5a21884d7d90059adabb"} Jul 07 14:34:24.105820 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:34:24.105773 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759ps8sv" event={"ID":"dd0c1d7e-44ea-4e2f-a1a4-d8b7877d2069","Type":"ContainerStarted","Data":"7beac85658fadac647fed9a6c4758ce080aa135de98b6620c9998f90bfd16fcb"} Jul 07 14:34:24.107597 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:34:24.107575 2557 generic.go:356] "Generic (PLEG): container finished" podID="ac44a2b1-1db5-4e2b-a3a4-7f08168b3902" containerID="01a6b17b0c43f0b05177d94eacf9393fcfb5e8aceb628a4a9ef35ffa782dfe42" exitCode=0 Jul 07 14:34:24.107704 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:34:24.107654 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0kmp97" event={"ID":"ac44a2b1-1db5-4e2b-a3a4-7f08168b3902","Type":"ContainerDied","Data":"01a6b17b0c43f0b05177d94eacf9393fcfb5e8aceb628a4a9ef35ffa782dfe42"} Jul 07 14:34:25.119028 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:34:25.118983 2557 generic.go:356] "Generic (PLEG): container finished" podID="740e7355-ef13-48ab-a537-87418f967bb4" containerID="a28d15f32e49b43130544da1d14ef03ee690152f43a10fe0110fc695092f2428" exitCode=0 Jul 07 14:34:25.119467 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:34:25.119080 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12m9zw" event={"ID":"740e7355-ef13-48ab-a537-87418f967bb4","Type":"ContainerDied","Data":"a28d15f32e49b43130544da1d14ef03ee690152f43a10fe0110fc695092f2428"} Jul 07 14:34:25.121540 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:34:25.121480 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759ps8sv" event={"ID":"dd0c1d7e-44ea-4e2f-a1a4-d8b7877d2069","Type":"ContainerStarted","Data":"839901a98a4378b0ff9e6dcc43ed9c701f472901c955b5c18f20d034ecaa8579"} Jul 07 14:34:25.123781 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:34:25.123751 2557 generic.go:356] "Generic (PLEG): container finished" podID="ac44a2b1-1db5-4e2b-a3a4-7f08168b3902" containerID="ebb07910cce72c579ea201cd49e0fd414c9f1a93ea1f744e7c3759fb4e4dfbf2" exitCode=0 Jul 07 14:34:25.123839 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:34:25.123825 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0kmp97" event={"ID":"ac44a2b1-1db5-4e2b-a3a4-7f08168b3902","Type":"ContainerDied","Data":"ebb07910cce72c579ea201cd49e0fd414c9f1a93ea1f744e7c3759fb4e4dfbf2"} Jul 07 14:34:25.256929 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:34:25.256906 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73pr78p" Jul 07 14:34:25.385113 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:34:25.385088 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wmcn5\" (UniqueName: \"kubernetes.io/projected/2870ed2a-a2da-44d8-9a7b-51b5b550469b-kube-api-access-wmcn5\") pod \"2870ed2a-a2da-44d8-9a7b-51b5b550469b\" (UID: \"2870ed2a-a2da-44d8-9a7b-51b5b550469b\") " Jul 07 14:34:25.385247 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:34:25.385121 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2870ed2a-a2da-44d8-9a7b-51b5b550469b-util\") pod \"2870ed2a-a2da-44d8-9a7b-51b5b550469b\" (UID: \"2870ed2a-a2da-44d8-9a7b-51b5b550469b\") " Jul 07 14:34:25.385247 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:34:25.385176 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2870ed2a-a2da-44d8-9a7b-51b5b550469b-bundle\") pod \"2870ed2a-a2da-44d8-9a7b-51b5b550469b\" (UID: \"2870ed2a-a2da-44d8-9a7b-51b5b550469b\") " Jul 07 14:34:25.385810 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:34:25.385787 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2870ed2a-a2da-44d8-9a7b-51b5b550469b-bundle" (OuterVolumeSpecName: "bundle") pod "2870ed2a-a2da-44d8-9a7b-51b5b550469b" (UID: "2870ed2a-a2da-44d8-9a7b-51b5b550469b"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 14:34:25.386980 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:34:25.386958 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2870ed2a-a2da-44d8-9a7b-51b5b550469b-kube-api-access-wmcn5" (OuterVolumeSpecName: "kube-api-access-wmcn5") pod "2870ed2a-a2da-44d8-9a7b-51b5b550469b" (UID: "2870ed2a-a2da-44d8-9a7b-51b5b550469b"). InnerVolumeSpecName "kube-api-access-wmcn5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 14:34:25.390725 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:34:25.390701 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2870ed2a-a2da-44d8-9a7b-51b5b550469b-util" (OuterVolumeSpecName: "util") pod "2870ed2a-a2da-44d8-9a7b-51b5b550469b" (UID: "2870ed2a-a2da-44d8-9a7b-51b5b550469b"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 14:34:25.486368 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:34:25.486347 2557 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2870ed2a-a2da-44d8-9a7b-51b5b550469b-bundle\") on node \"ip-10-0-134-42.ec2.internal\" DevicePath \"\"" Jul 07 14:34:25.486368 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:34:25.486369 2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wmcn5\" (UniqueName: \"kubernetes.io/projected/2870ed2a-a2da-44d8-9a7b-51b5b550469b-kube-api-access-wmcn5\") on node \"ip-10-0-134-42.ec2.internal\" DevicePath \"\"" Jul 07 14:34:25.486480 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:34:25.486379 2557 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2870ed2a-a2da-44d8-9a7b-51b5b550469b-util\") on node \"ip-10-0-134-42.ec2.internal\" DevicePath \"\"" Jul 07 14:34:26.131390 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:34:26.131344 2557 generic.go:356] "Generic (PLEG): container finished" podID="740e7355-ef13-48ab-a537-87418f967bb4" containerID="7df1c7b0be9c3c0df996d5155f527459385be1afbb2ae6ffd1165eb496ed2cbf" exitCode=0 Jul 07 14:34:26.131837 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:34:26.131450 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12m9zw" event={"ID":"740e7355-ef13-48ab-a537-87418f967bb4","Type":"ContainerDied","Data":"7df1c7b0be9c3c0df996d5155f527459385be1afbb2ae6ffd1165eb496ed2cbf"} Jul 07 14:34:26.133573 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:34:26.133547 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73pr78p" event={"ID":"2870ed2a-a2da-44d8-9a7b-51b5b550469b","Type":"ContainerDied","Data":"ebc460235f1eb17a0325c77e3e65f3a062b2ecd1288722847ed9db6f89ec22aa"} Jul 07 14:34:26.133699 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:34:26.133580 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73pr78p" Jul 07 14:34:26.133699 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:34:26.133588 2557 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ebc460235f1eb17a0325c77e3e65f3a062b2ecd1288722847ed9db6f89ec22aa" Jul 07 14:34:26.135988 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:34:26.135961 2557 generic.go:356] "Generic (PLEG): container finished" podID="dd0c1d7e-44ea-4e2f-a1a4-d8b7877d2069" containerID="839901a98a4378b0ff9e6dcc43ed9c701f472901c955b5c18f20d034ecaa8579" exitCode=0 Jul 07 14:34:26.136114 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:34:26.136082 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759ps8sv" event={"ID":"dd0c1d7e-44ea-4e2f-a1a4-d8b7877d2069","Type":"ContainerDied","Data":"839901a98a4378b0ff9e6dcc43ed9c701f472901c955b5c18f20d034ecaa8579"} Jul 07 14:34:26.268934 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:34:26.268914 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0kmp97" Jul 07 14:34:26.395140 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:34:26.395077 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ac44a2b1-1db5-4e2b-a3a4-7f08168b3902-bundle\") pod \"ac44a2b1-1db5-4e2b-a3a4-7f08168b3902\" (UID: \"ac44a2b1-1db5-4e2b-a3a4-7f08168b3902\") " Jul 07 14:34:26.395140 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:34:26.395123 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hgb5v\" (UniqueName: \"kubernetes.io/projected/ac44a2b1-1db5-4e2b-a3a4-7f08168b3902-kube-api-access-hgb5v\") pod \"ac44a2b1-1db5-4e2b-a3a4-7f08168b3902\" (UID: \"ac44a2b1-1db5-4e2b-a3a4-7f08168b3902\") " Jul 07 14:34:26.395329 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:34:26.395174 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ac44a2b1-1db5-4e2b-a3a4-7f08168b3902-util\") pod \"ac44a2b1-1db5-4e2b-a3a4-7f08168b3902\" (UID: \"ac44a2b1-1db5-4e2b-a3a4-7f08168b3902\") " Jul 07 14:34:26.395632 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:34:26.395608 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ac44a2b1-1db5-4e2b-a3a4-7f08168b3902-bundle" (OuterVolumeSpecName: "bundle") pod "ac44a2b1-1db5-4e2b-a3a4-7f08168b3902" (UID: "ac44a2b1-1db5-4e2b-a3a4-7f08168b3902"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 14:34:26.397155 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:34:26.397134 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ac44a2b1-1db5-4e2b-a3a4-7f08168b3902-kube-api-access-hgb5v" (OuterVolumeSpecName: "kube-api-access-hgb5v") pod "ac44a2b1-1db5-4e2b-a3a4-7f08168b3902" (UID: "ac44a2b1-1db5-4e2b-a3a4-7f08168b3902"). InnerVolumeSpecName "kube-api-access-hgb5v". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 14:34:26.400375 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:34:26.400353 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ac44a2b1-1db5-4e2b-a3a4-7f08168b3902-util" (OuterVolumeSpecName: "util") pod "ac44a2b1-1db5-4e2b-a3a4-7f08168b3902" (UID: "ac44a2b1-1db5-4e2b-a3a4-7f08168b3902"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 14:34:26.496440 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:34:26.496406 2557 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ac44a2b1-1db5-4e2b-a3a4-7f08168b3902-util\") on node \"ip-10-0-134-42.ec2.internal\" DevicePath \"\"" Jul 07 14:34:26.496440 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:34:26.496432 2557 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ac44a2b1-1db5-4e2b-a3a4-7f08168b3902-bundle\") on node \"ip-10-0-134-42.ec2.internal\" DevicePath \"\"" Jul 07 14:34:26.496440 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:34:26.496442 2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hgb5v\" (UniqueName: \"kubernetes.io/projected/ac44a2b1-1db5-4e2b-a3a4-7f08168b3902-kube-api-access-hgb5v\") on node \"ip-10-0-134-42.ec2.internal\" DevicePath \"\"" Jul 07 14:34:27.143242 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:34:27.143210 2557 generic.go:356] "Generic (PLEG): container finished" podID="dd0c1d7e-44ea-4e2f-a1a4-d8b7877d2069" containerID="c2c74588be2d8897e20b3c8b3eb2559bac50c0b201db9d8b70415b856619b9d0" exitCode=0 Jul 07 14:34:27.143664 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:34:27.143290 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759ps8sv" event={"ID":"dd0c1d7e-44ea-4e2f-a1a4-d8b7877d2069","Type":"ContainerDied","Data":"c2c74588be2d8897e20b3c8b3eb2559bac50c0b201db9d8b70415b856619b9d0"} Jul 07 14:34:27.145191 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:34:27.145173 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0kmp97" Jul 07 14:34:27.145332 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:34:27.145204 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0kmp97" event={"ID":"ac44a2b1-1db5-4e2b-a3a4-7f08168b3902","Type":"ContainerDied","Data":"e409f08b90b38ff8a0e27e97d2409b353d06eebf53a09303526a85141d1503d7"} Jul 07 14:34:27.145332 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:34:27.145233 2557 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e409f08b90b38ff8a0e27e97d2409b353d06eebf53a09303526a85141d1503d7" Jul 07 14:34:27.269370 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:34:27.269350 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12m9zw" Jul 07 14:34:27.406119 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:34:27.406057 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/740e7355-ef13-48ab-a537-87418f967bb4-util\") pod \"740e7355-ef13-48ab-a537-87418f967bb4\" (UID: \"740e7355-ef13-48ab-a537-87418f967bb4\") " Jul 07 14:34:27.406119 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:34:27.406117 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8z64m\" (UniqueName: \"kubernetes.io/projected/740e7355-ef13-48ab-a537-87418f967bb4-kube-api-access-8z64m\") pod \"740e7355-ef13-48ab-a537-87418f967bb4\" (UID: \"740e7355-ef13-48ab-a537-87418f967bb4\") " Jul 07 14:34:27.406260 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:34:27.406141 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/740e7355-ef13-48ab-a537-87418f967bb4-bundle\") pod \"740e7355-ef13-48ab-a537-87418f967bb4\" (UID: \"740e7355-ef13-48ab-a537-87418f967bb4\") " Jul 07 14:34:27.406720 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:34:27.406692 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/740e7355-ef13-48ab-a537-87418f967bb4-bundle" (OuterVolumeSpecName: "bundle") pod "740e7355-ef13-48ab-a537-87418f967bb4" (UID: "740e7355-ef13-48ab-a537-87418f967bb4"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 14:34:27.408119 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:34:27.408098 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/740e7355-ef13-48ab-a537-87418f967bb4-kube-api-access-8z64m" (OuterVolumeSpecName: "kube-api-access-8z64m") pod "740e7355-ef13-48ab-a537-87418f967bb4" (UID: "740e7355-ef13-48ab-a537-87418f967bb4"). InnerVolumeSpecName "kube-api-access-8z64m". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 14:34:27.411159 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:34:27.411122 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/740e7355-ef13-48ab-a537-87418f967bb4-util" (OuterVolumeSpecName: "util") pod "740e7355-ef13-48ab-a537-87418f967bb4" (UID: "740e7355-ef13-48ab-a537-87418f967bb4"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 14:34:27.507382 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:34:27.507354 2557 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/740e7355-ef13-48ab-a537-87418f967bb4-util\") on node \"ip-10-0-134-42.ec2.internal\" DevicePath \"\"" Jul 07 14:34:27.507382 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:34:27.507380 2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8z64m\" (UniqueName: \"kubernetes.io/projected/740e7355-ef13-48ab-a537-87418f967bb4-kube-api-access-8z64m\") on node \"ip-10-0-134-42.ec2.internal\" DevicePath \"\"" Jul 07 14:34:27.507583 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:34:27.507394 2557 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/740e7355-ef13-48ab-a537-87418f967bb4-bundle\") on node \"ip-10-0-134-42.ec2.internal\" DevicePath \"\"" Jul 07 14:34:28.151923 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:34:28.151843 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12m9zw" event={"ID":"740e7355-ef13-48ab-a537-87418f967bb4","Type":"ContainerDied","Data":"a359cb299b2c1198d7d6d12e67a846d3fec063270d519ffc7150ac1f46d09048"} Jul 07 14:34:28.151923 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:34:28.151870 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12m9zw" Jul 07 14:34:28.151923 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:34:28.151893 2557 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a359cb299b2c1198d7d6d12e67a846d3fec063270d519ffc7150ac1f46d09048" Jul 07 14:34:28.279583 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:34:28.279561 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759ps8sv" Jul 07 14:34:28.414072 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:34:28.413992 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rcxss\" (UniqueName: \"kubernetes.io/projected/dd0c1d7e-44ea-4e2f-a1a4-d8b7877d2069-kube-api-access-rcxss\") pod \"dd0c1d7e-44ea-4e2f-a1a4-d8b7877d2069\" (UID: \"dd0c1d7e-44ea-4e2f-a1a4-d8b7877d2069\") " Jul 07 14:34:28.414209 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:34:28.414089 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/dd0c1d7e-44ea-4e2f-a1a4-d8b7877d2069-util\") pod \"dd0c1d7e-44ea-4e2f-a1a4-d8b7877d2069\" (UID: \"dd0c1d7e-44ea-4e2f-a1a4-d8b7877d2069\") " Jul 07 14:34:28.414209 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:34:28.414123 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/dd0c1d7e-44ea-4e2f-a1a4-d8b7877d2069-bundle\") pod \"dd0c1d7e-44ea-4e2f-a1a4-d8b7877d2069\" (UID: \"dd0c1d7e-44ea-4e2f-a1a4-d8b7877d2069\") " Jul 07 14:34:28.414598 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:34:28.414567 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dd0c1d7e-44ea-4e2f-a1a4-d8b7877d2069-bundle" (OuterVolumeSpecName: "bundle") pod "dd0c1d7e-44ea-4e2f-a1a4-d8b7877d2069" (UID: "dd0c1d7e-44ea-4e2f-a1a4-d8b7877d2069"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 14:34:28.416170 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:34:28.416138 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dd0c1d7e-44ea-4e2f-a1a4-d8b7877d2069-kube-api-access-rcxss" (OuterVolumeSpecName: "kube-api-access-rcxss") pod "dd0c1d7e-44ea-4e2f-a1a4-d8b7877d2069" (UID: "dd0c1d7e-44ea-4e2f-a1a4-d8b7877d2069"). InnerVolumeSpecName "kube-api-access-rcxss". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 14:34:28.419941 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:34:28.419915 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dd0c1d7e-44ea-4e2f-a1a4-d8b7877d2069-util" (OuterVolumeSpecName: "util") pod "dd0c1d7e-44ea-4e2f-a1a4-d8b7877d2069" (UID: "dd0c1d7e-44ea-4e2f-a1a4-d8b7877d2069"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 14:34:28.514847 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:34:28.514818 2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rcxss\" (UniqueName: \"kubernetes.io/projected/dd0c1d7e-44ea-4e2f-a1a4-d8b7877d2069-kube-api-access-rcxss\") on node \"ip-10-0-134-42.ec2.internal\" DevicePath \"\"" Jul 07 14:34:28.514847 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:34:28.514847 2557 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/dd0c1d7e-44ea-4e2f-a1a4-d8b7877d2069-util\") on node \"ip-10-0-134-42.ec2.internal\" DevicePath \"\"" Jul 07 14:34:28.514964 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:34:28.514858 2557 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/dd0c1d7e-44ea-4e2f-a1a4-d8b7877d2069-bundle\") on node \"ip-10-0-134-42.ec2.internal\" DevicePath \"\"" Jul 07 14:34:29.158824 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:34:29.158797 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759ps8sv" Jul 07 14:34:29.159172 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:34:29.158796 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759ps8sv" event={"ID":"dd0c1d7e-44ea-4e2f-a1a4-d8b7877d2069","Type":"ContainerDied","Data":"7beac85658fadac647fed9a6c4758ce080aa135de98b6620c9998f90bfd16fcb"} Jul 07 14:34:29.159172 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:34:29.158936 2557 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7beac85658fadac647fed9a6c4758ce080aa135de98b6620c9998f90bfd16fcb" Jul 07 14:34:32.711040 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:34:32.711004 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:34:35.066961 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:34:35.066917 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:34:37.707612 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:34:37.707573 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-v9rnn"] Jul 07 14:34:37.708058 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:34:37.708040 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2870ed2a-a2da-44d8-9a7b-51b5b550469b" containerName="extract" Jul 07 14:34:37.708058 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:34:37.708058 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="2870ed2a-a2da-44d8-9a7b-51b5b550469b" containerName="extract" Jul 07 14:34:37.708181 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:34:37.708079 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="740e7355-ef13-48ab-a537-87418f967bb4" containerName="extract" Jul 07 14:34:37.708181 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:34:37.708088 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="740e7355-ef13-48ab-a537-87418f967bb4" containerName="extract" Jul 07 14:34:37.708181 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:34:37.708105 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2870ed2a-a2da-44d8-9a7b-51b5b550469b" containerName="util" Jul 07 14:34:37.708181 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:34:37.708114 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="2870ed2a-a2da-44d8-9a7b-51b5b550469b" containerName="util" Jul 07 14:34:37.708181 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:34:37.708125 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dd0c1d7e-44ea-4e2f-a1a4-d8b7877d2069" containerName="util" Jul 07 14:34:37.708181 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:34:37.708133 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="dd0c1d7e-44ea-4e2f-a1a4-d8b7877d2069" containerName="util" Jul 07 14:34:37.708181 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:34:37.708143 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="740e7355-ef13-48ab-a537-87418f967bb4" containerName="pull" Jul 07 14:34:37.708181 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:34:37.708152 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="740e7355-ef13-48ab-a537-87418f967bb4" containerName="pull" Jul 07 14:34:37.708181 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:34:37.708162 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ac44a2b1-1db5-4e2b-a3a4-7f08168b3902" containerName="util" Jul 07 14:34:37.708181 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:34:37.708170 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="ac44a2b1-1db5-4e2b-a3a4-7f08168b3902" containerName="util" Jul 07 14:34:37.708181 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:34:37.708181 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2870ed2a-a2da-44d8-9a7b-51b5b550469b" containerName="pull" Jul 07 14:34:37.708716 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:34:37.708189 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="2870ed2a-a2da-44d8-9a7b-51b5b550469b" containerName="pull" Jul 07 14:34:37.708716 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:34:37.708201 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ac44a2b1-1db5-4e2b-a3a4-7f08168b3902" containerName="extract" Jul 07 14:34:37.708716 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:34:37.708210 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="ac44a2b1-1db5-4e2b-a3a4-7f08168b3902" containerName="extract" Jul 07 14:34:37.708716 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:34:37.708225 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="740e7355-ef13-48ab-a537-87418f967bb4" containerName="util" Jul 07 14:34:37.708716 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:34:37.708232 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="740e7355-ef13-48ab-a537-87418f967bb4" containerName="util" Jul 07 14:34:37.708716 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:34:37.708241 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dd0c1d7e-44ea-4e2f-a1a4-d8b7877d2069" containerName="pull" Jul 07 14:34:37.708716 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:34:37.708248 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="dd0c1d7e-44ea-4e2f-a1a4-d8b7877d2069" containerName="pull" Jul 07 14:34:37.708716 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:34:37.708258 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ac44a2b1-1db5-4e2b-a3a4-7f08168b3902" containerName="pull" Jul 07 14:34:37.708716 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:34:37.708266 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="ac44a2b1-1db5-4e2b-a3a4-7f08168b3902" containerName="pull" Jul 07 14:34:37.708716 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:34:37.708275 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dd0c1d7e-44ea-4e2f-a1a4-d8b7877d2069" containerName="extract" Jul 07 14:34:37.708716 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:34:37.708282 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="dd0c1d7e-44ea-4e2f-a1a4-d8b7877d2069" containerName="extract" Jul 07 14:34:37.708716 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:34:37.708386 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="dd0c1d7e-44ea-4e2f-a1a4-d8b7877d2069" containerName="extract" Jul 07 14:34:37.708716 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:34:37.708399 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="740e7355-ef13-48ab-a537-87418f967bb4" containerName="extract" Jul 07 14:34:37.708716 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:34:37.708407 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="ac44a2b1-1db5-4e2b-a3a4-7f08168b3902" containerName="extract" Jul 07 14:34:37.708716 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:34:37.708419 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="2870ed2a-a2da-44d8-9a7b-51b5b550469b" containerName="extract" Jul 07 14:34:37.712922 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:34:37.712901 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-v9rnn" Jul 07 14:34:37.715761 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:34:37.715741 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-cpkbs\"" Jul 07 14:34:37.718845 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:34:37.718825 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-v9rnn"] Jul 07 14:34:37.785178 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:34:37.785153 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d5dfc\" (UniqueName: \"kubernetes.io/projected/33044d3f-ecc4-4307-ad7e-a4d16219dd93-kube-api-access-d5dfc\") pod \"limitador-operator-controller-manager-859d7b78bd-v9rnn\" (UID: \"33044d3f-ecc4-4307-ad7e-a4d16219dd93\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-v9rnn" Jul 07 14:34:37.885744 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:34:37.885711 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d5dfc\" (UniqueName: \"kubernetes.io/projected/33044d3f-ecc4-4307-ad7e-a4d16219dd93-kube-api-access-d5dfc\") pod \"limitador-operator-controller-manager-859d7b78bd-v9rnn\" (UID: \"33044d3f-ecc4-4307-ad7e-a4d16219dd93\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-v9rnn" Jul 07 14:34:37.903327 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:34:37.903300 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d5dfc\" (UniqueName: \"kubernetes.io/projected/33044d3f-ecc4-4307-ad7e-a4d16219dd93-kube-api-access-d5dfc\") pod \"limitador-operator-controller-manager-859d7b78bd-v9rnn\" (UID: \"33044d3f-ecc4-4307-ad7e-a4d16219dd93\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-v9rnn" Jul 07 14:34:38.024678 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:34:38.024589 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-v9rnn" Jul 07 14:34:38.155968 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:34:38.155946 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-v9rnn"] Jul 07 14:34:38.157500 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:34:38.157474 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod33044d3f_ecc4_4307_ad7e_a4d16219dd93.slice/crio-de9937b888fe251bf360e86a6493ae8c5710b536ee56f7e4d6721d6b123b37f4 WatchSource:0}: Error finding container de9937b888fe251bf360e86a6493ae8c5710b536ee56f7e4d6721d6b123b37f4: Status 404 returned error can't find the container with id de9937b888fe251bf360e86a6493ae8c5710b536ee56f7e4d6721d6b123b37f4 Jul 07 14:34:38.205750 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:34:38.205721 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-v9rnn" event={"ID":"33044d3f-ecc4-4307-ad7e-a4d16219dd93","Type":"ContainerStarted","Data":"de9937b888fe251bf360e86a6493ae8c5710b536ee56f7e4d6721d6b123b37f4"} Jul 07 14:34:38.659235 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:34:38.659199 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:34:38.759597 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:34:38.759559 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:34:41.224279 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:34:41.224239 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-v9rnn" event={"ID":"33044d3f-ecc4-4307-ad7e-a4d16219dd93","Type":"ContainerStarted","Data":"232af2531ed381f27b8be06c28a8a9c742dbbc68539b7963a576ec89f4a7ca1a"} Jul 07 14:34:41.224736 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:34:41.224397 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-v9rnn" Jul 07 14:34:41.248667 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:34:41.248616 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-v9rnn" podStartSLOduration=2.007682048 podStartE2EDuration="4.248598233s" totalImagesPullingTime="2.240916185s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 14:34:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:34:41.247795915 +0000 UTC m=+517.956934294" watchObservedRunningTime="2026-07-07 14:34:41.248598233 +0000 UTC m=+517.957736614" Jul 07 14:34:43.227577 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:34:43.227547 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:34:43.676593 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:34:43.676560 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:34:45.106829 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:34:45.106797 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:34:45.475890 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:34:45.475812 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-vp6jj"] Jul 07 14:34:45.479856 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:34:45.479839 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-vp6jj" Jul 07 14:34:45.481881 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:34:45.481861 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-j29lt\"" Jul 07 14:34:45.490760 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:34:45.490740 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-vp6jj"] Jul 07 14:34:45.652823 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:34:45.652784 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-blcwc\" (UniqueName: \"kubernetes.io/projected/53de9697-efbf-4494-8566-f32c55621a29-kube-api-access-blcwc\") pod \"authorino-operator-75cc8d5cb7-vp6jj\" (UID: \"53de9697-efbf-4494-8566-f32c55621a29\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-vp6jj" Jul 07 14:34:45.753724 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:34:45.753627 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-blcwc\" (UniqueName: \"kubernetes.io/projected/53de9697-efbf-4494-8566-f32c55621a29-kube-api-access-blcwc\") pod \"authorino-operator-75cc8d5cb7-vp6jj\" (UID: \"53de9697-efbf-4494-8566-f32c55621a29\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-vp6jj" Jul 07 14:34:45.762109 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:34:45.762080 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-blcwc\" (UniqueName: \"kubernetes.io/projected/53de9697-efbf-4494-8566-f32c55621a29-kube-api-access-blcwc\") pod \"authorino-operator-75cc8d5cb7-vp6jj\" (UID: \"53de9697-efbf-4494-8566-f32c55621a29\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-vp6jj" Jul 07 14:34:45.790844 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:34:45.790812 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-vp6jj" Jul 07 14:34:45.927083 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:34:45.927057 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-vp6jj"] Jul 07 14:34:45.928594 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:34:45.928568 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod53de9697_efbf_4494_8566_f32c55621a29.slice/crio-2481bd579e37e912513e40d7f9d09eb8f47fcc64871f5512f266300a4dfb8f2a WatchSource:0}: Error finding container 2481bd579e37e912513e40d7f9d09eb8f47fcc64871f5512f266300a4dfb8f2a: Status 404 returned error can't find the container with id 2481bd579e37e912513e40d7f9d09eb8f47fcc64871f5512f266300a4dfb8f2a Jul 07 14:34:46.258289 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:34:46.258257 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-vp6jj" event={"ID":"53de9697-efbf-4494-8566-f32c55621a29","Type":"ContainerStarted","Data":"2481bd579e37e912513e40d7f9d09eb8f47fcc64871f5512f266300a4dfb8f2a"} Jul 07 14:34:46.650731 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:34:46.650688 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:34:47.562086 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:34:47.562022 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:34:47.675019 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:34:47.674953 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:34:48.276499 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:34:48.276464 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-vp6jj" event={"ID":"53de9697-efbf-4494-8566-f32c55621a29","Type":"ContainerStarted","Data":"83f2b6ce3ba9f06f67d7339a1d9845d0c285c6f716c4b1a738b4f37061b6e78b"} Jul 07 14:34:48.276671 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:34:48.276560 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-vp6jj" Jul 07 14:34:48.295060 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:34:48.295018 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-75cc8d5cb7-vp6jj" podStartSLOduration=1.162257219 podStartE2EDuration="3.295005091s" totalImagesPullingTime="2.132747872s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 14:34:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:34:48.292804442 +0000 UTC m=+525.001942821" watchObservedRunningTime="2026-07-07 14:34:48.295005091 +0000 UTC m=+525.004143469" Jul 07 14:34:49.477040 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:34:49.477005 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:34:50.839497 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:34:50.839459 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:34:50.939721 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:34:50.939694 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:34:52.231450 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:34:52.231418 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-v9rnn" Jul 07 14:34:53.945408 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:34:53.945373 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:34:54.112501 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:34:54.112465 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:34:54.230811 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:34:54.230739 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:34:54.294302 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:34:54.294277 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:34:59.284179 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:34:59.284106 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-vp6jj" Jul 07 14:35:02.917019 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:35:02.916972 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:35:03.162068 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:35:03.162028 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:35:04.302301 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:35:04.302268 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7s7l6"] Jul 07 14:35:04.305490 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:35:04.305470 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7s7l6" Jul 07 14:35:04.308651 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:35:04.308631 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-f2kgf\"" Jul 07 14:35:04.309325 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:35:04.309304 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-v9rnn"] Jul 07 14:35:04.309501 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:35:04.309467 2557 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-v9rnn" podUID="33044d3f-ecc4-4307-ad7e-a4d16219dd93" containerName="manager" containerID="cri-o://232af2531ed381f27b8be06c28a8a9c742dbbc68539b7963a576ec89f4a7ca1a" gracePeriod=2 Jul 07 14:35:04.323509 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:35:04.323484 2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-v9rnn"] Jul 07 14:35:04.351281 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:35:04.351252 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7s7l6"] Jul 07 14:35:04.391233 ip-10-0-134-42 kubenswrapper[2557]: E0707 14:35:04.391185 2557 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-v9rnn\" is forbidden: User \"system:node:ip-10-0-134-42.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-134-42.ec2.internal' and this object" podUID="33044d3f-ecc4-4307-ad7e-a4d16219dd93" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-v9rnn" Jul 07 14:35:04.400026 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:35:04.400001 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/8f5049b8-8e7a-42e0-8a5a-1ab93b34daee-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-7s7l6\" (UID: \"8f5049b8-8e7a-42e0-8a5a-1ab93b34daee\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7s7l6" Jul 07 14:35:04.400159 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:35:04.400105 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jrm7k\" (UniqueName: \"kubernetes.io/projected/8f5049b8-8e7a-42e0-8a5a-1ab93b34daee-kube-api-access-jrm7k\") pod \"kuadrant-operator-controller-manager-8684c9ff49-7s7l6\" (UID: \"8f5049b8-8e7a-42e0-8a5a-1ab93b34daee\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7s7l6" Jul 07 14:35:04.501246 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:35:04.501212 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jrm7k\" (UniqueName: \"kubernetes.io/projected/8f5049b8-8e7a-42e0-8a5a-1ab93b34daee-kube-api-access-jrm7k\") pod \"kuadrant-operator-controller-manager-8684c9ff49-7s7l6\" (UID: \"8f5049b8-8e7a-42e0-8a5a-1ab93b34daee\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7s7l6" Jul 07 14:35:04.501367 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:35:04.501296 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/8f5049b8-8e7a-42e0-8a5a-1ab93b34daee-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-7s7l6\" (UID: \"8f5049b8-8e7a-42e0-8a5a-1ab93b34daee\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7s7l6" Jul 07 14:35:04.501677 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:35:04.501659 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/8f5049b8-8e7a-42e0-8a5a-1ab93b34daee-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-7s7l6\" (UID: \"8f5049b8-8e7a-42e0-8a5a-1ab93b34daee\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7s7l6" Jul 07 14:35:04.511177 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:35:04.511157 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jrm7k\" (UniqueName: \"kubernetes.io/projected/8f5049b8-8e7a-42e0-8a5a-1ab93b34daee-kube-api-access-jrm7k\") pod \"kuadrant-operator-controller-manager-8684c9ff49-7s7l6\" (UID: \"8f5049b8-8e7a-42e0-8a5a-1ab93b34daee\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7s7l6" Jul 07 14:35:04.548827 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:35:04.548810 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-v9rnn" Jul 07 14:35:04.550598 ip-10-0-134-42 kubenswrapper[2557]: E0707 14:35:04.550575 2557 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-v9rnn\" is forbidden: User \"system:node:ip-10-0-134-42.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-134-42.ec2.internal' and this object" podUID="33044d3f-ecc4-4307-ad7e-a4d16219dd93" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-v9rnn" Jul 07 14:35:04.602454 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:35:04.602419 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d5dfc\" (UniqueName: \"kubernetes.io/projected/33044d3f-ecc4-4307-ad7e-a4d16219dd93-kube-api-access-d5dfc\") pod \"33044d3f-ecc4-4307-ad7e-a4d16219dd93\" (UID: \"33044d3f-ecc4-4307-ad7e-a4d16219dd93\") " Jul 07 14:35:04.604259 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:35:04.604232 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/33044d3f-ecc4-4307-ad7e-a4d16219dd93-kube-api-access-d5dfc" (OuterVolumeSpecName: "kube-api-access-d5dfc") pod "33044d3f-ecc4-4307-ad7e-a4d16219dd93" (UID: "33044d3f-ecc4-4307-ad7e-a4d16219dd93"). InnerVolumeSpecName "kube-api-access-d5dfc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 14:35:04.618420 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:35:04.618404 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7s7l6" Jul 07 14:35:04.704206 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:35:04.704175 2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d5dfc\" (UniqueName: \"kubernetes.io/projected/33044d3f-ecc4-4307-ad7e-a4d16219dd93-kube-api-access-d5dfc\") on node \"ip-10-0-134-42.ec2.internal\" DevicePath \"\"" Jul 07 14:35:04.764946 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:35:04.764923 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7s7l6"] Jul 07 14:35:04.767023 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:35:04.766998 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8f5049b8_8e7a_42e0_8a5a_1ab93b34daee.slice/crio-6856c901624d6f1000427eadde69df0cd3607c9b7717d42b14befcba3039c19c WatchSource:0}: Error finding container 6856c901624d6f1000427eadde69df0cd3607c9b7717d42b14befcba3039c19c: Status 404 returned error can't find the container with id 6856c901624d6f1000427eadde69df0cd3607c9b7717d42b14befcba3039c19c Jul 07 14:35:05.194407 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:35:05.194364 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:35:05.371331 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:35:05.371296 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7s7l6" event={"ID":"8f5049b8-8e7a-42e0-8a5a-1ab93b34daee","Type":"ContainerStarted","Data":"6856c901624d6f1000427eadde69df0cd3607c9b7717d42b14befcba3039c19c"} Jul 07 14:35:05.372708 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:35:05.372685 2557 generic.go:356] "Generic (PLEG): container finished" podID="33044d3f-ecc4-4307-ad7e-a4d16219dd93" containerID="232af2531ed381f27b8be06c28a8a9c742dbbc68539b7963a576ec89f4a7ca1a" exitCode=0 Jul 07 14:35:05.372797 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:35:05.372729 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-v9rnn" Jul 07 14:35:05.372797 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:35:05.372752 2557 scope.go:117] "RemoveContainer" containerID="232af2531ed381f27b8be06c28a8a9c742dbbc68539b7963a576ec89f4a7ca1a" Jul 07 14:35:05.380795 ip-10-0-134-42 kubenswrapper[2557]: E0707 14:35:05.380768 2557 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-v9rnn\" is forbidden: User \"system:node:ip-10-0-134-42.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-134-42.ec2.internal' and this object" podUID="33044d3f-ecc4-4307-ad7e-a4d16219dd93" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-v9rnn" Jul 07 14:35:05.381844 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:35:05.381829 2557 scope.go:117] "RemoveContainer" containerID="232af2531ed381f27b8be06c28a8a9c742dbbc68539b7963a576ec89f4a7ca1a" Jul 07 14:35:05.382116 ip-10-0-134-42 kubenswrapper[2557]: E0707 14:35:05.382100 2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"232af2531ed381f27b8be06c28a8a9c742dbbc68539b7963a576ec89f4a7ca1a\": container with ID starting with 232af2531ed381f27b8be06c28a8a9c742dbbc68539b7963a576ec89f4a7ca1a not found: ID does not exist" containerID="232af2531ed381f27b8be06c28a8a9c742dbbc68539b7963a576ec89f4a7ca1a" Jul 07 14:35:05.382166 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:35:05.382123 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"232af2531ed381f27b8be06c28a8a9c742dbbc68539b7963a576ec89f4a7ca1a"} err="failed to get container status \"232af2531ed381f27b8be06c28a8a9c742dbbc68539b7963a576ec89f4a7ca1a\": rpc error: code = NotFound desc = could not find container \"232af2531ed381f27b8be06c28a8a9c742dbbc68539b7963a576ec89f4a7ca1a\": container with ID starting with 232af2531ed381f27b8be06c28a8a9c742dbbc68539b7963a576ec89f4a7ca1a not found: ID does not exist" Jul 07 14:35:05.394211 ip-10-0-134-42 kubenswrapper[2557]: E0707 14:35:05.394186 2557 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-v9rnn\" is forbidden: User \"system:node:ip-10-0-134-42.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-134-42.ec2.internal' and this object" podUID="33044d3f-ecc4-4307-ad7e-a4d16219dd93" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-v9rnn" Jul 07 14:35:05.435014 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:35:05.434990 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:35:05.570015 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:35:05.569947 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:35:05.744468 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:35:05.744417 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:35:05.834955 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:35:05.834927 2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="33044d3f-ecc4-4307-ad7e-a4d16219dd93" path="/var/lib/kubelet/pods/33044d3f-ecc4-4307-ad7e-a4d16219dd93/volumes" Jul 07 14:35:09.158235 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:35:09.158198 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:35:09.260407 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:35:09.260346 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:35:09.716049 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:35:09.716003 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:35:09.813441 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:35:09.813366 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:35:11.409856 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:35:11.409821 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7s7l6" event={"ID":"8f5049b8-8e7a-42e0-8a5a-1ab93b34daee","Type":"ContainerStarted","Data":"daed825ed8fb27f1f13b24b590055325f831567d81fec2c262935c25e280fbf1"} Jul 07 14:35:11.410371 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:35:11.409903 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7s7l6" Jul 07 14:35:11.429208 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:35:11.429160 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7s7l6" podStartSLOduration=1.723869034 podStartE2EDuration="7.429147747s" totalImagesPullingTime="5.705278713s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 14:35:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:35:11.428069267 +0000 UTC m=+548.137207647" watchObservedRunningTime="2026-07-07 14:35:11.429147747 +0000 UTC m=+548.138286126" Jul 07 14:35:11.863140 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:35:11.863102 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:35:11.969838 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:35:11.969798 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:35:17.416275 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:35:17.416241 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7s7l6"] Jul 07 14:35:17.416723 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:35:17.416415 2557 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7s7l6" podUID="8f5049b8-8e7a-42e0-8a5a-1ab93b34daee" containerName="manager" containerID="cri-o://daed825ed8fb27f1f13b24b590055325f831567d81fec2c262935c25e280fbf1" gracePeriod=10 Jul 07 14:35:17.418176 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:35:17.418156 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7s7l6" Jul 07 14:35:17.661632 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:35:17.661609 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7s7l6" Jul 07 14:35:17.710398 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:35:17.710311 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/8f5049b8-8e7a-42e0-8a5a-1ab93b34daee-extensions-socket-volume\") pod \"8f5049b8-8e7a-42e0-8a5a-1ab93b34daee\" (UID: \"8f5049b8-8e7a-42e0-8a5a-1ab93b34daee\") " Jul 07 14:35:17.710562 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:35:17.710401 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jrm7k\" (UniqueName: \"kubernetes.io/projected/8f5049b8-8e7a-42e0-8a5a-1ab93b34daee-kube-api-access-jrm7k\") pod \"8f5049b8-8e7a-42e0-8a5a-1ab93b34daee\" (UID: \"8f5049b8-8e7a-42e0-8a5a-1ab93b34daee\") " Jul 07 14:35:17.710837 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:35:17.710812 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8f5049b8-8e7a-42e0-8a5a-1ab93b34daee-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "8f5049b8-8e7a-42e0-8a5a-1ab93b34daee" (UID: "8f5049b8-8e7a-42e0-8a5a-1ab93b34daee"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 14:35:17.712358 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:35:17.712335 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8f5049b8-8e7a-42e0-8a5a-1ab93b34daee-kube-api-access-jrm7k" (OuterVolumeSpecName: "kube-api-access-jrm7k") pod "8f5049b8-8e7a-42e0-8a5a-1ab93b34daee" (UID: "8f5049b8-8e7a-42e0-8a5a-1ab93b34daee"). InnerVolumeSpecName "kube-api-access-jrm7k". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 14:35:17.812030 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:35:17.811979 2557 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/8f5049b8-8e7a-42e0-8a5a-1ab93b34daee-extensions-socket-volume\") on node \"ip-10-0-134-42.ec2.internal\" DevicePath \"\"" Jul 07 14:35:17.812030 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:35:17.812025 2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jrm7k\" (UniqueName: \"kubernetes.io/projected/8f5049b8-8e7a-42e0-8a5a-1ab93b34daee-kube-api-access-jrm7k\") on node \"ip-10-0-134-42.ec2.internal\" DevicePath \"\"" Jul 07 14:35:18.447608 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:35:18.447578 2557 generic.go:356] "Generic (PLEG): container finished" podID="8f5049b8-8e7a-42e0-8a5a-1ab93b34daee" containerID="daed825ed8fb27f1f13b24b590055325f831567d81fec2c262935c25e280fbf1" exitCode=0 Jul 07 14:35:18.448034 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:35:18.447629 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7s7l6" Jul 07 14:35:18.448034 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:35:18.447629 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7s7l6" event={"ID":"8f5049b8-8e7a-42e0-8a5a-1ab93b34daee","Type":"ContainerDied","Data":"daed825ed8fb27f1f13b24b590055325f831567d81fec2c262935c25e280fbf1"} Jul 07 14:35:18.448034 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:35:18.447691 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7s7l6" event={"ID":"8f5049b8-8e7a-42e0-8a5a-1ab93b34daee","Type":"ContainerDied","Data":"6856c901624d6f1000427eadde69df0cd3607c9b7717d42b14befcba3039c19c"} Jul 07 14:35:18.448034 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:35:18.447721 2557 scope.go:117] "RemoveContainer" containerID="daed825ed8fb27f1f13b24b590055325f831567d81fec2c262935c25e280fbf1" Jul 07 14:35:18.456304 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:35:18.456288 2557 scope.go:117] "RemoveContainer" containerID="daed825ed8fb27f1f13b24b590055325f831567d81fec2c262935c25e280fbf1" Jul 07 14:35:18.456590 ip-10-0-134-42 kubenswrapper[2557]: E0707 14:35:18.456571 2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"daed825ed8fb27f1f13b24b590055325f831567d81fec2c262935c25e280fbf1\": container with ID starting with daed825ed8fb27f1f13b24b590055325f831567d81fec2c262935c25e280fbf1 not found: ID does not exist" containerID="daed825ed8fb27f1f13b24b590055325f831567d81fec2c262935c25e280fbf1" Jul 07 14:35:18.456644 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:35:18.456599 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"daed825ed8fb27f1f13b24b590055325f831567d81fec2c262935c25e280fbf1"} err="failed to get container status \"daed825ed8fb27f1f13b24b590055325f831567d81fec2c262935c25e280fbf1\": rpc error: code = NotFound desc = could not find container \"daed825ed8fb27f1f13b24b590055325f831567d81fec2c262935c25e280fbf1\": container with ID starting with daed825ed8fb27f1f13b24b590055325f831567d81fec2c262935c25e280fbf1 not found: ID does not exist" Jul 07 14:35:18.468646 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:35:18.468618 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7s7l6"] Jul 07 14:35:18.475369 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:35:18.475336 2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7s7l6"] Jul 07 14:35:19.835842 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:35:19.835801 2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8f5049b8-8e7a-42e0-8a5a-1ab93b34daee" path="/var/lib/kubelet/pods/8f5049b8-8e7a-42e0-8a5a-1ab93b34daee/volumes" Jul 07 14:35:21.262205 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:35:21.262171 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:35:21.329599 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:35:21.329570 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:35:21.870646 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:35:21.870614 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:35:21.932160 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:35:21.932124 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:35:26.863373 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:35:26.863326 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:35:27.312226 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:35:27.312141 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:35:32.268104 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:35:32.268067 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:35:32.374419 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:35:32.374385 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:35:33.866378 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:35:33.866348 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-8cnl6"] Jul 07 14:35:33.866876 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:35:33.866709 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8f5049b8-8e7a-42e0-8a5a-1ab93b34daee" containerName="manager" Jul 07 14:35:33.866876 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:35:33.866725 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="8f5049b8-8e7a-42e0-8a5a-1ab93b34daee" containerName="manager" Jul 07 14:35:33.866876 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:35:33.866744 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="33044d3f-ecc4-4307-ad7e-a4d16219dd93" containerName="manager" Jul 07 14:35:33.866876 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:35:33.866749 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="33044d3f-ecc4-4307-ad7e-a4d16219dd93" containerName="manager" Jul 07 14:35:33.866876 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:35:33.866853 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="8f5049b8-8e7a-42e0-8a5a-1ab93b34daee" containerName="manager" Jul 07 14:35:33.866876 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:35:33.866870 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="33044d3f-ecc4-4307-ad7e-a4d16219dd93" containerName="manager" Jul 07 14:35:33.869037 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:35:33.869019 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-8cnl6" Jul 07 14:35:33.871175 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:35:33.871145 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-t84px\"" Jul 07 14:35:33.884454 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:35:33.884433 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-8cnl6"] Jul 07 14:35:33.891989 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:35:33.891949 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:35:33.953089 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:35:33.953064 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1861c4fd-2974-42f3-96e8-79601e052d18-workload-certs\") pod \"maas-default-gateway-openshift-default-5896d4dd49-8cnl6\" (UID: \"1861c4fd-2974-42f3-96e8-79601e052d18\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-8cnl6" Jul 07 14:35:33.953200 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:35:33.953092 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1861c4fd-2974-42f3-96e8-79601e052d18-istio-data\") pod \"maas-default-gateway-openshift-default-5896d4dd49-8cnl6\" (UID: \"1861c4fd-2974-42f3-96e8-79601e052d18\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-8cnl6" Jul 07 14:35:33.953200 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:35:33.953125 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1861c4fd-2974-42f3-96e8-79601e052d18-workload-socket\") pod \"maas-default-gateway-openshift-default-5896d4dd49-8cnl6\" (UID: \"1861c4fd-2974-42f3-96e8-79601e052d18\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-8cnl6" Jul 07 14:35:33.953200 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:35:33.953146 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1861c4fd-2974-42f3-96e8-79601e052d18-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5896d4dd49-8cnl6\" (UID: \"1861c4fd-2974-42f3-96e8-79601e052d18\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-8cnl6" Jul 07 14:35:33.953415 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:35:33.953219 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5zhdj\" (UniqueName: \"kubernetes.io/projected/1861c4fd-2974-42f3-96e8-79601e052d18-kube-api-access-5zhdj\") pod \"maas-default-gateway-openshift-default-5896d4dd49-8cnl6\" (UID: \"1861c4fd-2974-42f3-96e8-79601e052d18\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-8cnl6" Jul 07 14:35:33.953415 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:35:33.953263 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1861c4fd-2974-42f3-96e8-79601e052d18-credential-socket\") pod \"maas-default-gateway-openshift-default-5896d4dd49-8cnl6\" (UID: \"1861c4fd-2974-42f3-96e8-79601e052d18\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-8cnl6" Jul 07 14:35:33.953415 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:35:33.953297 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1861c4fd-2974-42f3-96e8-79601e052d18-istio-token\") pod \"maas-default-gateway-openshift-default-5896d4dd49-8cnl6\" (UID: \"1861c4fd-2974-42f3-96e8-79601e052d18\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-8cnl6" Jul 07 14:35:33.953415 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:35:33.953327 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1861c4fd-2974-42f3-96e8-79601e052d18-istio-envoy\") pod \"maas-default-gateway-openshift-default-5896d4dd49-8cnl6\" (UID: \"1861c4fd-2974-42f3-96e8-79601e052d18\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-8cnl6" Jul 07 14:35:33.953415 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:35:33.953354 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1861c4fd-2974-42f3-96e8-79601e052d18-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5896d4dd49-8cnl6\" (UID: \"1861c4fd-2974-42f3-96e8-79601e052d18\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-8cnl6" Jul 07 14:35:34.054438 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:35:34.054408 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1861c4fd-2974-42f3-96e8-79601e052d18-credential-socket\") pod \"maas-default-gateway-openshift-default-5896d4dd49-8cnl6\" (UID: \"1861c4fd-2974-42f3-96e8-79601e052d18\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-8cnl6" Jul 07 14:35:34.054587 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:35:34.054443 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1861c4fd-2974-42f3-96e8-79601e052d18-istio-token\") pod \"maas-default-gateway-openshift-default-5896d4dd49-8cnl6\" (UID: \"1861c4fd-2974-42f3-96e8-79601e052d18\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-8cnl6" Jul 07 14:35:34.054587 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:35:34.054471 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1861c4fd-2974-42f3-96e8-79601e052d18-istio-envoy\") pod \"maas-default-gateway-openshift-default-5896d4dd49-8cnl6\" (UID: \"1861c4fd-2974-42f3-96e8-79601e052d18\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-8cnl6" Jul 07 14:35:34.054720 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:35:34.054590 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1861c4fd-2974-42f3-96e8-79601e052d18-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5896d4dd49-8cnl6\" (UID: \"1861c4fd-2974-42f3-96e8-79601e052d18\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-8cnl6" Jul 07 14:35:34.054720 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:35:34.054653 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1861c4fd-2974-42f3-96e8-79601e052d18-workload-certs\") pod \"maas-default-gateway-openshift-default-5896d4dd49-8cnl6\" (UID: \"1861c4fd-2974-42f3-96e8-79601e052d18\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-8cnl6" Jul 07 14:35:34.054720 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:35:34.054684 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1861c4fd-2974-42f3-96e8-79601e052d18-istio-data\") pod \"maas-default-gateway-openshift-default-5896d4dd49-8cnl6\" (UID: \"1861c4fd-2974-42f3-96e8-79601e052d18\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-8cnl6" Jul 07 14:35:34.054873 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:35:34.054757 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1861c4fd-2974-42f3-96e8-79601e052d18-workload-socket\") pod \"maas-default-gateway-openshift-default-5896d4dd49-8cnl6\" (UID: \"1861c4fd-2974-42f3-96e8-79601e052d18\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-8cnl6" Jul 07 14:35:34.054873 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:35:34.054784 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1861c4fd-2974-42f3-96e8-79601e052d18-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5896d4dd49-8cnl6\" (UID: \"1861c4fd-2974-42f3-96e8-79601e052d18\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-8cnl6" Jul 07 14:35:34.054873 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:35:34.054788 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1861c4fd-2974-42f3-96e8-79601e052d18-credential-socket\") pod \"maas-default-gateway-openshift-default-5896d4dd49-8cnl6\" (UID: \"1861c4fd-2974-42f3-96e8-79601e052d18\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-8cnl6" Jul 07 14:35:34.054873 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:35:34.054844 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5zhdj\" (UniqueName: \"kubernetes.io/projected/1861c4fd-2974-42f3-96e8-79601e052d18-kube-api-access-5zhdj\") pod \"maas-default-gateway-openshift-default-5896d4dd49-8cnl6\" (UID: \"1861c4fd-2974-42f3-96e8-79601e052d18\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-8cnl6" Jul 07 14:35:34.055081 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:35:34.054978 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1861c4fd-2974-42f3-96e8-79601e052d18-workload-certs\") pod \"maas-default-gateway-openshift-default-5896d4dd49-8cnl6\" (UID: \"1861c4fd-2974-42f3-96e8-79601e052d18\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-8cnl6" Jul 07 14:35:34.055141 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:35:34.055102 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1861c4fd-2974-42f3-96e8-79601e052d18-istio-data\") pod \"maas-default-gateway-openshift-default-5896d4dd49-8cnl6\" (UID: \"1861c4fd-2974-42f3-96e8-79601e052d18\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-8cnl6" Jul 07 14:35:34.055201 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:35:34.055184 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1861c4fd-2974-42f3-96e8-79601e052d18-workload-socket\") pod \"maas-default-gateway-openshift-default-5896d4dd49-8cnl6\" (UID: \"1861c4fd-2974-42f3-96e8-79601e052d18\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-8cnl6" Jul 07 14:35:34.055299 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:35:34.055278 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1861c4fd-2974-42f3-96e8-79601e052d18-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5896d4dd49-8cnl6\" (UID: \"1861c4fd-2974-42f3-96e8-79601e052d18\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-8cnl6" Jul 07 14:35:34.056597 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:35:34.056577 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1861c4fd-2974-42f3-96e8-79601e052d18-istio-envoy\") pod \"maas-default-gateway-openshift-default-5896d4dd49-8cnl6\" (UID: \"1861c4fd-2974-42f3-96e8-79601e052d18\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-8cnl6" Jul 07 14:35:34.056987 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:35:34.056970 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1861c4fd-2974-42f3-96e8-79601e052d18-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5896d4dd49-8cnl6\" (UID: \"1861c4fd-2974-42f3-96e8-79601e052d18\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-8cnl6" Jul 07 14:35:34.061853 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:35:34.061829 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1861c4fd-2974-42f3-96e8-79601e052d18-istio-token\") pod \"maas-default-gateway-openshift-default-5896d4dd49-8cnl6\" (UID: \"1861c4fd-2974-42f3-96e8-79601e052d18\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-8cnl6" Jul 07 14:35:34.062013 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:35:34.061997 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5zhdj\" (UniqueName: \"kubernetes.io/projected/1861c4fd-2974-42f3-96e8-79601e052d18-kube-api-access-5zhdj\") pod \"maas-default-gateway-openshift-default-5896d4dd49-8cnl6\" (UID: \"1861c4fd-2974-42f3-96e8-79601e052d18\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-8cnl6" Jul 07 14:35:34.180341 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:35:34.180283 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-8cnl6" Jul 07 14:35:34.250413 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:35:34.250380 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:35:34.315672 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:35:34.315650 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-8cnl6"] Jul 07 14:35:34.317369 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:35:34.317343 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1861c4fd_2974_42f3_96e8_79601e052d18.slice/crio-e83f28e9e64b04076640875db541a4e5c3d3b13659961d45f85bd524c86c069a WatchSource:0}: Error finding container e83f28e9e64b04076640875db541a4e5c3d3b13659961d45f85bd524c86c069a: Status 404 returned error can't find the container with id e83f28e9e64b04076640875db541a4e5c3d3b13659961d45f85bd524c86c069a Jul 07 14:35:34.535351 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:35:34.535264 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-8cnl6" event={"ID":"1861c4fd-2974-42f3-96e8-79601e052d18","Type":"ContainerStarted","Data":"e83f28e9e64b04076640875db541a4e5c3d3b13659961d45f85bd524c86c069a"} Jul 07 14:35:36.464364 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:35:36.464330 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:35:36.519200 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:35:36.519165 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:35:36.903128 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:35:36.903087 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 07 14:35:36.903254 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:35:36.903168 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 07 14:35:36.903254 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:35:36.903208 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 07 14:35:37.553807 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:35:37.553774 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-8cnl6" event={"ID":"1861c4fd-2974-42f3-96e8-79601e052d18","Type":"ContainerStarted","Data":"cedd1d73a62e75103cbdbf9ac31733dd6b6ad5bcbf58aa922b768c0b08c1786c"} Jul 07 14:35:37.576118 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:35:37.576072 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-8cnl6" podStartSLOduration=1.992602816 podStartE2EDuration="4.57605695s" totalImagesPullingTime="2.583454134s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 14:35:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:35:37.573161951 +0000 UTC m=+574.282300354" watchObservedRunningTime="2026-07-07 14:35:37.57605695 +0000 UTC m=+574.285195329" Jul 07 14:35:38.180823 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:35:38.180787 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-8cnl6" Jul 07 14:35:38.185986 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:35:38.185962 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-8cnl6" Jul 07 14:35:38.560889 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:35:38.560804 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-8cnl6" Jul 07 14:35:38.561701 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:35:38.561684 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-8cnl6" Jul 07 14:35:38.698840 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:35:38.698810 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-gh87n"] Jul 07 14:35:38.701185 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:35:38.701157 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:35:38.701705 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:35:38.701690 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-gh87n" Jul 07 14:35:38.703741 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:35:38.703721 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" Jul 07 14:35:38.703844 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:35:38.703755 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-gbxt7\"" Jul 07 14:35:38.713688 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:35:38.713670 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-gh87n"] Jul 07 14:35:38.797412 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:35:38.797373 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tkg6h\" (UniqueName: \"kubernetes.io/projected/452beab1-a636-4d39-b971-f7de1619fb47-kube-api-access-tkg6h\") pod \"limitador-limitador-64c94fd64f-gh87n\" (UID: \"452beab1-a636-4d39-b971-f7de1619fb47\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-gh87n" Jul 07 14:35:38.797588 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:35:38.797455 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/452beab1-a636-4d39-b971-f7de1619fb47-config-file\") pod \"limitador-limitador-64c94fd64f-gh87n\" (UID: \"452beab1-a636-4d39-b971-f7de1619fb47\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-gh87n" Jul 07 14:35:38.803268 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:35:38.803243 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-gh87n"] Jul 07 14:35:38.805170 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:35:38.805141 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:35:38.898122 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:35:38.898097 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tkg6h\" (UniqueName: \"kubernetes.io/projected/452beab1-a636-4d39-b971-f7de1619fb47-kube-api-access-tkg6h\") pod \"limitador-limitador-64c94fd64f-gh87n\" (UID: \"452beab1-a636-4d39-b971-f7de1619fb47\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-gh87n" Jul 07 14:35:38.898440 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:35:38.898412 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/452beab1-a636-4d39-b971-f7de1619fb47-config-file\") pod \"limitador-limitador-64c94fd64f-gh87n\" (UID: \"452beab1-a636-4d39-b971-f7de1619fb47\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-gh87n" Jul 07 14:35:38.899172 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:35:38.899148 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/452beab1-a636-4d39-b971-f7de1619fb47-config-file\") pod \"limitador-limitador-64c94fd64f-gh87n\" (UID: \"452beab1-a636-4d39-b971-f7de1619fb47\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-gh87n" Jul 07 14:35:38.909021 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:35:38.908993 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:35:38.910482 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:35:38.910462 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tkg6h\" (UniqueName: \"kubernetes.io/projected/452beab1-a636-4d39-b971-f7de1619fb47-kube-api-access-tkg6h\") pod \"limitador-limitador-64c94fd64f-gh87n\" (UID: \"452beab1-a636-4d39-b971-f7de1619fb47\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-gh87n" Jul 07 14:35:39.011631 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:35:39.011608 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-gh87n" Jul 07 14:35:39.139384 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:35:39.139354 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-gh87n"] Jul 07 14:35:39.141472 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:35:39.141445 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod452beab1_a636_4d39_b971_f7de1619fb47.slice/crio-7c07939c4d655d6511fc9bb134644c1db59af60550fac3eeae1e471a21ee9dba WatchSource:0}: Error finding container 7c07939c4d655d6511fc9bb134644c1db59af60550fac3eeae1e471a21ee9dba: Status 404 returned error can't find the container with id 7c07939c4d655d6511fc9bb134644c1db59af60550fac3eeae1e471a21ee9dba Jul 07 14:35:39.499767 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:35:39.499735 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-58cd955675-5p6wn"] Jul 07 14:35:39.502911 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:35:39.502891 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-5p6wn" Jul 07 14:35:39.504831 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:35:39.504809 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-d8cm8\"" Jul 07 14:35:39.507853 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:35:39.507832 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-5p6wn"] Jul 07 14:35:39.523757 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:35:39.523734 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:35:39.568066 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:35:39.568028 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-gh87n" event={"ID":"452beab1-a636-4d39-b971-f7de1619fb47","Type":"ContainerStarted","Data":"7c07939c4d655d6511fc9bb134644c1db59af60550fac3eeae1e471a21ee9dba"} Jul 07 14:35:39.594915 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:35:39.594894 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:35:39.604008 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:35:39.603989 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4wg2h\" (UniqueName: \"kubernetes.io/projected/a679630b-4858-40b0-92ac-b4fcf71409cd-kube-api-access-4wg2h\") pod \"authorino-58cd955675-5p6wn\" (UID: \"a679630b-4858-40b0-92ac-b4fcf71409cd\") " pod="kuadrant-system/authorino-58cd955675-5p6wn" Jul 07 14:35:39.705468 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:35:39.705444 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4wg2h\" (UniqueName: \"kubernetes.io/projected/a679630b-4858-40b0-92ac-b4fcf71409cd-kube-api-access-4wg2h\") pod \"authorino-58cd955675-5p6wn\" (UID: \"a679630b-4858-40b0-92ac-b4fcf71409cd\") " pod="kuadrant-system/authorino-58cd955675-5p6wn" Jul 07 14:35:39.712932 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:35:39.712905 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4wg2h\" (UniqueName: \"kubernetes.io/projected/a679630b-4858-40b0-92ac-b4fcf71409cd-kube-api-access-4wg2h\") pod \"authorino-58cd955675-5p6wn\" (UID: \"a679630b-4858-40b0-92ac-b4fcf71409cd\") " pod="kuadrant-system/authorino-58cd955675-5p6wn" Jul 07 14:35:39.813658 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:35:39.813600 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-5p6wn" Jul 07 14:35:39.980071 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:35:39.980037 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-5p6wn"] Jul 07 14:35:39.987992 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:35:39.987963 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda679630b_4858_40b0_92ac_b4fcf71409cd.slice/crio-393280320e0ef8a7489d4aeb5b3929e33165b8fa1b9bc67051593de4ad2abe1e WatchSource:0}: Error finding container 393280320e0ef8a7489d4aeb5b3929e33165b8fa1b9bc67051593de4ad2abe1e: Status 404 returned error can't find the container with id 393280320e0ef8a7489d4aeb5b3929e33165b8fa1b9bc67051593de4ad2abe1e Jul 07 14:35:40.578085 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:35:40.578046 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-5p6wn" event={"ID":"a679630b-4858-40b0-92ac-b4fcf71409cd","Type":"ContainerStarted","Data":"393280320e0ef8a7489d4aeb5b3929e33165b8fa1b9bc67051593de4ad2abe1e"} Jul 07 14:35:42.592712 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:35:42.592673 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-gh87n" event={"ID":"452beab1-a636-4d39-b971-f7de1619fb47","Type":"ContainerStarted","Data":"6115488d760b258198275ead09cc854663e1b27270871c83db65b557b5e88e04"} Jul 07 14:35:42.593267 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:35:42.592957 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-gh87n" Jul 07 14:35:42.612610 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:35:42.612309 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-64c94fd64f-gh87n" podStartSLOduration=1.599993284 podStartE2EDuration="4.612289919s" totalImagesPullingTime="3.012296635s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 14:35:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:35:42.611446569 +0000 UTC m=+579.320584949" watchObservedRunningTime="2026-07-07 14:35:42.612289919 +0000 UTC m=+579.321428301" Jul 07 14:35:45.360359 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:35:45.360321 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-5p6wn"] Jul 07 14:35:45.611463 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:35:45.611386 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-5p6wn" event={"ID":"a679630b-4858-40b0-92ac-b4fcf71409cd","Type":"ContainerStarted","Data":"f0986eef9f6d392eca5468b9645c32993817532a3c22d4d676c6b7376829805a"} Jul 07 14:35:45.626927 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:35:45.626879 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-58cd955675-5p6wn" podStartSLOduration=2.001452267 podStartE2EDuration="6.626864625s" totalImagesPullingTime="4.625412358s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 14:35:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:35:45.625362197 +0000 UTC m=+582.334500570" watchObservedRunningTime="2026-07-07 14:35:45.626864625 +0000 UTC m=+582.336003035" Jul 07 14:35:46.617776 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:35:46.617730 2557 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-58cd955675-5p6wn" podUID="a679630b-4858-40b0-92ac-b4fcf71409cd" containerName="authorino" containerID="cri-o://f0986eef9f6d392eca5468b9645c32993817532a3c22d4d676c6b7376829805a" gracePeriod=30 Jul 07 14:35:46.857720 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:35:46.857683 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:35:46.865374 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:35:46.865353 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-5p6wn" Jul 07 14:35:46.957889 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:35:46.957815 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:35:46.979834 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:35:46.979810 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4wg2h\" (UniqueName: \"kubernetes.io/projected/a679630b-4858-40b0-92ac-b4fcf71409cd-kube-api-access-4wg2h\") pod \"a679630b-4858-40b0-92ac-b4fcf71409cd\" (UID: \"a679630b-4858-40b0-92ac-b4fcf71409cd\") " Jul 07 14:35:46.981627 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:35:46.981609 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a679630b-4858-40b0-92ac-b4fcf71409cd-kube-api-access-4wg2h" (OuterVolumeSpecName: "kube-api-access-4wg2h") pod "a679630b-4858-40b0-92ac-b4fcf71409cd" (UID: "a679630b-4858-40b0-92ac-b4fcf71409cd"). InnerVolumeSpecName "kube-api-access-4wg2h". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 14:35:47.081236 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:35:47.081206 2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4wg2h\" (UniqueName: \"kubernetes.io/projected/a679630b-4858-40b0-92ac-b4fcf71409cd-kube-api-access-4wg2h\") on node \"ip-10-0-134-42.ec2.internal\" DevicePath \"\"" Jul 07 14:35:47.277243 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:35:47.277165 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:35:47.344279 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:35:47.344250 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:35:47.629663 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:35:47.629629 2557 generic.go:356] "Generic (PLEG): container finished" podID="a679630b-4858-40b0-92ac-b4fcf71409cd" containerID="f0986eef9f6d392eca5468b9645c32993817532a3c22d4d676c6b7376829805a" exitCode=0 Jul 07 14:35:47.630135 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:35:47.629686 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-5p6wn" Jul 07 14:35:47.630135 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:35:47.629711 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-5p6wn" event={"ID":"a679630b-4858-40b0-92ac-b4fcf71409cd","Type":"ContainerDied","Data":"f0986eef9f6d392eca5468b9645c32993817532a3c22d4d676c6b7376829805a"} Jul 07 14:35:47.630135 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:35:47.629764 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-5p6wn" event={"ID":"a679630b-4858-40b0-92ac-b4fcf71409cd","Type":"ContainerDied","Data":"393280320e0ef8a7489d4aeb5b3929e33165b8fa1b9bc67051593de4ad2abe1e"} Jul 07 14:35:47.630135 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:35:47.629790 2557 scope.go:117] "RemoveContainer" containerID="f0986eef9f6d392eca5468b9645c32993817532a3c22d4d676c6b7376829805a" Jul 07 14:35:47.639481 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:35:47.639464 2557 scope.go:117] "RemoveContainer" containerID="f0986eef9f6d392eca5468b9645c32993817532a3c22d4d676c6b7376829805a" Jul 07 14:35:47.639756 ip-10-0-134-42 kubenswrapper[2557]: E0707 14:35:47.639735 2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f0986eef9f6d392eca5468b9645c32993817532a3c22d4d676c6b7376829805a\": container with ID starting with f0986eef9f6d392eca5468b9645c32993817532a3c22d4d676c6b7376829805a not found: ID does not exist" containerID="f0986eef9f6d392eca5468b9645c32993817532a3c22d4d676c6b7376829805a" Jul 07 14:35:47.639817 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:35:47.639768 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f0986eef9f6d392eca5468b9645c32993817532a3c22d4d676c6b7376829805a"} err="failed to get container status \"f0986eef9f6d392eca5468b9645c32993817532a3c22d4d676c6b7376829805a\": rpc error: code = NotFound desc = could not find container \"f0986eef9f6d392eca5468b9645c32993817532a3c22d4d676c6b7376829805a\": container with ID starting with f0986eef9f6d392eca5468b9645c32993817532a3c22d4d676c6b7376829805a not found: ID does not exist" Jul 07 14:35:47.653280 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:35:47.653259 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-5p6wn"] Jul 07 14:35:47.657220 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:35:47.657202 2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-58cd955675-5p6wn"] Jul 07 14:35:47.835683 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:35:47.835652 2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a679630b-4858-40b0-92ac-b4fcf71409cd" path="/var/lib/kubelet/pods/a679630b-4858-40b0-92ac-b4fcf71409cd/volumes" Jul 07 14:35:48.654756 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:35:48.654721 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:35:48.715495 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:35:48.715462 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:35:53.600553 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:35:53.600506 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-gh87n" Jul 07 14:35:54.369857 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:35:54.369829 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-gh87n"] Jul 07 14:35:54.370021 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:35:54.369996 2557 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-64c94fd64f-gh87n" podUID="452beab1-a636-4d39-b971-f7de1619fb47" containerName="limitador" containerID="cri-o://6115488d760b258198275ead09cc854663e1b27270871c83db65b557b5e88e04" gracePeriod=30 Jul 07 14:35:54.916918 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:35:54.916896 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-gh87n" Jul 07 14:35:55.048082 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:35:55.048023 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tkg6h\" (UniqueName: \"kubernetes.io/projected/452beab1-a636-4d39-b971-f7de1619fb47-kube-api-access-tkg6h\") pod \"452beab1-a636-4d39-b971-f7de1619fb47\" (UID: \"452beab1-a636-4d39-b971-f7de1619fb47\") " Jul 07 14:35:55.048082 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:35:55.048060 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/452beab1-a636-4d39-b971-f7de1619fb47-config-file\") pod \"452beab1-a636-4d39-b971-f7de1619fb47\" (UID: \"452beab1-a636-4d39-b971-f7de1619fb47\") " Jul 07 14:35:55.048446 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:35:55.048423 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/452beab1-a636-4d39-b971-f7de1619fb47-config-file" (OuterVolumeSpecName: "config-file") pod "452beab1-a636-4d39-b971-f7de1619fb47" (UID: "452beab1-a636-4d39-b971-f7de1619fb47"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 14:35:55.050009 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:35:55.049990 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/452beab1-a636-4d39-b971-f7de1619fb47-kube-api-access-tkg6h" (OuterVolumeSpecName: "kube-api-access-tkg6h") pod "452beab1-a636-4d39-b971-f7de1619fb47" (UID: "452beab1-a636-4d39-b971-f7de1619fb47"). InnerVolumeSpecName "kube-api-access-tkg6h". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 14:35:55.149696 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:35:55.149672 2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tkg6h\" (UniqueName: \"kubernetes.io/projected/452beab1-a636-4d39-b971-f7de1619fb47-kube-api-access-tkg6h\") on node \"ip-10-0-134-42.ec2.internal\" DevicePath \"\"" Jul 07 14:35:55.149696 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:35:55.149696 2557 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/452beab1-a636-4d39-b971-f7de1619fb47-config-file\") on node \"ip-10-0-134-42.ec2.internal\" DevicePath \"\"" Jul 07 14:35:55.674747 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:35:55.674712 2557 generic.go:356] "Generic (PLEG): container finished" podID="452beab1-a636-4d39-b971-f7de1619fb47" containerID="6115488d760b258198275ead09cc854663e1b27270871c83db65b557b5e88e04" exitCode=0 Jul 07 14:35:55.674941 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:35:55.674758 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-gh87n" Jul 07 14:35:55.674941 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:35:55.674795 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-gh87n" event={"ID":"452beab1-a636-4d39-b971-f7de1619fb47","Type":"ContainerDied","Data":"6115488d760b258198275ead09cc854663e1b27270871c83db65b557b5e88e04"} Jul 07 14:35:55.674941 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:35:55.674849 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-gh87n" event={"ID":"452beab1-a636-4d39-b971-f7de1619fb47","Type":"ContainerDied","Data":"7c07939c4d655d6511fc9bb134644c1db59af60550fac3eeae1e471a21ee9dba"} Jul 07 14:35:55.674941 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:35:55.674871 2557 scope.go:117] "RemoveContainer" containerID="6115488d760b258198275ead09cc854663e1b27270871c83db65b557b5e88e04" Jul 07 14:35:55.683612 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:35:55.683599 2557 scope.go:117] "RemoveContainer" containerID="6115488d760b258198275ead09cc854663e1b27270871c83db65b557b5e88e04" Jul 07 14:35:55.683851 ip-10-0-134-42 kubenswrapper[2557]: E0707 14:35:55.683835 2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6115488d760b258198275ead09cc854663e1b27270871c83db65b557b5e88e04\": container with ID starting with 6115488d760b258198275ead09cc854663e1b27270871c83db65b557b5e88e04 not found: ID does not exist" containerID="6115488d760b258198275ead09cc854663e1b27270871c83db65b557b5e88e04" Jul 07 14:35:55.683913 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:35:55.683859 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6115488d760b258198275ead09cc854663e1b27270871c83db65b557b5e88e04"} err="failed to get container status \"6115488d760b258198275ead09cc854663e1b27270871c83db65b557b5e88e04\": rpc error: code = NotFound desc = could not find container \"6115488d760b258198275ead09cc854663e1b27270871c83db65b557b5e88e04\": container with ID starting with 6115488d760b258198275ead09cc854663e1b27270871c83db65b557b5e88e04 not found: ID does not exist" Jul 07 14:35:55.694969 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:35:55.694948 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-gh87n"] Jul 07 14:35:55.698882 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:35:55.698858 2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-gh87n"] Jul 07 14:35:55.835930 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:35:55.835896 2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="452beab1-a636-4d39-b971-f7de1619fb47" path="/var/lib/kubelet/pods/452beab1-a636-4d39-b971-f7de1619fb47/volumes" Jul 07 14:35:56.855642 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:35:56.855601 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:36:00.507291 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:36:00.507261 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/postgres-69fb4594cf-dzxfg"] Jul 07 14:36:00.507687 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:36:00.507642 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a679630b-4858-40b0-92ac-b4fcf71409cd" containerName="authorino" Jul 07 14:36:00.507687 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:36:00.507655 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="a679630b-4858-40b0-92ac-b4fcf71409cd" containerName="authorino" Jul 07 14:36:00.507687 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:36:00.507679 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="452beab1-a636-4d39-b971-f7de1619fb47" containerName="limitador" Jul 07 14:36:00.507687 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:36:00.507687 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="452beab1-a636-4d39-b971-f7de1619fb47" containerName="limitador" Jul 07 14:36:00.507806 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:36:00.507739 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="452beab1-a636-4d39-b971-f7de1619fb47" containerName="limitador" Jul 07 14:36:00.507806 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:36:00.507750 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="a679630b-4858-40b0-92ac-b4fcf71409cd" containerName="authorino" Jul 07 14:36:00.510860 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:36:00.510842 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/postgres-69fb4594cf-dzxfg" Jul 07 14:36:00.512838 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:36:00.512816 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"postgres-creds\"" Jul 07 14:36:00.512950 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:36:00.512913 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-hbxdv\"" Jul 07 14:36:00.521073 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:36:00.521050 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/postgres-69fb4594cf-dzxfg"] Jul 07 14:36:00.538644 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:36:00.538619 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:36:00.594784 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:36:00.594761 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2vqdm\" (UniqueName: \"kubernetes.io/projected/b1bfa496-aad2-43a3-9e1e-7068752146e5-kube-api-access-2vqdm\") pod \"postgres-69fb4594cf-dzxfg\" (UID: \"b1bfa496-aad2-43a3-9e1e-7068752146e5\") " pod="opendatahub/postgres-69fb4594cf-dzxfg" Jul 07 14:36:00.594891 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:36:00.594874 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/b1bfa496-aad2-43a3-9e1e-7068752146e5-data\") pod \"postgres-69fb4594cf-dzxfg\" (UID: \"b1bfa496-aad2-43a3-9e1e-7068752146e5\") " pod="opendatahub/postgres-69fb4594cf-dzxfg" Jul 07 14:36:00.696188 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:36:00.696155 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/b1bfa496-aad2-43a3-9e1e-7068752146e5-data\") pod \"postgres-69fb4594cf-dzxfg\" (UID: \"b1bfa496-aad2-43a3-9e1e-7068752146e5\") " pod="opendatahub/postgres-69fb4594cf-dzxfg" Jul 07 14:36:00.696342 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:36:00.696203 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2vqdm\" (UniqueName: \"kubernetes.io/projected/b1bfa496-aad2-43a3-9e1e-7068752146e5-kube-api-access-2vqdm\") pod \"postgres-69fb4594cf-dzxfg\" (UID: \"b1bfa496-aad2-43a3-9e1e-7068752146e5\") " pod="opendatahub/postgres-69fb4594cf-dzxfg" Jul 07 14:36:00.696610 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:36:00.696587 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/b1bfa496-aad2-43a3-9e1e-7068752146e5-data\") pod \"postgres-69fb4594cf-dzxfg\" (UID: \"b1bfa496-aad2-43a3-9e1e-7068752146e5\") " pod="opendatahub/postgres-69fb4594cf-dzxfg" Jul 07 14:36:00.703639 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:36:00.703616 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2vqdm\" (UniqueName: \"kubernetes.io/projected/b1bfa496-aad2-43a3-9e1e-7068752146e5-kube-api-access-2vqdm\") pod \"postgres-69fb4594cf-dzxfg\" (UID: \"b1bfa496-aad2-43a3-9e1e-7068752146e5\") " pod="opendatahub/postgres-69fb4594cf-dzxfg" Jul 07 14:36:00.823348 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:36:00.823325 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/postgres-69fb4594cf-dzxfg" Jul 07 14:36:00.951794 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:36:00.951769 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/postgres-69fb4594cf-dzxfg"] Jul 07 14:36:00.953500 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:36:00.953472 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb1bfa496_aad2_43a3_9e1e_7068752146e5.slice/crio-2b18fd7fece3bd1e8cb3e8af538a4424e67979887f46720a69b93ee2a8a6b4ff WatchSource:0}: Error finding container 2b18fd7fece3bd1e8cb3e8af538a4424e67979887f46720a69b93ee2a8a6b4ff: Status 404 returned error can't find the container with id 2b18fd7fece3bd1e8cb3e8af538a4424e67979887f46720a69b93ee2a8a6b4ff Jul 07 14:36:01.717003 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:36:01.716959 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/postgres-69fb4594cf-dzxfg" event={"ID":"b1bfa496-aad2-43a3-9e1e-7068752146e5","Type":"ContainerStarted","Data":"2b18fd7fece3bd1e8cb3e8af538a4424e67979887f46720a69b93ee2a8a6b4ff"} Jul 07 14:36:05.180848 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:36:05.180700 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:36:05.281388 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:36:05.281351 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:36:05.467692 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:36:05.467671 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-vd9g9_0fa80df4-c933-4758-b75f-3f70b7217c0d/console-operator/2.log" Jul 07 14:36:05.468620 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:36:05.468603 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-vd9g9_0fa80df4-c933-4758-b75f-3f70b7217c0d/console-operator/2.log" Jul 07 14:36:05.472167 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:36:05.472152 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-scfg4_90d2dce1-be42-4595-b9c4-74a024eef0e9/ovn-acl-logging/0.log" Jul 07 14:36:05.472792 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:36:05.472774 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-scfg4_90d2dce1-be42-4595-b9c4-74a024eef0e9/ovn-acl-logging/0.log" Jul 07 14:36:05.493220 ip-10-0-134-42 kubenswrapper[2557]: E0707 14:36:05.493196 2557 log.go:32] "ImageFsInfo from image service failed" err="rpc error: code = Unknown desc = get image fs info unable to get usage for /var/lib/containers/storage/overlay-images: get disk usage for path /var/lib/containers/storage/overlay-images: lstat /var/lib/containers/storage/overlay-images/.tmp-images.json3418685942: no such file or directory" Jul 07 14:36:05.493299 ip-10-0-134-42 kubenswrapper[2557]: E0707 14:36:05.493235 2557 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get imageFs stats: missing image stats: " Jul 07 14:36:05.513031 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:36:05.513016 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"postgres-creds\"" Jul 07 14:36:05.746211 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:36:05.746122 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/postgres-69fb4594cf-dzxfg" event={"ID":"b1bfa496-aad2-43a3-9e1e-7068752146e5","Type":"ContainerStarted","Data":"fc5484d914b88fe86b892ebc8f918bc5ecf8e73d168e2bb55f46122b84a1a0b6"} Jul 07 14:36:05.746374 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:36:05.746317 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/postgres-69fb4594cf-dzxfg" Jul 07 14:36:05.762380 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:36:05.762325 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/postgres-69fb4594cf-dzxfg" podStartSLOduration=1.206299722 podStartE2EDuration="5.762303778s" totalImagesPullingTime="4.556004056s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 14:36:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:36:05.76130112 +0000 UTC m=+602.470439502" watchObservedRunningTime="2026-07-07 14:36:05.762303778 +0000 UTC m=+602.471442161" Jul 07 14:36:11.780067 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:36:11.780042 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/postgres-69fb4594cf-dzxfg" Jul 07 14:36:12.121077 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:36:12.121045 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:36:12.385729 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:36:12.385655 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5d4b67c446-hq87n"] Jul 07 14:36:12.388096 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:36:12.388079 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-hq87n" Jul 07 14:36:12.389815 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:36:12.389785 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-d8cm8\"" Jul 07 14:36:12.397989 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:36:12.397969 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-hq87n"] Jul 07 14:36:12.450618 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:36:12.450590 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:36:12.501127 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:36:12.501107 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sqxz7\" (UniqueName: \"kubernetes.io/projected/4743abb1-ed65-438d-a0a4-0035e618919f-kube-api-access-sqxz7\") pod \"authorino-5d4b67c446-hq87n\" (UID: \"4743abb1-ed65-438d-a0a4-0035e618919f\") " pod="kuadrant-system/authorino-5d4b67c446-hq87n" Jul 07 14:36:12.554293 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:36:12.554268 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-hq87n"] Jul 07 14:36:12.554469 ip-10-0-134-42 kubenswrapper[2557]: E0707 14:36:12.554452 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-sqxz7], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-5d4b67c446-hq87n" podUID="4743abb1-ed65-438d-a0a4-0035e618919f" Jul 07 14:36:12.585229 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:36:12.585207 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-76877994-xn8f4"] Jul 07 14:36:12.587544 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:36:12.587508 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-76877994-xn8f4" Jul 07 14:36:12.589351 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:36:12.589333 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\"" Jul 07 14:36:12.599220 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:36:12.599193 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-76877994-xn8f4"] Jul 07 14:36:12.602969 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:36:12.602943 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sqxz7\" (UniqueName: \"kubernetes.io/projected/4743abb1-ed65-438d-a0a4-0035e618919f-kube-api-access-sqxz7\") pod \"authorino-5d4b67c446-hq87n\" (UID: \"4743abb1-ed65-438d-a0a4-0035e618919f\") " pod="kuadrant-system/authorino-5d4b67c446-hq87n" Jul 07 14:36:12.619536 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:36:12.616016 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sqxz7\" (UniqueName: \"kubernetes.io/projected/4743abb1-ed65-438d-a0a4-0035e618919f-kube-api-access-sqxz7\") pod \"authorino-5d4b67c446-hq87n\" (UID: \"4743abb1-ed65-438d-a0a4-0035e618919f\") " pod="kuadrant-system/authorino-5d4b67c446-hq87n" Jul 07 14:36:12.677965 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:36:12.677910 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-76877994-xn8f4"] Jul 07 14:36:12.678168 ip-10-0-134-42 kubenswrapper[2557]: E0707 14:36:12.678149 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-gfpmz tls-cert], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-76877994-xn8f4" podUID="65a84609-4c4e-4c03-a84f-f666f5de12c7" Jul 07 14:36:12.704079 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:36:12.704055 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gfpmz\" (UniqueName: \"kubernetes.io/projected/65a84609-4c4e-4c03-a84f-f666f5de12c7-kube-api-access-gfpmz\") pod \"authorino-76877994-xn8f4\" (UID: \"65a84609-4c4e-4c03-a84f-f666f5de12c7\") " pod="kuadrant-system/authorino-76877994-xn8f4" Jul 07 14:36:12.704175 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:36:12.704085 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/65a84609-4c4e-4c03-a84f-f666f5de12c7-tls-cert\") pod \"authorino-76877994-xn8f4\" (UID: \"65a84609-4c4e-4c03-a84f-f666f5de12c7\") " pod="kuadrant-system/authorino-76877994-xn8f4" Jul 07 14:36:12.781602 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:36:12.781581 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-76877994-xn8f4" Jul 07 14:36:12.781890 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:36:12.781586 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-hq87n" Jul 07 14:36:12.787685 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:36:12.787669 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-76877994-xn8f4" Jul 07 14:36:12.790701 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:36:12.790685 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-hq87n" Jul 07 14:36:12.805045 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:36:12.805025 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gfpmz\" (UniqueName: \"kubernetes.io/projected/65a84609-4c4e-4c03-a84f-f666f5de12c7-kube-api-access-gfpmz\") pod \"authorino-76877994-xn8f4\" (UID: \"65a84609-4c4e-4c03-a84f-f666f5de12c7\") " pod="kuadrant-system/authorino-76877994-xn8f4" Jul 07 14:36:12.805149 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:36:12.805061 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/65a84609-4c4e-4c03-a84f-f666f5de12c7-tls-cert\") pod \"authorino-76877994-xn8f4\" (UID: \"65a84609-4c4e-4c03-a84f-f666f5de12c7\") " pod="kuadrant-system/authorino-76877994-xn8f4" Jul 07 14:36:12.807295 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:36:12.807277 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/65a84609-4c4e-4c03-a84f-f666f5de12c7-tls-cert\") pod \"authorino-76877994-xn8f4\" (UID: \"65a84609-4c4e-4c03-a84f-f666f5de12c7\") " pod="kuadrant-system/authorino-76877994-xn8f4" Jul 07 14:36:12.813760 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:36:12.813743 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gfpmz\" (UniqueName: \"kubernetes.io/projected/65a84609-4c4e-4c03-a84f-f666f5de12c7-kube-api-access-gfpmz\") pod \"authorino-76877994-xn8f4\" (UID: \"65a84609-4c4e-4c03-a84f-f666f5de12c7\") " pod="kuadrant-system/authorino-76877994-xn8f4" Jul 07 14:36:12.905344 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:36:12.905325 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sqxz7\" (UniqueName: \"kubernetes.io/projected/4743abb1-ed65-438d-a0a4-0035e618919f-kube-api-access-sqxz7\") pod \"4743abb1-ed65-438d-a0a4-0035e618919f\" (UID: \"4743abb1-ed65-438d-a0a4-0035e618919f\") " Jul 07 14:36:12.905461 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:36:12.905436 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gfpmz\" (UniqueName: \"kubernetes.io/projected/65a84609-4c4e-4c03-a84f-f666f5de12c7-kube-api-access-gfpmz\") pod \"65a84609-4c4e-4c03-a84f-f666f5de12c7\" (UID: \"65a84609-4c4e-4c03-a84f-f666f5de12c7\") " Jul 07 14:36:12.905551 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:36:12.905466 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/65a84609-4c4e-4c03-a84f-f666f5de12c7-tls-cert\") pod \"65a84609-4c4e-4c03-a84f-f666f5de12c7\" (UID: \"65a84609-4c4e-4c03-a84f-f666f5de12c7\") " Jul 07 14:36:12.907374 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:36:12.907345 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4743abb1-ed65-438d-a0a4-0035e618919f-kube-api-access-sqxz7" (OuterVolumeSpecName: "kube-api-access-sqxz7") pod "4743abb1-ed65-438d-a0a4-0035e618919f" (UID: "4743abb1-ed65-438d-a0a4-0035e618919f"). InnerVolumeSpecName "kube-api-access-sqxz7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 14:36:12.907459 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:36:12.907427 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/65a84609-4c4e-4c03-a84f-f666f5de12c7-kube-api-access-gfpmz" (OuterVolumeSpecName: "kube-api-access-gfpmz") pod "65a84609-4c4e-4c03-a84f-f666f5de12c7" (UID: "65a84609-4c4e-4c03-a84f-f666f5de12c7"). InnerVolumeSpecName "kube-api-access-gfpmz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 14:36:12.907497 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:36:12.907456 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/65a84609-4c4e-4c03-a84f-f666f5de12c7-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "65a84609-4c4e-4c03-a84f-f666f5de12c7" (UID: "65a84609-4c4e-4c03-a84f-f666f5de12c7"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 14:36:13.006899 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:36:13.006849 2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sqxz7\" (UniqueName: \"kubernetes.io/projected/4743abb1-ed65-438d-a0a4-0035e618919f-kube-api-access-sqxz7\") on node \"ip-10-0-134-42.ec2.internal\" DevicePath \"\"" Jul 07 14:36:13.006899 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:36:13.006868 2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gfpmz\" (UniqueName: \"kubernetes.io/projected/65a84609-4c4e-4c03-a84f-f666f5de12c7-kube-api-access-gfpmz\") on node \"ip-10-0-134-42.ec2.internal\" DevicePath \"\"" Jul 07 14:36:13.006899 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:36:13.006877 2557 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/65a84609-4c4e-4c03-a84f-f666f5de12c7-tls-cert\") on node \"ip-10-0-134-42.ec2.internal\" DevicePath \"\"" Jul 07 14:36:13.786663 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:36:13.786629 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-76877994-xn8f4" Jul 07 14:36:13.787136 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:36:13.786629 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-hq87n" Jul 07 14:36:13.820908 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:36:13.820879 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-hq87n"] Jul 07 14:36:13.825177 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:36:13.825152 2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-hq87n"] Jul 07 14:36:13.837511 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:36:13.837485 2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4743abb1-ed65-438d-a0a4-0035e618919f" path="/var/lib/kubelet/pods/4743abb1-ed65-438d-a0a4-0035e618919f/volumes" Jul 07 14:36:13.845559 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:36:13.845540 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-76877994-xn8f4"] Jul 07 14:36:13.848699 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:36:13.848682 2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-76877994-xn8f4"] Jul 07 14:36:15.835583 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:36:15.835555 2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="65a84609-4c4e-4c03-a84f-f666f5de12c7" path="/var/lib/kubelet/pods/65a84609-4c4e-4c03-a84f-f666f5de12c7/volumes" Jul 07 14:36:19.154084 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:36:19.154046 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:36:19.174843 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:36:19.174817 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-965bcc7bc-p4fm9"] Jul 07 14:36:19.182848 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:36:19.182828 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-965bcc7bc-p4fm9" Jul 07 14:36:19.185204 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:36:19.185177 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jul 07 14:36:19.185316 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:36:19.185204 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jul 07 14:36:19.185316 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:36:19.185208 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-nzgrn\"" Jul 07 14:36:19.188922 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:36:19.188900 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-965bcc7bc-p4fm9"] Jul 07 14:36:19.237781 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:36:19.237749 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:36:19.256756 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:36:19.256729 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ecdfd85c-4a53-43f2-b81a-8d052735f624-webhook-cert\") pod \"maas-controller-965bcc7bc-p4fm9\" (UID: \"ecdfd85c-4a53-43f2-b81a-8d052735f624\") " pod="opendatahub/maas-controller-965bcc7bc-p4fm9" Jul 07 14:36:19.256854 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:36:19.256823 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n44hj\" (UniqueName: \"kubernetes.io/projected/ecdfd85c-4a53-43f2-b81a-8d052735f624-kube-api-access-n44hj\") pod \"maas-controller-965bcc7bc-p4fm9\" (UID: \"ecdfd85c-4a53-43f2-b81a-8d052735f624\") " pod="opendatahub/maas-controller-965bcc7bc-p4fm9" Jul 07 14:36:19.292051 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:36:19.292026 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:36:19.354157 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:36:19.354133 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-d4dcbc546-nwm2q"] Jul 07 14:36:19.356993 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:36:19.356977 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-d4dcbc546-nwm2q" Jul 07 14:36:19.357170 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:36:19.357153 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n44hj\" (UniqueName: \"kubernetes.io/projected/ecdfd85c-4a53-43f2-b81a-8d052735f624-kube-api-access-n44hj\") pod \"maas-controller-965bcc7bc-p4fm9\" (UID: \"ecdfd85c-4a53-43f2-b81a-8d052735f624\") " pod="opendatahub/maas-controller-965bcc7bc-p4fm9" Jul 07 14:36:19.357278 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:36:19.357260 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ecdfd85c-4a53-43f2-b81a-8d052735f624-webhook-cert\") pod \"maas-controller-965bcc7bc-p4fm9\" (UID: \"ecdfd85c-4a53-43f2-b81a-8d052735f624\") " pod="opendatahub/maas-controller-965bcc7bc-p4fm9" Jul 07 14:36:19.359568 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:36:19.359547 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ecdfd85c-4a53-43f2-b81a-8d052735f624-webhook-cert\") pod \"maas-controller-965bcc7bc-p4fm9\" (UID: \"ecdfd85c-4a53-43f2-b81a-8d052735f624\") " pod="opendatahub/maas-controller-965bcc7bc-p4fm9" Jul 07 14:36:19.368980 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:36:19.368962 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-d4dcbc546-nwm2q"] Jul 07 14:36:19.371022 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:36:19.371000 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n44hj\" (UniqueName: \"kubernetes.io/projected/ecdfd85c-4a53-43f2-b81a-8d052735f624-kube-api-access-n44hj\") pod \"maas-controller-965bcc7bc-p4fm9\" (UID: \"ecdfd85c-4a53-43f2-b81a-8d052735f624\") " pod="opendatahub/maas-controller-965bcc7bc-p4fm9" Jul 07 14:36:19.406173 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:36:19.406116 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:36:19.458364 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:36:19.458339 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d3af5d7a-0e15-4e9a-8409-ac49fb63b4c9-webhook-cert\") pod \"maas-controller-d4dcbc546-nwm2q\" (UID: \"d3af5d7a-0e15-4e9a-8409-ac49fb63b4c9\") " pod="opendatahub/maas-controller-d4dcbc546-nwm2q" Jul 07 14:36:19.458454 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:36:19.458409 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pbjmd\" (UniqueName: \"kubernetes.io/projected/d3af5d7a-0e15-4e9a-8409-ac49fb63b4c9-kube-api-access-pbjmd\") pod \"maas-controller-d4dcbc546-nwm2q\" (UID: \"d3af5d7a-0e15-4e9a-8409-ac49fb63b4c9\") " pod="opendatahub/maas-controller-d4dcbc546-nwm2q" Jul 07 14:36:19.493332 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:36:19.493312 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-965bcc7bc-p4fm9" Jul 07 14:36:19.559731 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:36:19.559701 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pbjmd\" (UniqueName: \"kubernetes.io/projected/d3af5d7a-0e15-4e9a-8409-ac49fb63b4c9-kube-api-access-pbjmd\") pod \"maas-controller-d4dcbc546-nwm2q\" (UID: \"d3af5d7a-0e15-4e9a-8409-ac49fb63b4c9\") " pod="opendatahub/maas-controller-d4dcbc546-nwm2q" Jul 07 14:36:19.559881 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:36:19.559775 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d3af5d7a-0e15-4e9a-8409-ac49fb63b4c9-webhook-cert\") pod \"maas-controller-d4dcbc546-nwm2q\" (UID: \"d3af5d7a-0e15-4e9a-8409-ac49fb63b4c9\") " pod="opendatahub/maas-controller-d4dcbc546-nwm2q" Jul 07 14:36:19.562091 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:36:19.562070 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d3af5d7a-0e15-4e9a-8409-ac49fb63b4c9-webhook-cert\") pod \"maas-controller-d4dcbc546-nwm2q\" (UID: \"d3af5d7a-0e15-4e9a-8409-ac49fb63b4c9\") " pod="opendatahub/maas-controller-d4dcbc546-nwm2q" Jul 07 14:36:19.567686 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:36:19.567664 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pbjmd\" (UniqueName: \"kubernetes.io/projected/d3af5d7a-0e15-4e9a-8409-ac49fb63b4c9-kube-api-access-pbjmd\") pod \"maas-controller-d4dcbc546-nwm2q\" (UID: \"d3af5d7a-0e15-4e9a-8409-ac49fb63b4c9\") " pod="opendatahub/maas-controller-d4dcbc546-nwm2q" Jul 07 14:36:19.623650 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:36:19.623626 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-965bcc7bc-p4fm9"] Jul 07 14:36:19.625295 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:36:19.625241 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podecdfd85c_4a53_43f2_b81a_8d052735f624.slice/crio-8e19d3907aca5a3a317bddac499d4d5d5f0bf90b1d74b5e421f4337305fe2127 WatchSource:0}: Error finding container 8e19d3907aca5a3a317bddac499d4d5d5f0bf90b1d74b5e421f4337305fe2127: Status 404 returned error can't find the container with id 8e19d3907aca5a3a317bddac499d4d5d5f0bf90b1d74b5e421f4337305fe2127 Jul 07 14:36:19.682290 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:36:19.682212 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-d4dcbc546-nwm2q" Jul 07 14:36:19.812770 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:36:19.812745 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-d4dcbc546-nwm2q"] Jul 07 14:36:19.814511 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:36:19.814490 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd3af5d7a_0e15_4e9a_8409_ac49fb63b4c9.slice/crio-c6693a11954126637070beb7761d0a1de81e143ee92d22fdafd83ca90031117b WatchSource:0}: Error finding container c6693a11954126637070beb7761d0a1de81e143ee92d22fdafd83ca90031117b: Status 404 returned error can't find the container with id c6693a11954126637070beb7761d0a1de81e143ee92d22fdafd83ca90031117b Jul 07 14:36:19.824375 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:36:19.824350 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-965bcc7bc-p4fm9" event={"ID":"ecdfd85c-4a53-43f2-b81a-8d052735f624","Type":"ContainerStarted","Data":"8e19d3907aca5a3a317bddac499d4d5d5f0bf90b1d74b5e421f4337305fe2127"} Jul 07 14:36:19.825329 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:36:19.825308 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-d4dcbc546-nwm2q" event={"ID":"d3af5d7a-0e15-4e9a-8409-ac49fb63b4c9","Type":"ContainerStarted","Data":"c6693a11954126637070beb7761d0a1de81e143ee92d22fdafd83ca90031117b"} Jul 07 14:36:21.844146 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:36:21.844115 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-d4dcbc546-nwm2q" event={"ID":"d3af5d7a-0e15-4e9a-8409-ac49fb63b4c9","Type":"ContainerStarted","Data":"8b539dddc64ad3859a5271d34f89edf6c7a02b693aad620e46d9e415e2d7c86d"} Jul 07 14:36:21.845836 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:36:21.845813 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-965bcc7bc-p4fm9" event={"ID":"ecdfd85c-4a53-43f2-b81a-8d052735f624","Type":"ContainerStarted","Data":"d69d986c0e03372c41a336bc5fa2bf4480a82740617093d4561ae7f6aa431ca6"} Jul 07 14:36:21.966348 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:36:21.966314 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:36:22.069948 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:36:22.069839 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:36:22.858018 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:36:22.857983 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-965bcc7bc-p4fm9" Jul 07 14:36:22.880942 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:36:22.880902 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-d4dcbc546-nwm2q" podStartSLOduration=1.920332883 podStartE2EDuration="3.880889878s" totalImagesPullingTime="1.960556995s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 14:36:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:36:22.877275756 +0000 UTC m=+619.586414137" watchObservedRunningTime="2026-07-07 14:36:22.880889878 +0000 UTC m=+619.590028301" Jul 07 14:36:22.898744 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:36:22.898704 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-965bcc7bc-p4fm9" podStartSLOduration=1.758828201 podStartE2EDuration="3.898691447s" totalImagesPullingTime="2.139863246s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 14:36:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:36:22.89722833 +0000 UTC m=+619.606366736" watchObservedRunningTime="2026-07-07 14:36:22.898691447 +0000 UTC m=+619.607829843" Jul 07 14:36:32.858758 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:36:32.858678 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-d4dcbc546-nwm2q" Jul 07 14:36:32.863890 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:36:32.863861 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-d4dcbc546-nwm2q" Jul 07 14:36:32.909379 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:36:32.909350 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-965bcc7bc-p4fm9"] Jul 07 14:36:32.909673 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:36:32.909615 2557 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-controller-965bcc7bc-p4fm9" podUID="ecdfd85c-4a53-43f2-b81a-8d052735f624" containerName="manager" containerID="cri-o://d69d986c0e03372c41a336bc5fa2bf4480a82740617093d4561ae7f6aa431ca6" gracePeriod=30 Jul 07 14:36:32.916623 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:36:32.916601 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-965bcc7bc-p4fm9" Jul 07 14:36:33.149092 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:36:33.149067 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-965bcc7bc-p4fm9" Jul 07 14:36:33.281640 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:36:33.281614 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ecdfd85c-4a53-43f2-b81a-8d052735f624-webhook-cert\") pod \"ecdfd85c-4a53-43f2-b81a-8d052735f624\" (UID: \"ecdfd85c-4a53-43f2-b81a-8d052735f624\") " Jul 07 14:36:33.281764 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:36:33.281653 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n44hj\" (UniqueName: \"kubernetes.io/projected/ecdfd85c-4a53-43f2-b81a-8d052735f624-kube-api-access-n44hj\") pod \"ecdfd85c-4a53-43f2-b81a-8d052735f624\" (UID: \"ecdfd85c-4a53-43f2-b81a-8d052735f624\") " Jul 07 14:36:33.283697 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:36:33.283667 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ecdfd85c-4a53-43f2-b81a-8d052735f624-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "ecdfd85c-4a53-43f2-b81a-8d052735f624" (UID: "ecdfd85c-4a53-43f2-b81a-8d052735f624"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 14:36:33.283788 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:36:33.283714 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ecdfd85c-4a53-43f2-b81a-8d052735f624-kube-api-access-n44hj" (OuterVolumeSpecName: "kube-api-access-n44hj") pod "ecdfd85c-4a53-43f2-b81a-8d052735f624" (UID: "ecdfd85c-4a53-43f2-b81a-8d052735f624"). InnerVolumeSpecName "kube-api-access-n44hj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 14:36:33.383449 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:36:33.383344 2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n44hj\" (UniqueName: \"kubernetes.io/projected/ecdfd85c-4a53-43f2-b81a-8d052735f624-kube-api-access-n44hj\") on node \"ip-10-0-134-42.ec2.internal\" DevicePath \"\"" Jul 07 14:36:33.383449 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:36:33.383395 2557 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ecdfd85c-4a53-43f2-b81a-8d052735f624-webhook-cert\") on node \"ip-10-0-134-42.ec2.internal\" DevicePath \"\"" Jul 07 14:36:33.913566 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:36:33.913513 2557 generic.go:356] "Generic (PLEG): container finished" podID="ecdfd85c-4a53-43f2-b81a-8d052735f624" containerID="d69d986c0e03372c41a336bc5fa2bf4480a82740617093d4561ae7f6aa431ca6" exitCode=0 Jul 07 14:36:33.913566 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:36:33.913550 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-965bcc7bc-p4fm9" event={"ID":"ecdfd85c-4a53-43f2-b81a-8d052735f624","Type":"ContainerDied","Data":"d69d986c0e03372c41a336bc5fa2bf4480a82740617093d4561ae7f6aa431ca6"} Jul 07 14:36:33.913566 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:36:33.913568 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-965bcc7bc-p4fm9" Jul 07 14:36:33.914009 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:36:33.913599 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-965bcc7bc-p4fm9" event={"ID":"ecdfd85c-4a53-43f2-b81a-8d052735f624","Type":"ContainerDied","Data":"8e19d3907aca5a3a317bddac499d4d5d5f0bf90b1d74b5e421f4337305fe2127"} Jul 07 14:36:33.914009 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:36:33.913623 2557 scope.go:117] "RemoveContainer" containerID="d69d986c0e03372c41a336bc5fa2bf4480a82740617093d4561ae7f6aa431ca6" Jul 07 14:36:33.922722 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:36:33.922711 2557 scope.go:117] "RemoveContainer" containerID="d69d986c0e03372c41a336bc5fa2bf4480a82740617093d4561ae7f6aa431ca6" Jul 07 14:36:33.922957 ip-10-0-134-42 kubenswrapper[2557]: E0707 14:36:33.922938 2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d69d986c0e03372c41a336bc5fa2bf4480a82740617093d4561ae7f6aa431ca6\": container with ID starting with d69d986c0e03372c41a336bc5fa2bf4480a82740617093d4561ae7f6aa431ca6 not found: ID does not exist" containerID="d69d986c0e03372c41a336bc5fa2bf4480a82740617093d4561ae7f6aa431ca6" Jul 07 14:36:33.923007 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:36:33.922965 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d69d986c0e03372c41a336bc5fa2bf4480a82740617093d4561ae7f6aa431ca6"} err="failed to get container status \"d69d986c0e03372c41a336bc5fa2bf4480a82740617093d4561ae7f6aa431ca6\": rpc error: code = NotFound desc = could not find container \"d69d986c0e03372c41a336bc5fa2bf4480a82740617093d4561ae7f6aa431ca6\": container with ID starting with d69d986c0e03372c41a336bc5fa2bf4480a82740617093d4561ae7f6aa431ca6 not found: ID does not exist" Jul 07 14:36:33.928975 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:36:33.928956 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-965bcc7bc-p4fm9"] Jul 07 14:36:33.932311 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:36:33.932292 2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-965bcc7bc-p4fm9"] Jul 07 14:36:35.835614 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:36:35.835583 2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ecdfd85c-4a53-43f2-b81a-8d052735f624" path="/var/lib/kubelet/pods/ecdfd85c-4a53-43f2-b81a-8d052735f624/volumes" Jul 07 14:36:52.138590 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:36:52.138557 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:36:52.586226 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:36:52.586186 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-6d878c79-c99rg"] Jul 07 14:36:52.586568 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:36:52.586555 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ecdfd85c-4a53-43f2-b81a-8d052735f624" containerName="manager" Jul 07 14:36:52.586620 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:36:52.586569 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="ecdfd85c-4a53-43f2-b81a-8d052735f624" containerName="manager" Jul 07 14:36:52.586656 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:36:52.586635 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="ecdfd85c-4a53-43f2-b81a-8d052735f624" containerName="manager" Jul 07 14:36:52.610806 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:36:52.610765 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-6d878c79-c99rg"] Jul 07 14:36:52.610957 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:36:52.610864 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-6d878c79-c99rg" Jul 07 14:36:52.612998 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:36:52.612978 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-pf8n9\"" Jul 07 14:36:52.613452 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:36:52.613437 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-serving-cert\"" Jul 07 14:36:52.643093 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:36:52.643062 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2f7b375b-aaed-4a14-bec8-8b72bfbe89fa-maas-api-tls\") pod \"maas-api-6d878c79-c99rg\" (UID: \"2f7b375b-aaed-4a14-bec8-8b72bfbe89fa\") " pod="opendatahub/maas-api-6d878c79-c99rg" Jul 07 14:36:52.643229 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:36:52.643115 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r7x2p\" (UniqueName: \"kubernetes.io/projected/2f7b375b-aaed-4a14-bec8-8b72bfbe89fa-kube-api-access-r7x2p\") pod \"maas-api-6d878c79-c99rg\" (UID: \"2f7b375b-aaed-4a14-bec8-8b72bfbe89fa\") " pod="opendatahub/maas-api-6d878c79-c99rg" Jul 07 14:36:52.643229 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:36:52.643181 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2f7b375b-aaed-4a14-bec8-8b72bfbe89fa-openshift-service-ca\") pod \"maas-api-6d878c79-c99rg\" (UID: \"2f7b375b-aaed-4a14-bec8-8b72bfbe89fa\") " pod="opendatahub/maas-api-6d878c79-c99rg" Jul 07 14:36:52.744563 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:36:52.744507 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2f7b375b-aaed-4a14-bec8-8b72bfbe89fa-openshift-service-ca\") pod \"maas-api-6d878c79-c99rg\" (UID: \"2f7b375b-aaed-4a14-bec8-8b72bfbe89fa\") " pod="opendatahub/maas-api-6d878c79-c99rg" Jul 07 14:36:52.744731 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:36:52.744625 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2f7b375b-aaed-4a14-bec8-8b72bfbe89fa-maas-api-tls\") pod \"maas-api-6d878c79-c99rg\" (UID: \"2f7b375b-aaed-4a14-bec8-8b72bfbe89fa\") " pod="opendatahub/maas-api-6d878c79-c99rg" Jul 07 14:36:52.744731 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:36:52.744682 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r7x2p\" (UniqueName: \"kubernetes.io/projected/2f7b375b-aaed-4a14-bec8-8b72bfbe89fa-kube-api-access-r7x2p\") pod \"maas-api-6d878c79-c99rg\" (UID: \"2f7b375b-aaed-4a14-bec8-8b72bfbe89fa\") " pod="opendatahub/maas-api-6d878c79-c99rg" Jul 07 14:36:52.745260 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:36:52.745242 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2f7b375b-aaed-4a14-bec8-8b72bfbe89fa-openshift-service-ca\") pod \"maas-api-6d878c79-c99rg\" (UID: \"2f7b375b-aaed-4a14-bec8-8b72bfbe89fa\") " pod="opendatahub/maas-api-6d878c79-c99rg" Jul 07 14:36:52.746941 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:36:52.746924 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2f7b375b-aaed-4a14-bec8-8b72bfbe89fa-maas-api-tls\") pod \"maas-api-6d878c79-c99rg\" (UID: \"2f7b375b-aaed-4a14-bec8-8b72bfbe89fa\") " pod="opendatahub/maas-api-6d878c79-c99rg" Jul 07 14:36:52.752287 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:36:52.752270 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r7x2p\" (UniqueName: \"kubernetes.io/projected/2f7b375b-aaed-4a14-bec8-8b72bfbe89fa-kube-api-access-r7x2p\") pod \"maas-api-6d878c79-c99rg\" (UID: \"2f7b375b-aaed-4a14-bec8-8b72bfbe89fa\") " pod="opendatahub/maas-api-6d878c79-c99rg" Jul 07 14:36:52.920981 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:36:52.920894 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-6d878c79-c99rg" Jul 07 14:36:53.064553 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:36:53.064480 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-6d878c79-c99rg"] Jul 07 14:36:53.066019 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:36:53.065995 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2f7b375b_aaed_4a14_bec8_8b72bfbe89fa.slice/crio-21efef95c4ac4e4ad5fc97ca0fb6fa1a8c09fc5efe44d7a0953b6935b0710cfb WatchSource:0}: Error finding container 21efef95c4ac4e4ad5fc97ca0fb6fa1a8c09fc5efe44d7a0953b6935b0710cfb: Status 404 returned error can't find the container with id 21efef95c4ac4e4ad5fc97ca0fb6fa1a8c09fc5efe44d7a0953b6935b0710cfb Jul 07 14:36:53.067694 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:36:53.067674 2557 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 07 14:36:53.191660 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:36:53.191579 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-7964457696-5wrr4"] Jul 07 14:36:53.194690 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:36:53.194670 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7964457696-5wrr4" Jul 07 14:36:53.200467 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:36:53.200444 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\"" Jul 07 14:36:53.200605 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:36:53.200468 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-wczqw\"" Jul 07 14:36:53.207580 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:36:53.207556 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7964457696-5wrr4"] Jul 07 14:36:53.249026 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:36:53.248985 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6mczs\" (UniqueName: \"kubernetes.io/projected/511ba927-b39a-4c92-a3c8-2ec64e3472ba-kube-api-access-6mczs\") pod \"payload-pre-processing-7964457696-5wrr4\" (UID: \"511ba927-b39a-4c92-a3c8-2ec64e3472ba\") " pod="openshift-ingress/payload-pre-processing-7964457696-5wrr4" Jul 07 14:36:53.249189 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:36:53.249135 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/511ba927-b39a-4c92-a3c8-2ec64e3472ba-plugins-config-volume\") pod \"payload-pre-processing-7964457696-5wrr4\" (UID: \"511ba927-b39a-4c92-a3c8-2ec64e3472ba\") " pod="openshift-ingress/payload-pre-processing-7964457696-5wrr4" Jul 07 14:36:53.277290 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:36:53.277255 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-58f86c7798-nwgts"] Jul 07 14:36:53.279737 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:36:53.279719 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-58f86c7798-nwgts" Jul 07 14:36:53.296628 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:36:53.296601 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-58f86c7798-nwgts"] Jul 07 14:36:53.350348 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:36:53.350312 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/511ba927-b39a-4c92-a3c8-2ec64e3472ba-plugins-config-volume\") pod \"payload-pre-processing-7964457696-5wrr4\" (UID: \"511ba927-b39a-4c92-a3c8-2ec64e3472ba\") " pod="openshift-ingress/payload-pre-processing-7964457696-5wrr4" Jul 07 14:36:53.350548 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:36:53.350351 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/37f81559-4444-49b6-9fea-263efdf23c0a-plugins-config-volume\") pod \"payload-processing-58f86c7798-nwgts\" (UID: \"37f81559-4444-49b6-9fea-263efdf23c0a\") " pod="openshift-ingress/payload-processing-58f86c7798-nwgts" Jul 07 14:36:53.350548 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:36:53.350434 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6mczs\" (UniqueName: \"kubernetes.io/projected/511ba927-b39a-4c92-a3c8-2ec64e3472ba-kube-api-access-6mczs\") pod \"payload-pre-processing-7964457696-5wrr4\" (UID: \"511ba927-b39a-4c92-a3c8-2ec64e3472ba\") " pod="openshift-ingress/payload-pre-processing-7964457696-5wrr4" Jul 07 14:36:53.350548 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:36:53.350505 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bbhkf\" (UniqueName: \"kubernetes.io/projected/37f81559-4444-49b6-9fea-263efdf23c0a-kube-api-access-bbhkf\") pod \"payload-processing-58f86c7798-nwgts\" (UID: \"37f81559-4444-49b6-9fea-263efdf23c0a\") " pod="openshift-ingress/payload-processing-58f86c7798-nwgts" Jul 07 14:36:53.351067 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:36:53.351048 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/511ba927-b39a-4c92-a3c8-2ec64e3472ba-plugins-config-volume\") pod \"payload-pre-processing-7964457696-5wrr4\" (UID: \"511ba927-b39a-4c92-a3c8-2ec64e3472ba\") " pod="openshift-ingress/payload-pre-processing-7964457696-5wrr4" Jul 07 14:36:53.358464 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:36:53.358442 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6mczs\" (UniqueName: \"kubernetes.io/projected/511ba927-b39a-4c92-a3c8-2ec64e3472ba-kube-api-access-6mczs\") pod \"payload-pre-processing-7964457696-5wrr4\" (UID: \"511ba927-b39a-4c92-a3c8-2ec64e3472ba\") " pod="openshift-ingress/payload-pre-processing-7964457696-5wrr4" Jul 07 14:36:53.451932 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:36:53.451843 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bbhkf\" (UniqueName: \"kubernetes.io/projected/37f81559-4444-49b6-9fea-263efdf23c0a-kube-api-access-bbhkf\") pod \"payload-processing-58f86c7798-nwgts\" (UID: \"37f81559-4444-49b6-9fea-263efdf23c0a\") " pod="openshift-ingress/payload-processing-58f86c7798-nwgts" Jul 07 14:36:53.451932 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:36:53.451918 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/37f81559-4444-49b6-9fea-263efdf23c0a-plugins-config-volume\") pod \"payload-processing-58f86c7798-nwgts\" (UID: \"37f81559-4444-49b6-9fea-263efdf23c0a\") " pod="openshift-ingress/payload-processing-58f86c7798-nwgts" Jul 07 14:36:53.452725 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:36:53.452702 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/37f81559-4444-49b6-9fea-263efdf23c0a-plugins-config-volume\") pod \"payload-processing-58f86c7798-nwgts\" (UID: \"37f81559-4444-49b6-9fea-263efdf23c0a\") " pod="openshift-ingress/payload-processing-58f86c7798-nwgts" Jul 07 14:36:53.461923 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:36:53.461893 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bbhkf\" (UniqueName: \"kubernetes.io/projected/37f81559-4444-49b6-9fea-263efdf23c0a-kube-api-access-bbhkf\") pod \"payload-processing-58f86c7798-nwgts\" (UID: \"37f81559-4444-49b6-9fea-263efdf23c0a\") " pod="openshift-ingress/payload-processing-58f86c7798-nwgts" Jul 07 14:36:53.505166 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:36:53.505135 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7964457696-5wrr4" Jul 07 14:36:53.594939 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:36:53.594671 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-58f86c7798-nwgts" Jul 07 14:36:53.688751 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:36:53.688722 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7964457696-5wrr4"] Jul 07 14:36:53.689989 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:36:53.689961 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod511ba927_b39a_4c92_a3c8_2ec64e3472ba.slice/crio-7fc3f0e15f1777475dddf23ca6401acf9b34013a7483c5b30f7e1869e8b7b18d WatchSource:0}: Error finding container 7fc3f0e15f1777475dddf23ca6401acf9b34013a7483c5b30f7e1869e8b7b18d: Status 404 returned error can't find the container with id 7fc3f0e15f1777475dddf23ca6401acf9b34013a7483c5b30f7e1869e8b7b18d Jul 07 14:36:53.770487 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:36:53.770461 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-58f86c7798-nwgts"] Jul 07 14:36:53.773263 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:36:53.773229 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod37f81559_4444_49b6_9fea_263efdf23c0a.slice/crio-5e4dcb06aaa35da047247a5fbf988d25d3e18ef9ee26766ec87987b1a07b9e6c WatchSource:0}: Error finding container 5e4dcb06aaa35da047247a5fbf988d25d3e18ef9ee26766ec87987b1a07b9e6c: Status 404 returned error can't find the container with id 5e4dcb06aaa35da047247a5fbf988d25d3e18ef9ee26766ec87987b1a07b9e6c Jul 07 14:36:53.862381 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:36:53.862347 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:36:54.024974 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:36:54.024869 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7964457696-5wrr4" event={"ID":"511ba927-b39a-4c92-a3c8-2ec64e3472ba","Type":"ContainerStarted","Data":"7fc3f0e15f1777475dddf23ca6401acf9b34013a7483c5b30f7e1869e8b7b18d"} Jul 07 14:36:54.026333 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:36:54.026307 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-6d878c79-c99rg" event={"ID":"2f7b375b-aaed-4a14-bec8-8b72bfbe89fa","Type":"ContainerStarted","Data":"21efef95c4ac4e4ad5fc97ca0fb6fa1a8c09fc5efe44d7a0953b6935b0710cfb"} Jul 07 14:36:54.027555 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:36:54.027501 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-58f86c7798-nwgts" event={"ID":"37f81559-4444-49b6-9fea-263efdf23c0a","Type":"ContainerStarted","Data":"5e4dcb06aaa35da047247a5fbf988d25d3e18ef9ee26766ec87987b1a07b9e6c"} Jul 07 14:36:56.041534 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:36:56.041414 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7964457696-5wrr4" event={"ID":"511ba927-b39a-4c92-a3c8-2ec64e3472ba","Type":"ContainerStarted","Data":"90ce6d14b555ea6947c3c35a375a2546074519387ed46ac613a71ffbcf43dc64"} Jul 07 14:36:56.042269 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:36:56.042245 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-7964457696-5wrr4" Jul 07 14:36:56.044277 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:36:56.044248 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-6d878c79-c99rg" event={"ID":"2f7b375b-aaed-4a14-bec8-8b72bfbe89fa","Type":"ContainerStarted","Data":"126049581ca1d776adc273a89af0e224cd57e86194b643ad720e7eb6ed7c830a"} Jul 07 14:36:56.044823 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:36:56.044803 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-6d878c79-c99rg" Jul 07 14:36:56.047167 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:36:56.047145 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-58f86c7798-nwgts" event={"ID":"37f81559-4444-49b6-9fea-263efdf23c0a","Type":"ContainerStarted","Data":"b9ad30fa5ad4eb2ea1aced182212c436eed670d0123d44bf5e90e7bcf3072e74"} Jul 07 14:36:56.047630 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:36:56.047610 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-58f86c7798-nwgts" Jul 07 14:36:56.059065 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:36:56.059020 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-7964457696-5wrr4" podStartSLOduration=1.01853601 podStartE2EDuration="3.059002934s" totalImagesPullingTime="2.040466924s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 14:36:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:36:56.057180875 +0000 UTC m=+652.766319282" watchObservedRunningTime="2026-07-07 14:36:56.059002934 +0000 UTC m=+652.768141314" Jul 07 14:36:56.074375 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:36:56.074300 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-6d878c79-c99rg" podStartSLOduration=1.411906013 podStartE2EDuration="4.074279975s" totalImagesPullingTime="2.662373962s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 14:36:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:36:56.072912041 +0000 UTC m=+652.782050449" watchObservedRunningTime="2026-07-07 14:36:56.074279975 +0000 UTC m=+652.783418356" Jul 07 14:36:56.091004 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:36:56.090958 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-58f86c7798-nwgts" podStartSLOduration=1.135872621 podStartE2EDuration="3.090944306s" totalImagesPullingTime="1.955071685s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 14:36:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:36:56.087825294 +0000 UTC m=+652.796963673" watchObservedRunningTime="2026-07-07 14:36:56.090944306 +0000 UTC m=+652.800082696" Jul 07 14:37:01.077832 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:37:01.077795 2557 generic.go:356] "Generic (PLEG): container finished" podID="1861c4fd-2974-42f3-96e8-79601e052d18" containerID="cedd1d73a62e75103cbdbf9ac31733dd6b6ad5bcbf58aa922b768c0b08c1786c" exitCode=0 Jul 07 14:37:01.078252 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:37:01.077867 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-8cnl6" event={"ID":"1861c4fd-2974-42f3-96e8-79601e052d18","Type":"ContainerDied","Data":"cedd1d73a62e75103cbdbf9ac31733dd6b6ad5bcbf58aa922b768c0b08c1786c"} Jul 07 14:37:01.078252 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:37:01.078249 2557 scope.go:117] "RemoveContainer" containerID="cedd1d73a62e75103cbdbf9ac31733dd6b6ad5bcbf58aa922b768c0b08c1786c" Jul 07 14:37:01.079200 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:37:01.079169 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 07 14:37:01.079261 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:37:01.079239 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 07 14:37:01.079298 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:37:01.079269 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 07 14:37:02.084717 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:37:02.084685 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-8cnl6" event={"ID":"1861c4fd-2974-42f3-96e8-79601e052d18","Type":"ContainerStarted","Data":"c925be8544c06ff52aa34debc81070b36198649272d8f558549531feadf797b7"} Jul 07 14:37:02.181494 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:37:02.181457 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-8cnl6" Jul 07 14:37:02.181713 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:37:02.181691 2557 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5896d4dd49-8cnl6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused" start-of-body= Jul 07 14:37:02.181758 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:37:02.181739 2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-8cnl6" podUID="1861c4fd-2974-42f3-96e8-79601e052d18" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused" Jul 07 14:37:03.068041 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:37:03.068015 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-6d878c79-c99rg" Jul 07 14:37:03.181131 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:37:03.181095 2557 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5896d4dd49-8cnl6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused" start-of-body= Jul 07 14:37:03.181540 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:37:03.181149 2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-8cnl6" podUID="1861c4fd-2974-42f3-96e8-79601e052d18" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused" Jul 07 14:37:04.181024 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:37:04.180992 2557 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5896d4dd49-8cnl6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused" start-of-body= Jul 07 14:37:04.181195 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:37:04.181046 2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-8cnl6" podUID="1861c4fd-2974-42f3-96e8-79601e052d18" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused" Jul 07 14:37:04.215679 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:37:04.215644 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:37:04.354658 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:37:04.354598 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:37:05.181290 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:37:05.181250 2557 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5896d4dd49-8cnl6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused" start-of-body= Jul 07 14:37:05.181804 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:37:05.181329 2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-8cnl6" podUID="1861c4fd-2974-42f3-96e8-79601e052d18" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused" Jul 07 14:37:06.184723 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:37:06.184694 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-8cnl6" Jul 07 14:37:06.185113 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:37:06.184966 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-8cnl6" Jul 07 14:37:06.185622 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:37:06.185605 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-8cnl6" Jul 07 14:37:08.063161 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:37:08.063128 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-58f86c7798-nwgts" Jul 07 14:37:08.063584 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:37:08.063195 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-7964457696-5wrr4" Jul 07 14:37:25.641047 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:37:25.640988 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-pqlxw"] Jul 07 14:37:25.644639 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:37:25.644611 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-pqlxw" Jul 07 14:37:25.646477 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:37:25.646454 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 07 14:37:25.646993 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:37:25.646972 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 07 14:37:25.647103 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:37:25.647053 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-wvkhz\"" Jul 07 14:37:25.647103 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:37:25.647068 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\"" Jul 07 14:37:25.653228 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:37:25.653207 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-pqlxw"] Jul 07 14:37:25.749927 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:37:25.749899 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6d9a58b7-bb87-448c-b196-6f3756c687da-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-pqlxw\" (UID: \"6d9a58b7-bb87-448c-b196-6f3756c687da\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-pqlxw" Jul 07 14:37:25.750040 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:37:25.749934 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sqnk5\" (UniqueName: \"kubernetes.io/projected/6d9a58b7-bb87-448c-b196-6f3756c687da-kube-api-access-sqnk5\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-pqlxw\" (UID: \"6d9a58b7-bb87-448c-b196-6f3756c687da\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-pqlxw" Jul 07 14:37:25.750040 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:37:25.749964 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6d9a58b7-bb87-448c-b196-6f3756c687da-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-pqlxw\" (UID: \"6d9a58b7-bb87-448c-b196-6f3756c687da\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-pqlxw" Jul 07 14:37:25.750040 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:37:25.749994 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6d9a58b7-bb87-448c-b196-6f3756c687da-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-pqlxw\" (UID: \"6d9a58b7-bb87-448c-b196-6f3756c687da\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-pqlxw" Jul 07 14:37:25.750040 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:37:25.750031 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6d9a58b7-bb87-448c-b196-6f3756c687da-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-pqlxw\" (UID: \"6d9a58b7-bb87-448c-b196-6f3756c687da\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-pqlxw" Jul 07 14:37:25.750186 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:37:25.750145 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6d9a58b7-bb87-448c-b196-6f3756c687da-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-pqlxw\" (UID: \"6d9a58b7-bb87-448c-b196-6f3756c687da\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-pqlxw" Jul 07 14:37:25.850923 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:37:25.850897 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6d9a58b7-bb87-448c-b196-6f3756c687da-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-pqlxw\" (UID: \"6d9a58b7-bb87-448c-b196-6f3756c687da\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-pqlxw" Jul 07 14:37:25.851045 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:37:25.850927 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sqnk5\" (UniqueName: \"kubernetes.io/projected/6d9a58b7-bb87-448c-b196-6f3756c687da-kube-api-access-sqnk5\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-pqlxw\" (UID: \"6d9a58b7-bb87-448c-b196-6f3756c687da\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-pqlxw" Jul 07 14:37:25.851045 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:37:25.850946 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6d9a58b7-bb87-448c-b196-6f3756c687da-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-pqlxw\" (UID: \"6d9a58b7-bb87-448c-b196-6f3756c687da\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-pqlxw" Jul 07 14:37:25.851045 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:37:25.850962 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6d9a58b7-bb87-448c-b196-6f3756c687da-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-pqlxw\" (UID: \"6d9a58b7-bb87-448c-b196-6f3756c687da\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-pqlxw" Jul 07 14:37:25.851045 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:37:25.850985 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6d9a58b7-bb87-448c-b196-6f3756c687da-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-pqlxw\" (UID: \"6d9a58b7-bb87-448c-b196-6f3756c687da\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-pqlxw" Jul 07 14:37:25.851252 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:37:25.851074 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6d9a58b7-bb87-448c-b196-6f3756c687da-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-pqlxw\" (UID: \"6d9a58b7-bb87-448c-b196-6f3756c687da\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-pqlxw" Jul 07 14:37:25.851387 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:37:25.851362 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6d9a58b7-bb87-448c-b196-6f3756c687da-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-pqlxw\" (UID: \"6d9a58b7-bb87-448c-b196-6f3756c687da\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-pqlxw" Jul 07 14:37:25.851451 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:37:25.851386 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6d9a58b7-bb87-448c-b196-6f3756c687da-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-pqlxw\" (UID: \"6d9a58b7-bb87-448c-b196-6f3756c687da\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-pqlxw" Jul 07 14:37:25.851451 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:37:25.851423 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6d9a58b7-bb87-448c-b196-6f3756c687da-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-pqlxw\" (UID: \"6d9a58b7-bb87-448c-b196-6f3756c687da\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-pqlxw" Jul 07 14:37:25.852994 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:37:25.852977 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6d9a58b7-bb87-448c-b196-6f3756c687da-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-pqlxw\" (UID: \"6d9a58b7-bb87-448c-b196-6f3756c687da\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-pqlxw" Jul 07 14:37:25.853388 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:37:25.853369 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6d9a58b7-bb87-448c-b196-6f3756c687da-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-pqlxw\" (UID: \"6d9a58b7-bb87-448c-b196-6f3756c687da\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-pqlxw" Jul 07 14:37:25.859619 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:37:25.859601 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sqnk5\" (UniqueName: \"kubernetes.io/projected/6d9a58b7-bb87-448c-b196-6f3756c687da-kube-api-access-sqnk5\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-pqlxw\" (UID: \"6d9a58b7-bb87-448c-b196-6f3756c687da\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-pqlxw" Jul 07 14:37:25.956033 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:37:25.955970 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-pqlxw" Jul 07 14:37:26.101497 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:37:26.101469 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-pqlxw"] Jul 07 14:37:26.103043 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:37:26.103019 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6d9a58b7_bb87_448c_b196_6f3756c687da.slice/crio-07f91c4300186160999cbdfebbe4232f3d6d0be227fd809e24afbe1e33de6368 WatchSource:0}: Error finding container 07f91c4300186160999cbdfebbe4232f3d6d0be227fd809e24afbe1e33de6368: Status 404 returned error can't find the container with id 07f91c4300186160999cbdfebbe4232f3d6d0be227fd809e24afbe1e33de6368 Jul 07 14:37:26.212391 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:37:26.212311 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-pqlxw" event={"ID":"6d9a58b7-bb87-448c-b196-6f3756c687da","Type":"ContainerStarted","Data":"07f91c4300186160999cbdfebbe4232f3d6d0be227fd809e24afbe1e33de6368"} Jul 07 14:37:28.224999 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:37:28.224955 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-pqlxw" event={"ID":"6d9a58b7-bb87-448c-b196-6f3756c687da","Type":"ContainerStarted","Data":"2655512700b0e61855b5d3c2d928015304b9da2bede367ee643e5412635455a1"} Jul 07 14:37:28.244675 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:37:28.244624 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-pqlxw" podStartSLOduration=2.108167276 podStartE2EDuration="3.244609104s" totalImagesPullingTime="1.136441828s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 14:37:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:37:28.243163418 +0000 UTC m=+684.952301800" watchObservedRunningTime="2026-07-07 14:37:28.244609104 +0000 UTC m=+684.953747484" Jul 07 14:37:32.270493 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:37:32.270427 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:37:35.957153 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:37:35.957117 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-pqlxw" Jul 07 14:37:35.957570 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:37:35.957181 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-pqlxw" Jul 07 14:37:35.969299 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:37:35.969279 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-pqlxw" Jul 07 14:37:36.283100 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:37:36.283018 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-pqlxw" Jul 07 14:37:47.275542 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:37:47.275498 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-crpwq"] Jul 07 14:37:47.278731 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:37:47.278710 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-crpwq" Jul 07 14:37:47.283250 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:37:47.283232 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\"" Jul 07 14:37:47.300508 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:37:47.300481 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-crpwq"] Jul 07 14:37:47.328446 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:37:47.328400 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:37:47.341337 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:37:47.341317 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9f32c8e5-c239-40f9-bcc2-1b13930acf48-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-crpwq\" (UID: \"9f32c8e5-c239-40f9-bcc2-1b13930acf48\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-crpwq" Jul 07 14:37:47.341442 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:37:47.341346 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9f32c8e5-c239-40f9-bcc2-1b13930acf48-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-crpwq\" (UID: \"9f32c8e5-c239-40f9-bcc2-1b13930acf48\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-crpwq" Jul 07 14:37:47.341442 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:37:47.341376 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9f32c8e5-c239-40f9-bcc2-1b13930acf48-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-crpwq\" (UID: \"9f32c8e5-c239-40f9-bcc2-1b13930acf48\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-crpwq" Jul 07 14:37:47.341511 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:37:47.341440 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tsx8m\" (UniqueName: \"kubernetes.io/projected/9f32c8e5-c239-40f9-bcc2-1b13930acf48-kube-api-access-tsx8m\") pod \"e2e-trlp-test-simulated-kserve-598466548d-crpwq\" (UID: \"9f32c8e5-c239-40f9-bcc2-1b13930acf48\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-crpwq" Jul 07 14:37:47.341511 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:37:47.341465 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9f32c8e5-c239-40f9-bcc2-1b13930acf48-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-crpwq\" (UID: \"9f32c8e5-c239-40f9-bcc2-1b13930acf48\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-crpwq" Jul 07 14:37:47.341611 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:37:47.341583 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9f32c8e5-c239-40f9-bcc2-1b13930acf48-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-crpwq\" (UID: \"9f32c8e5-c239-40f9-bcc2-1b13930acf48\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-crpwq" Jul 07 14:37:47.442992 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:37:47.442963 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9f32c8e5-c239-40f9-bcc2-1b13930acf48-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-crpwq\" (UID: \"9f32c8e5-c239-40f9-bcc2-1b13930acf48\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-crpwq" Jul 07 14:37:47.443123 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:37:47.443050 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9f32c8e5-c239-40f9-bcc2-1b13930acf48-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-crpwq\" (UID: \"9f32c8e5-c239-40f9-bcc2-1b13930acf48\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-crpwq" Jul 07 14:37:47.443123 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:37:47.443095 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9f32c8e5-c239-40f9-bcc2-1b13930acf48-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-crpwq\" (UID: \"9f32c8e5-c239-40f9-bcc2-1b13930acf48\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-crpwq" Jul 07 14:37:47.443207 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:37:47.443121 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9f32c8e5-c239-40f9-bcc2-1b13930acf48-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-crpwq\" (UID: \"9f32c8e5-c239-40f9-bcc2-1b13930acf48\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-crpwq" Jul 07 14:37:47.443207 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:37:47.443158 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9f32c8e5-c239-40f9-bcc2-1b13930acf48-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-crpwq\" (UID: \"9f32c8e5-c239-40f9-bcc2-1b13930acf48\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-crpwq" Jul 07 14:37:47.443207 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:37:47.443196 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tsx8m\" (UniqueName: \"kubernetes.io/projected/9f32c8e5-c239-40f9-bcc2-1b13930acf48-kube-api-access-tsx8m\") pod \"e2e-trlp-test-simulated-kserve-598466548d-crpwq\" (UID: \"9f32c8e5-c239-40f9-bcc2-1b13930acf48\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-crpwq" Jul 07 14:37:47.443492 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:37:47.443465 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9f32c8e5-c239-40f9-bcc2-1b13930acf48-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-crpwq\" (UID: \"9f32c8e5-c239-40f9-bcc2-1b13930acf48\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-crpwq" Jul 07 14:37:47.443573 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:37:47.443507 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9f32c8e5-c239-40f9-bcc2-1b13930acf48-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-crpwq\" (UID: \"9f32c8e5-c239-40f9-bcc2-1b13930acf48\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-crpwq" Jul 07 14:37:47.443573 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:37:47.443552 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9f32c8e5-c239-40f9-bcc2-1b13930acf48-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-crpwq\" (UID: \"9f32c8e5-c239-40f9-bcc2-1b13930acf48\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-crpwq" Jul 07 14:37:47.445355 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:37:47.445324 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9f32c8e5-c239-40f9-bcc2-1b13930acf48-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-crpwq\" (UID: \"9f32c8e5-c239-40f9-bcc2-1b13930acf48\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-crpwq" Jul 07 14:37:47.445751 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:37:47.445730 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9f32c8e5-c239-40f9-bcc2-1b13930acf48-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-crpwq\" (UID: \"9f32c8e5-c239-40f9-bcc2-1b13930acf48\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-crpwq" Jul 07 14:37:47.450846 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:37:47.450829 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tsx8m\" (UniqueName: \"kubernetes.io/projected/9f32c8e5-c239-40f9-bcc2-1b13930acf48-kube-api-access-tsx8m\") pod \"e2e-trlp-test-simulated-kserve-598466548d-crpwq\" (UID: \"9f32c8e5-c239-40f9-bcc2-1b13930acf48\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-crpwq" Jul 07 14:37:47.589027 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:37:47.588997 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-crpwq" Jul 07 14:37:47.720764 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:37:47.720741 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-crpwq"] Jul 07 14:37:47.722585 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:37:47.722561 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9f32c8e5_c239_40f9_bcc2_1b13930acf48.slice/crio-4bc25e402325e9544f6ce3797cfb5725fb9579c4fb83f07cffd514212129e0d6 WatchSource:0}: Error finding container 4bc25e402325e9544f6ce3797cfb5725fb9579c4fb83f07cffd514212129e0d6: Status 404 returned error can't find the container with id 4bc25e402325e9544f6ce3797cfb5725fb9579c4fb83f07cffd514212129e0d6 Jul 07 14:37:48.338262 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:37:48.338228 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-crpwq" event={"ID":"9f32c8e5-c239-40f9-bcc2-1b13930acf48","Type":"ContainerStarted","Data":"4147e9c4d6257a8ff4d6e20b00cdfd88d2002da8876df655d2a24087a70213a6"} Jul 07 14:37:48.338674 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:37:48.338307 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-crpwq" event={"ID":"9f32c8e5-c239-40f9-bcc2-1b13930acf48","Type":"ContainerStarted","Data":"4bc25e402325e9544f6ce3797cfb5725fb9579c4fb83f07cffd514212129e0d6"} Jul 07 14:37:48.355624 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:37:48.355563 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-crpwq" podStartSLOduration=1.181053536 podStartE2EDuration="1.355550071s" totalImagesPullingTime="174.496535ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 14:37:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:37:48.355261294 +0000 UTC m=+705.064399685" watchObservedRunningTime="2026-07-07 14:37:48.355550071 +0000 UTC m=+705.064688451" Jul 07 14:37:57.589883 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:37:57.589771 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-crpwq" Jul 07 14:37:57.589883 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:37:57.589844 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-crpwq" Jul 07 14:37:57.601950 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:37:57.601921 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-crpwq" Jul 07 14:37:58.406086 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:37:58.406060 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-crpwq" Jul 07 14:38:04.572203 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:38:04.572171 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:38:08.534600 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:38:08.534563 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-nwzgs"] Jul 07 14:38:08.537694 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:38:08.537668 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-nwzgs" Jul 07 14:38:08.539710 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:38:08.539691 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\"" Jul 07 14:38:08.550506 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:38:08.550481 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-nwzgs"] Jul 07 14:38:08.560303 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:38:08.560277 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:38:08.617976 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:38:08.617943 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/352914b1-246c-4f00-a424-29b9323b9223-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-nwzgs\" (UID: \"352914b1-246c-4f00-a424-29b9323b9223\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-nwzgs" Jul 07 14:38:08.617976 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:38:08.617975 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/352914b1-246c-4f00-a424-29b9323b9223-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-nwzgs\" (UID: \"352914b1-246c-4f00-a424-29b9323b9223\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-nwzgs" Jul 07 14:38:08.618174 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:38:08.617996 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/352914b1-246c-4f00-a424-29b9323b9223-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-nwzgs\" (UID: \"352914b1-246c-4f00-a424-29b9323b9223\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-nwzgs" Jul 07 14:38:08.618174 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:38:08.618106 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/352914b1-246c-4f00-a424-29b9323b9223-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-nwzgs\" (UID: \"352914b1-246c-4f00-a424-29b9323b9223\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-nwzgs" Jul 07 14:38:08.618174 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:38:08.618137 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4lp8h\" (UniqueName: \"kubernetes.io/projected/352914b1-246c-4f00-a424-29b9323b9223-kube-api-access-4lp8h\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-nwzgs\" (UID: \"352914b1-246c-4f00-a424-29b9323b9223\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-nwzgs" Jul 07 14:38:08.618319 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:38:08.618208 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/352914b1-246c-4f00-a424-29b9323b9223-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-nwzgs\" (UID: \"352914b1-246c-4f00-a424-29b9323b9223\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-nwzgs" Jul 07 14:38:08.719817 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:38:08.719780 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/352914b1-246c-4f00-a424-29b9323b9223-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-nwzgs\" (UID: \"352914b1-246c-4f00-a424-29b9323b9223\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-nwzgs" Jul 07 14:38:08.720008 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:38:08.719895 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/352914b1-246c-4f00-a424-29b9323b9223-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-nwzgs\" (UID: \"352914b1-246c-4f00-a424-29b9323b9223\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-nwzgs" Jul 07 14:38:08.720008 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:38:08.719914 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/352914b1-246c-4f00-a424-29b9323b9223-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-nwzgs\" (UID: \"352914b1-246c-4f00-a424-29b9323b9223\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-nwzgs" Jul 07 14:38:08.720008 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:38:08.719932 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/352914b1-246c-4f00-a424-29b9323b9223-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-nwzgs\" (UID: \"352914b1-246c-4f00-a424-29b9323b9223\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-nwzgs" Jul 07 14:38:08.720186 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:38:08.720019 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/352914b1-246c-4f00-a424-29b9323b9223-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-nwzgs\" (UID: \"352914b1-246c-4f00-a424-29b9323b9223\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-nwzgs" Jul 07 14:38:08.720186 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:38:08.720045 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4lp8h\" (UniqueName: \"kubernetes.io/projected/352914b1-246c-4f00-a424-29b9323b9223-kube-api-access-4lp8h\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-nwzgs\" (UID: \"352914b1-246c-4f00-a424-29b9323b9223\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-nwzgs" Jul 07 14:38:08.720324 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:38:08.720298 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/352914b1-246c-4f00-a424-29b9323b9223-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-nwzgs\" (UID: \"352914b1-246c-4f00-a424-29b9323b9223\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-nwzgs" Jul 07 14:38:08.720396 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:38:08.720361 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/352914b1-246c-4f00-a424-29b9323b9223-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-nwzgs\" (UID: \"352914b1-246c-4f00-a424-29b9323b9223\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-nwzgs" Jul 07 14:38:08.720572 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:38:08.720551 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/352914b1-246c-4f00-a424-29b9323b9223-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-nwzgs\" (UID: \"352914b1-246c-4f00-a424-29b9323b9223\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-nwzgs" Jul 07 14:38:08.722332 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:38:08.722311 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/352914b1-246c-4f00-a424-29b9323b9223-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-nwzgs\" (UID: \"352914b1-246c-4f00-a424-29b9323b9223\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-nwzgs" Jul 07 14:38:08.722671 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:38:08.722653 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/352914b1-246c-4f00-a424-29b9323b9223-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-nwzgs\" (UID: \"352914b1-246c-4f00-a424-29b9323b9223\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-nwzgs" Jul 07 14:38:08.726992 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:38:08.726969 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4lp8h\" (UniqueName: \"kubernetes.io/projected/352914b1-246c-4f00-a424-29b9323b9223-kube-api-access-4lp8h\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-nwzgs\" (UID: \"352914b1-246c-4f00-a424-29b9323b9223\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-nwzgs" Jul 07 14:38:08.847835 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:38:08.847804 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-nwzgs" Jul 07 14:38:08.978156 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:38:08.978128 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-nwzgs"] Jul 07 14:38:08.979384 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:38:08.979344 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod352914b1_246c_4f00_a424_29b9323b9223.slice/crio-17fee625e40d617c25ac2f4c37d1f9dff9296fd9d6888eeb25d1d9f703d3314f WatchSource:0}: Error finding container 17fee625e40d617c25ac2f4c37d1f9dff9296fd9d6888eeb25d1d9f703d3314f: Status 404 returned error can't find the container with id 17fee625e40d617c25ac2f4c37d1f9dff9296fd9d6888eeb25d1d9f703d3314f Jul 07 14:38:09.458298 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:38:09.458270 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-nwzgs" event={"ID":"352914b1-246c-4f00-a424-29b9323b9223","Type":"ContainerStarted","Data":"17fee625e40d617c25ac2f4c37d1f9dff9296fd9d6888eeb25d1d9f703d3314f"} Jul 07 14:38:10.465538 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:38:10.465484 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-nwzgs" event={"ID":"352914b1-246c-4f00-a424-29b9323b9223","Type":"ContainerStarted","Data":"5a0b6d1d0f353818e60a9ee888748c876e8f4f34f3c0bd3310d893fe14a7ad6d"} Jul 07 14:38:10.494742 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:38:10.494688 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-nwzgs" podStartSLOduration=1.872679369 podStartE2EDuration="2.494673808s" totalImagesPullingTime="621.994439ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 14:38:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:38:10.49159963 +0000 UTC m=+727.200738004" watchObservedRunningTime="2026-07-07 14:38:10.494673808 +0000 UTC m=+727.203812187" Jul 07 14:38:18.848681 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:38:18.848645 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-nwzgs" Jul 07 14:38:18.849135 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:38:18.848700 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-nwzgs" Jul 07 14:38:18.860959 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:38:18.860938 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-nwzgs" Jul 07 14:38:19.528082 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:38:19.528051 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-nwzgs" Jul 07 14:38:38.484850 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:38:38.484809 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:38:55.456430 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:38:55.456390 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:40:00.318226 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:40:00.318185 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:40:00.414878 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:40:00.414839 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:40:01.631250 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:40:01.631217 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:40:01.731655 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:40:01.731623 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:40:28.989140 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:40:28.989105 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:40:42.468810 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:40:42.468777 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:41:05.508539 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:41:05.508490 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-vd9g9_0fa80df4-c933-4758-b75f-3f70b7217c0d/console-operator/2.log" Jul 07 14:41:05.509142 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:41:05.509121 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-vd9g9_0fa80df4-c933-4758-b75f-3f70b7217c0d/console-operator/2.log" Jul 07 14:41:05.513030 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:41:05.513011 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-scfg4_90d2dce1-be42-4595-b9c4-74a024eef0e9/ovn-acl-logging/0.log" Jul 07 14:41:05.513433 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:41:05.513413 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-scfg4_90d2dce1-be42-4595-b9c4-74a024eef0e9/ovn-acl-logging/0.log" Jul 07 14:41:21.198341 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:41:21.198299 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:41:21.300356 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:41:21.300328 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-d4dcbc546-nwm2q"] Jul 07 14:41:21.300643 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:41:21.300616 2557 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-controller-d4dcbc546-nwm2q" podUID="d3af5d7a-0e15-4e9a-8409-ac49fb63b4c9" containerName="manager" containerID="cri-o://8b539dddc64ad3859a5271d34f89edf6c7a02b693aad620e46d9e415e2d7c86d" gracePeriod=30 Jul 07 14:41:21.537195 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:41:21.537145 2557 generic.go:356] "Generic (PLEG): container finished" podID="d3af5d7a-0e15-4e9a-8409-ac49fb63b4c9" containerID="8b539dddc64ad3859a5271d34f89edf6c7a02b693aad620e46d9e415e2d7c86d" exitCode=0 Jul 07 14:41:21.537334 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:41:21.537224 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-d4dcbc546-nwm2q" event={"ID":"d3af5d7a-0e15-4e9a-8409-ac49fb63b4c9","Type":"ContainerDied","Data":"8b539dddc64ad3859a5271d34f89edf6c7a02b693aad620e46d9e415e2d7c86d"} Jul 07 14:41:21.537334 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:41:21.537299 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-d4dcbc546-nwm2q" event={"ID":"d3af5d7a-0e15-4e9a-8409-ac49fb63b4c9","Type":"ContainerDied","Data":"c6693a11954126637070beb7761d0a1de81e143ee92d22fdafd83ca90031117b"} Jul 07 14:41:21.537334 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:41:21.537325 2557 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c6693a11954126637070beb7761d0a1de81e143ee92d22fdafd83ca90031117b" Jul 07 14:41:21.550780 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:41:21.550763 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-d4dcbc546-nwm2q" Jul 07 14:41:21.602157 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:41:21.602136 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d3af5d7a-0e15-4e9a-8409-ac49fb63b4c9-webhook-cert\") pod \"d3af5d7a-0e15-4e9a-8409-ac49fb63b4c9\" (UID: \"d3af5d7a-0e15-4e9a-8409-ac49fb63b4c9\") " Jul 07 14:41:21.602302 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:41:21.602225 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pbjmd\" (UniqueName: \"kubernetes.io/projected/d3af5d7a-0e15-4e9a-8409-ac49fb63b4c9-kube-api-access-pbjmd\") pod \"d3af5d7a-0e15-4e9a-8409-ac49fb63b4c9\" (UID: \"d3af5d7a-0e15-4e9a-8409-ac49fb63b4c9\") " Jul 07 14:41:21.604128 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:41:21.604099 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d3af5d7a-0e15-4e9a-8409-ac49fb63b4c9-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "d3af5d7a-0e15-4e9a-8409-ac49fb63b4c9" (UID: "d3af5d7a-0e15-4e9a-8409-ac49fb63b4c9"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 14:41:21.604247 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:41:21.604218 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d3af5d7a-0e15-4e9a-8409-ac49fb63b4c9-kube-api-access-pbjmd" (OuterVolumeSpecName: "kube-api-access-pbjmd") pod "d3af5d7a-0e15-4e9a-8409-ac49fb63b4c9" (UID: "d3af5d7a-0e15-4e9a-8409-ac49fb63b4c9"). InnerVolumeSpecName "kube-api-access-pbjmd". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 14:41:21.703110 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:41:21.703085 2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pbjmd\" (UniqueName: \"kubernetes.io/projected/d3af5d7a-0e15-4e9a-8409-ac49fb63b4c9-kube-api-access-pbjmd\") on node \"ip-10-0-134-42.ec2.internal\" DevicePath \"\"" Jul 07 14:41:21.703110 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:41:21.703108 2557 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d3af5d7a-0e15-4e9a-8409-ac49fb63b4c9-webhook-cert\") on node \"ip-10-0-134-42.ec2.internal\" DevicePath \"\"" Jul 07 14:41:22.543062 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:41:22.543030 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-d4dcbc546-nwm2q" Jul 07 14:41:22.558578 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:41:22.558550 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-d4dcbc546-nwm2q"] Jul 07 14:41:22.560795 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:41:22.560777 2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-d4dcbc546-nwm2q"] Jul 07 14:41:23.276198 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:41:23.276159 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:41:23.412012 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:41:23.411981 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-d4dcbc546-dp5b9"] Jul 07 14:41:23.412411 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:41:23.412399 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d3af5d7a-0e15-4e9a-8409-ac49fb63b4c9" containerName="manager" Jul 07 14:41:23.412456 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:41:23.412414 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="d3af5d7a-0e15-4e9a-8409-ac49fb63b4c9" containerName="manager" Jul 07 14:41:23.412503 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:41:23.412494 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="d3af5d7a-0e15-4e9a-8409-ac49fb63b4c9" containerName="manager" Jul 07 14:41:23.415014 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:41:23.414997 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-d4dcbc546-dp5b9" Jul 07 14:41:23.417462 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:41:23.417436 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jul 07 14:41:23.418462 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:41:23.418431 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jul 07 14:41:23.418584 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:41:23.418567 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-nzgrn\"" Jul 07 14:41:23.431588 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:41:23.431565 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-d4dcbc546-dp5b9"] Jul 07 14:41:23.518794 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:41:23.518768 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/12fd21f2-da38-49be-a702-eb46c7d7ea97-webhook-cert\") pod \"maas-controller-d4dcbc546-dp5b9\" (UID: \"12fd21f2-da38-49be-a702-eb46c7d7ea97\") " pod="opendatahub/maas-controller-d4dcbc546-dp5b9" Jul 07 14:41:23.518913 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:41:23.518808 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fcmpt\" (UniqueName: \"kubernetes.io/projected/12fd21f2-da38-49be-a702-eb46c7d7ea97-kube-api-access-fcmpt\") pod \"maas-controller-d4dcbc546-dp5b9\" (UID: \"12fd21f2-da38-49be-a702-eb46c7d7ea97\") " pod="opendatahub/maas-controller-d4dcbc546-dp5b9" Jul 07 14:41:23.620094 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:41:23.620056 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/12fd21f2-da38-49be-a702-eb46c7d7ea97-webhook-cert\") pod \"maas-controller-d4dcbc546-dp5b9\" (UID: \"12fd21f2-da38-49be-a702-eb46c7d7ea97\") " pod="opendatahub/maas-controller-d4dcbc546-dp5b9" Jul 07 14:41:23.620478 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:41:23.620117 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fcmpt\" (UniqueName: \"kubernetes.io/projected/12fd21f2-da38-49be-a702-eb46c7d7ea97-kube-api-access-fcmpt\") pod \"maas-controller-d4dcbc546-dp5b9\" (UID: \"12fd21f2-da38-49be-a702-eb46c7d7ea97\") " pod="opendatahub/maas-controller-d4dcbc546-dp5b9" Jul 07 14:41:23.622505 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:41:23.622484 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/12fd21f2-da38-49be-a702-eb46c7d7ea97-webhook-cert\") pod \"maas-controller-d4dcbc546-dp5b9\" (UID: \"12fd21f2-da38-49be-a702-eb46c7d7ea97\") " pod="opendatahub/maas-controller-d4dcbc546-dp5b9" Jul 07 14:41:23.630246 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:41:23.630222 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fcmpt\" (UniqueName: \"kubernetes.io/projected/12fd21f2-da38-49be-a702-eb46c7d7ea97-kube-api-access-fcmpt\") pod \"maas-controller-d4dcbc546-dp5b9\" (UID: \"12fd21f2-da38-49be-a702-eb46c7d7ea97\") " pod="opendatahub/maas-controller-d4dcbc546-dp5b9" Jul 07 14:41:23.726613 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:41:23.726578 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-d4dcbc546-dp5b9" Jul 07 14:41:23.838879 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:41:23.838849 2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d3af5d7a-0e15-4e9a-8409-ac49fb63b4c9" path="/var/lib/kubelet/pods/d3af5d7a-0e15-4e9a-8409-ac49fb63b4c9/volumes" Jul 07 14:41:23.858130 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:41:23.858101 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-d4dcbc546-dp5b9"] Jul 07 14:41:23.859873 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:41:23.859844 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod12fd21f2_da38_49be_a702_eb46c7d7ea97.slice/crio-ef6d9b159ae005ccc11d236f09844142a642761bc29903cb87cbdd26ccd3da56 WatchSource:0}: Error finding container ef6d9b159ae005ccc11d236f09844142a642761bc29903cb87cbdd26ccd3da56: Status 404 returned error can't find the container with id ef6d9b159ae005ccc11d236f09844142a642761bc29903cb87cbdd26ccd3da56 Jul 07 14:41:24.489149 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:41:24.489121 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:41:24.562401 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:41:24.562329 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-d4dcbc546-dp5b9" event={"ID":"12fd21f2-da38-49be-a702-eb46c7d7ea97","Type":"ContainerStarted","Data":"8e25d9303227f8e3ff369ec454845671f8bf37c2b5f184f054827efc7648b63a"} Jul 07 14:41:24.562401 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:41:24.562386 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-d4dcbc546-dp5b9" event={"ID":"12fd21f2-da38-49be-a702-eb46c7d7ea97","Type":"ContainerStarted","Data":"ef6d9b159ae005ccc11d236f09844142a642761bc29903cb87cbdd26ccd3da56"} Jul 07 14:41:24.562587 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:41:24.562413 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-d4dcbc546-dp5b9" Jul 07 14:41:24.590637 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:41:24.590597 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-d4dcbc546-dp5b9" podStartSLOduration=1.153917278 podStartE2EDuration="1.590582176s" totalImagesPullingTime="436.664898ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 14:41:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:41:24.587536838 +0000 UTC m=+921.296675212" watchObservedRunningTime="2026-07-07 14:41:24.590582176 +0000 UTC m=+921.299720576" Jul 07 14:41:30.865202 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:41:30.865167 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:41:30.937153 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:41:30.937110 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:41:35.573009 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:41:35.572980 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-d4dcbc546-dp5b9" Jul 07 14:42:02.490086 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:42:02.490043 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:42:03.815918 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:42:03.815889 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:42:06.158271 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:42:06.158237 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:42:09.155166 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:42:09.155122 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:42:09.260590 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:42:09.260561 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:42:09.355671 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:42:09.355644 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:42:09.612831 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:42:09.612794 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:42:15.582631 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:42:15.582588 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:42:15.681993 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:42:15.681951 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:43:03.970199 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:43:03.970168 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:43:05.518341 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:43:05.518312 2557 scope.go:117] "RemoveContainer" containerID="8b539dddc64ad3859a5271d34f89edf6c7a02b693aad620e46d9e415e2d7c86d" Jul 07 14:43:12.734019 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:43:12.733982 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:45:00.141423 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:45:00.141388 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-key-cleanup-29723925-clxr2"] Jul 07 14:45:00.144811 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:45:00.144794 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29723925-clxr2" Jul 07 14:45:00.164230 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:45:00.164205 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-29723925-clxr2"] Jul 07 14:45:00.225405 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:45:00.225367 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jh8gd\" (UniqueName: \"kubernetes.io/projected/9b8a5584-1e8a-45df-a7d1-e9bf146ea96f-kube-api-access-jh8gd\") pod \"maas-api-key-cleanup-29723925-clxr2\" (UID: \"9b8a5584-1e8a-45df-a7d1-e9bf146ea96f\") " pod="opendatahub/maas-api-key-cleanup-29723925-clxr2" Jul 07 14:45:00.326704 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:45:00.326651 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jh8gd\" (UniqueName: \"kubernetes.io/projected/9b8a5584-1e8a-45df-a7d1-e9bf146ea96f-kube-api-access-jh8gd\") pod \"maas-api-key-cleanup-29723925-clxr2\" (UID: \"9b8a5584-1e8a-45df-a7d1-e9bf146ea96f\") " pod="opendatahub/maas-api-key-cleanup-29723925-clxr2" Jul 07 14:45:00.334717 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:45:00.334693 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jh8gd\" (UniqueName: \"kubernetes.io/projected/9b8a5584-1e8a-45df-a7d1-e9bf146ea96f-kube-api-access-jh8gd\") pod \"maas-api-key-cleanup-29723925-clxr2\" (UID: \"9b8a5584-1e8a-45df-a7d1-e9bf146ea96f\") " pod="opendatahub/maas-api-key-cleanup-29723925-clxr2" Jul 07 14:45:00.454949 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:45:00.454868 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29723925-clxr2" Jul 07 14:45:00.584677 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:45:00.584650 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-29723925-clxr2"] Jul 07 14:45:00.590705 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:45:00.590678 2557 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 07 14:45:00.807140 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:45:00.807054 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29723925-clxr2" event={"ID":"9b8a5584-1e8a-45df-a7d1-e9bf146ea96f","Type":"ContainerStarted","Data":"b4992d08f2bb25378b835319a82bbc312c6a49356fa6ce1b83dde227b067e529"} Jul 07 14:45:04.831473 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:45:04.831432 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29723925-clxr2" event={"ID":"9b8a5584-1e8a-45df-a7d1-e9bf146ea96f","Type":"ContainerStarted","Data":"1b2318126b7f933e20b18bc961ace60ae5228818e881f29f131fdc7aa5929e3f"} Jul 07 14:45:04.848090 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:45:04.848038 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-key-cleanup-29723925-clxr2" podStartSLOduration=1.022640342 podStartE2EDuration="4.848024017s" totalImagesPullingTime="3.825383675s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 14:45:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:45:04.845372906 +0000 UTC m=+1141.554511296" watchObservedRunningTime="2026-07-07 14:45:04.848024017 +0000 UTC m=+1141.557162397" Jul 07 14:45:05.838535 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:45:05.838486 2557 generic.go:356] "Generic (PLEG): container finished" podID="9b8a5584-1e8a-45df-a7d1-e9bf146ea96f" containerID="1b2318126b7f933e20b18bc961ace60ae5228818e881f29f131fdc7aa5929e3f" exitCode=0 Jul 07 14:45:05.838901 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:45:05.838552 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29723925-clxr2" event={"ID":"9b8a5584-1e8a-45df-a7d1-e9bf146ea96f","Type":"ContainerDied","Data":"1b2318126b7f933e20b18bc961ace60ae5228818e881f29f131fdc7aa5929e3f"} Jul 07 14:45:06.983628 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:45:06.983599 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29723925-clxr2" Jul 07 14:45:07.088141 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:45:07.088114 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jh8gd\" (UniqueName: \"kubernetes.io/projected/9b8a5584-1e8a-45df-a7d1-e9bf146ea96f-kube-api-access-jh8gd\") pod \"9b8a5584-1e8a-45df-a7d1-e9bf146ea96f\" (UID: \"9b8a5584-1e8a-45df-a7d1-e9bf146ea96f\") " Jul 07 14:45:07.089894 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:45:07.089871 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9b8a5584-1e8a-45df-a7d1-e9bf146ea96f-kube-api-access-jh8gd" (OuterVolumeSpecName: "kube-api-access-jh8gd") pod "9b8a5584-1e8a-45df-a7d1-e9bf146ea96f" (UID: "9b8a5584-1e8a-45df-a7d1-e9bf146ea96f"). InnerVolumeSpecName "kube-api-access-jh8gd". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 14:45:07.189391 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:45:07.189364 2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jh8gd\" (UniqueName: \"kubernetes.io/projected/9b8a5584-1e8a-45df-a7d1-e9bf146ea96f-kube-api-access-jh8gd\") on node \"ip-10-0-134-42.ec2.internal\" DevicePath \"\"" Jul 07 14:45:07.849991 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:45:07.849962 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29723925-clxr2" event={"ID":"9b8a5584-1e8a-45df-a7d1-e9bf146ea96f","Type":"ContainerDied","Data":"b4992d08f2bb25378b835319a82bbc312c6a49356fa6ce1b83dde227b067e529"} Jul 07 14:45:07.850125 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:45:07.850009 2557 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b4992d08f2bb25378b835319a82bbc312c6a49356fa6ce1b83dde227b067e529" Jul 07 14:45:07.850125 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:45:07.849971 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29723925-clxr2" Jul 07 14:46:05.545029 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:46:05.544999 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-vd9g9_0fa80df4-c933-4758-b75f-3f70b7217c0d/console-operator/2.log" Jul 07 14:46:05.548988 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:46:05.548966 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-vd9g9_0fa80df4-c933-4758-b75f-3f70b7217c0d/console-operator/2.log" Jul 07 14:46:05.549657 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:46:05.549636 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-scfg4_90d2dce1-be42-4595-b9c4-74a024eef0e9/ovn-acl-logging/0.log" Jul 07 14:46:05.553062 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:46:05.553044 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-scfg4_90d2dce1-be42-4595-b9c4-74a024eef0e9/ovn-acl-logging/0.log" Jul 07 14:47:22.475359 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:47:22.475321 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:48:28.315447 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:48:28.315312 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:48:28.440616 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:48:28.440584 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:51:05.583949 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:51:05.583918 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-vd9g9_0fa80df4-c933-4758-b75f-3f70b7217c0d/console-operator/2.log" Jul 07 14:51:05.588708 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:51:05.588688 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-scfg4_90d2dce1-be42-4595-b9c4-74a024eef0e9/ovn-acl-logging/0.log" Jul 07 14:51:05.590834 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:51:05.590809 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-vd9g9_0fa80df4-c933-4758-b75f-3f70b7217c0d/console-operator/2.log" Jul 07 14:51:05.595394 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:51:05.595371 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-scfg4_90d2dce1-be42-4595-b9c4-74a024eef0e9/ovn-acl-logging/0.log" Jul 07 14:52:12.984485 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:52:12.984448 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:52:49.385297 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:52:49.385257 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:53:04.834194 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:53:04.834154 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:53:14.951039 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:53:14.950971 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:53:23.305723 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:53:23.305678 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:53:28.461788 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:53:28.461752 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:53:31.531923 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:53:31.531882 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:53:31.629078 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:53:31.629050 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:53:32.065115 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:53:32.065076 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:53:38.240383 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:53:38.240351 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:53:38.342687 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:53:38.342653 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:53:38.491118 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:53:38.491026 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:53:38.591307 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:53:38.591266 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:53:39.584386 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:53:39.584352 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:53:39.650457 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:53:39.650422 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-ait-9e0aa07d-7588b756c-f529v"] Jul 07 14:53:39.651059 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:53:39.651034 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9b8a5584-1e8a-45df-a7d1-e9bf146ea96f" containerName="cleanup" Jul 07 14:53:39.651059 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:53:39.651060 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="9b8a5584-1e8a-45df-a7d1-e9bf146ea96f" containerName="cleanup" Jul 07 14:53:39.651214 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:53:39.651202 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="9b8a5584-1e8a-45df-a7d1-e9bf146ea96f" containerName="cleanup" Jul 07 14:53:39.654761 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:53:39.654743 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-9e0aa07d-7588b756c-f529v" Jul 07 14:53:39.656620 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:53:39.656604 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-ait-9e0aa07d-serving-cert\"" Jul 07 14:53:39.668196 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:53:39.668179 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-ait-9e0aa07d-7588b756c-f529v"] Jul 07 14:53:39.671264 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:53:39.671243 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:53:39.742028 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:53:39.742008 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:53:39.770726 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:53:39.770703 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f995898a-20c5-4271-9ed8-c5d5c36b9a9a-openshift-service-ca\") pod \"maas-api-e2e-ait-9e0aa07d-7588b756c-f529v\" (UID: \"f995898a-20c5-4271-9ed8-c5d5c36b9a9a\") " pod="opendatahub/maas-api-e2e-ait-9e0aa07d-7588b756c-f529v" Jul 07 14:53:39.770876 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:53:39.770754 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f995898a-20c5-4271-9ed8-c5d5c36b9a9a-maas-api-tls\") pod \"maas-api-e2e-ait-9e0aa07d-7588b756c-f529v\" (UID: \"f995898a-20c5-4271-9ed8-c5d5c36b9a9a\") " pod="opendatahub/maas-api-e2e-ait-9e0aa07d-7588b756c-f529v" Jul 07 14:53:39.770876 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:53:39.770848 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kvztv\" (UniqueName: \"kubernetes.io/projected/f995898a-20c5-4271-9ed8-c5d5c36b9a9a-kube-api-access-kvztv\") pod \"maas-api-e2e-ait-9e0aa07d-7588b756c-f529v\" (UID: \"f995898a-20c5-4271-9ed8-c5d5c36b9a9a\") " pod="opendatahub/maas-api-e2e-ait-9e0aa07d-7588b756c-f529v" Jul 07 14:53:39.871846 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:53:39.871783 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f995898a-20c5-4271-9ed8-c5d5c36b9a9a-openshift-service-ca\") pod \"maas-api-e2e-ait-9e0aa07d-7588b756c-f529v\" (UID: \"f995898a-20c5-4271-9ed8-c5d5c36b9a9a\") " pod="opendatahub/maas-api-e2e-ait-9e0aa07d-7588b756c-f529v" Jul 07 14:53:39.871846 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:53:39.871825 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f995898a-20c5-4271-9ed8-c5d5c36b9a9a-maas-api-tls\") pod \"maas-api-e2e-ait-9e0aa07d-7588b756c-f529v\" (UID: \"f995898a-20c5-4271-9ed8-c5d5c36b9a9a\") " pod="opendatahub/maas-api-e2e-ait-9e0aa07d-7588b756c-f529v" Jul 07 14:53:39.872002 ip-10-0-134-42 kubenswrapper[2557]: E0707 14:53:39.871910 2557 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-ait-9e0aa07d-serving-cert: secret "maas-api-e2e-ait-9e0aa07d-serving-cert" not found Jul 07 14:53:39.872002 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:53:39.871937 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kvztv\" (UniqueName: \"kubernetes.io/projected/f995898a-20c5-4271-9ed8-c5d5c36b9a9a-kube-api-access-kvztv\") pod \"maas-api-e2e-ait-9e0aa07d-7588b756c-f529v\" (UID: \"f995898a-20c5-4271-9ed8-c5d5c36b9a9a\") " pod="opendatahub/maas-api-e2e-ait-9e0aa07d-7588b756c-f529v" Jul 07 14:53:39.872002 ip-10-0-134-42 kubenswrapper[2557]: E0707 14:53:39.871962 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f995898a-20c5-4271-9ed8-c5d5c36b9a9a-maas-api-tls podName:f995898a-20c5-4271-9ed8-c5d5c36b9a9a nodeName:}" failed. No retries permitted until 2026-07-07 14:53:40.371948372 +0000 UTC m=+1657.081086731 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/f995898a-20c5-4271-9ed8-c5d5c36b9a9a-maas-api-tls") pod "maas-api-e2e-ait-9e0aa07d-7588b756c-f529v" (UID: "f995898a-20c5-4271-9ed8-c5d5c36b9a9a") : secret "maas-api-e2e-ait-9e0aa07d-serving-cert" not found Jul 07 14:53:39.872408 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:53:39.872392 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f995898a-20c5-4271-9ed8-c5d5c36b9a9a-openshift-service-ca\") pod \"maas-api-e2e-ait-9e0aa07d-7588b756c-f529v\" (UID: \"f995898a-20c5-4271-9ed8-c5d5c36b9a9a\") " pod="opendatahub/maas-api-e2e-ait-9e0aa07d-7588b756c-f529v" Jul 07 14:53:39.905712 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:53:39.905690 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kvztv\" (UniqueName: \"kubernetes.io/projected/f995898a-20c5-4271-9ed8-c5d5c36b9a9a-kube-api-access-kvztv\") pod \"maas-api-e2e-ait-9e0aa07d-7588b756c-f529v\" (UID: \"f995898a-20c5-4271-9ed8-c5d5c36b9a9a\") " pod="opendatahub/maas-api-e2e-ait-9e0aa07d-7588b756c-f529v" Jul 07 14:53:40.057299 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:53:40.057262 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:53:40.288075 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:53:40.287995 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:53:40.376761 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:53:40.376736 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f995898a-20c5-4271-9ed8-c5d5c36b9a9a-maas-api-tls\") pod \"maas-api-e2e-ait-9e0aa07d-7588b756c-f529v\" (UID: \"f995898a-20c5-4271-9ed8-c5d5c36b9a9a\") " pod="opendatahub/maas-api-e2e-ait-9e0aa07d-7588b756c-f529v" Jul 07 14:53:40.379083 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:53:40.379061 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f995898a-20c5-4271-9ed8-c5d5c36b9a9a-maas-api-tls\") pod \"maas-api-e2e-ait-9e0aa07d-7588b756c-f529v\" (UID: \"f995898a-20c5-4271-9ed8-c5d5c36b9a9a\") " pod="opendatahub/maas-api-e2e-ait-9e0aa07d-7588b756c-f529v" Jul 07 14:53:40.386082 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:53:40.386056 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:53:40.566263 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:53:40.566180 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-9e0aa07d-7588b756c-f529v" Jul 07 14:53:40.713783 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:53:40.713751 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-ait-9e0aa07d-7588b756c-f529v"] Jul 07 14:53:40.715099 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:53:40.715076 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf995898a_20c5_4271_9ed8_c5d5c36b9a9a.slice/crio-46ff42782e53e1e16b596db0d410c08b278388d1bda9e2a0ae89c512d9aa2a35 WatchSource:0}: Error finding container 46ff42782e53e1e16b596db0d410c08b278388d1bda9e2a0ae89c512d9aa2a35: Status 404 returned error can't find the container with id 46ff42782e53e1e16b596db0d410c08b278388d1bda9e2a0ae89c512d9aa2a35 Jul 07 14:53:40.716454 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:53:40.716439 2557 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 07 14:53:40.764856 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:53:40.764824 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-9e0aa07d-7588b756c-f529v" event={"ID":"f995898a-20c5-4271-9ed8-c5d5c36b9a9a","Type":"ContainerStarted","Data":"46ff42782e53e1e16b596db0d410c08b278388d1bda9e2a0ae89c512d9aa2a35"} Jul 07 14:53:41.772107 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:53:41.772072 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-9e0aa07d-7588b756c-f529v" event={"ID":"f995898a-20c5-4271-9ed8-c5d5c36b9a9a","Type":"ContainerStarted","Data":"c69ccbdc86166982a92e66b2f81d3f19997b1dea5810a68f0f46f35e1797a6f5"} Jul 07 14:53:41.772471 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:53:41.772172 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-ait-9e0aa07d-7588b756c-f529v" Jul 07 14:53:41.792396 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:53:41.792349 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-ait-9e0aa07d-7588b756c-f529v" podStartSLOduration=2.425982695 podStartE2EDuration="2.792337061s" totalImagesPullingTime="366.354366ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 14:53:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:53:41.790825404 +0000 UTC m=+1658.499963783" watchObservedRunningTime="2026-07-07 14:53:41.792337061 +0000 UTC m=+1658.501475441" Jul 07 14:53:42.728974 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:53:42.728939 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:53:44.490856 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:53:44.490824 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:53:44.594327 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:53:44.594286 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:53:44.806718 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:53:44.806608 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:53:45.018231 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:53:45.018195 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:53:45.117738 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:53:45.117706 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:53:45.292661 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:53:45.292631 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-ait-9e0aa07d-7588b756c-f529v"] Jul 07 14:53:45.292913 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:53:45.292880 2557 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-ait-9e0aa07d-7588b756c-f529v" podUID="f995898a-20c5-4271-9ed8-c5d5c36b9a9a" containerName="maas-api" containerID="cri-o://c69ccbdc86166982a92e66b2f81d3f19997b1dea5810a68f0f46f35e1797a6f5" gracePeriod=30 Jul 07 14:53:45.333921 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:53:45.333890 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:53:45.541350 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:53:45.541320 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-9e0aa07d-7588b756c-f529v" Jul 07 14:53:45.625557 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:53:45.625507 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f995898a-20c5-4271-9ed8-c5d5c36b9a9a-openshift-service-ca\") pod \"f995898a-20c5-4271-9ed8-c5d5c36b9a9a\" (UID: \"f995898a-20c5-4271-9ed8-c5d5c36b9a9a\") " Jul 07 14:53:45.625557 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:53:45.625562 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kvztv\" (UniqueName: \"kubernetes.io/projected/f995898a-20c5-4271-9ed8-c5d5c36b9a9a-kube-api-access-kvztv\") pod \"f995898a-20c5-4271-9ed8-c5d5c36b9a9a\" (UID: \"f995898a-20c5-4271-9ed8-c5d5c36b9a9a\") " Jul 07 14:53:45.625764 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:53:45.625596 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f995898a-20c5-4271-9ed8-c5d5c36b9a9a-maas-api-tls\") pod \"f995898a-20c5-4271-9ed8-c5d5c36b9a9a\" (UID: \"f995898a-20c5-4271-9ed8-c5d5c36b9a9a\") " Jul 07 14:53:45.625879 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:53:45.625857 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f995898a-20c5-4271-9ed8-c5d5c36b9a9a-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "f995898a-20c5-4271-9ed8-c5d5c36b9a9a" (UID: "f995898a-20c5-4271-9ed8-c5d5c36b9a9a"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 14:53:45.627669 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:53:45.627648 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f995898a-20c5-4271-9ed8-c5d5c36b9a9a-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "f995898a-20c5-4271-9ed8-c5d5c36b9a9a" (UID: "f995898a-20c5-4271-9ed8-c5d5c36b9a9a"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 14:53:45.627669 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:53:45.627654 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f995898a-20c5-4271-9ed8-c5d5c36b9a9a-kube-api-access-kvztv" (OuterVolumeSpecName: "kube-api-access-kvztv") pod "f995898a-20c5-4271-9ed8-c5d5c36b9a9a" (UID: "f995898a-20c5-4271-9ed8-c5d5c36b9a9a"). InnerVolumeSpecName "kube-api-access-kvztv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 14:53:45.726815 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:53:45.726713 2557 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f995898a-20c5-4271-9ed8-c5d5c36b9a9a-openshift-service-ca\") on node \"ip-10-0-134-42.ec2.internal\" DevicePath \"\"" Jul 07 14:53:45.726815 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:53:45.726756 2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kvztv\" (UniqueName: \"kubernetes.io/projected/f995898a-20c5-4271-9ed8-c5d5c36b9a9a-kube-api-access-kvztv\") on node \"ip-10-0-134-42.ec2.internal\" DevicePath \"\"" Jul 07 14:53:45.726815 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:53:45.726766 2557 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f995898a-20c5-4271-9ed8-c5d5c36b9a9a-maas-api-tls\") on node \"ip-10-0-134-42.ec2.internal\" DevicePath \"\"" Jul 07 14:53:45.796728 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:53:45.796694 2557 generic.go:356] "Generic (PLEG): container finished" podID="f995898a-20c5-4271-9ed8-c5d5c36b9a9a" containerID="c69ccbdc86166982a92e66b2f81d3f19997b1dea5810a68f0f46f35e1797a6f5" exitCode=0 Jul 07 14:53:45.796907 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:53:45.796747 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-9e0aa07d-7588b756c-f529v" Jul 07 14:53:45.796907 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:53:45.796782 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-9e0aa07d-7588b756c-f529v" event={"ID":"f995898a-20c5-4271-9ed8-c5d5c36b9a9a","Type":"ContainerDied","Data":"c69ccbdc86166982a92e66b2f81d3f19997b1dea5810a68f0f46f35e1797a6f5"} Jul 07 14:53:45.796907 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:53:45.796835 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-9e0aa07d-7588b756c-f529v" event={"ID":"f995898a-20c5-4271-9ed8-c5d5c36b9a9a","Type":"ContainerDied","Data":"46ff42782e53e1e16b596db0d410c08b278388d1bda9e2a0ae89c512d9aa2a35"} Jul 07 14:53:45.796907 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:53:45.796856 2557 scope.go:117] "RemoveContainer" containerID="c69ccbdc86166982a92e66b2f81d3f19997b1dea5810a68f0f46f35e1797a6f5" Jul 07 14:53:45.807043 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:53:45.807029 2557 scope.go:117] "RemoveContainer" containerID="c69ccbdc86166982a92e66b2f81d3f19997b1dea5810a68f0f46f35e1797a6f5" Jul 07 14:53:45.807290 ip-10-0-134-42 kubenswrapper[2557]: E0707 14:53:45.807274 2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c69ccbdc86166982a92e66b2f81d3f19997b1dea5810a68f0f46f35e1797a6f5\": container with ID starting with c69ccbdc86166982a92e66b2f81d3f19997b1dea5810a68f0f46f35e1797a6f5 not found: ID does not exist" containerID="c69ccbdc86166982a92e66b2f81d3f19997b1dea5810a68f0f46f35e1797a6f5" Jul 07 14:53:45.807342 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:53:45.807298 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c69ccbdc86166982a92e66b2f81d3f19997b1dea5810a68f0f46f35e1797a6f5"} err="failed to get container status \"c69ccbdc86166982a92e66b2f81d3f19997b1dea5810a68f0f46f35e1797a6f5\": rpc error: code = NotFound desc = could not find container \"c69ccbdc86166982a92e66b2f81d3f19997b1dea5810a68f0f46f35e1797a6f5\": container with ID starting with c69ccbdc86166982a92e66b2f81d3f19997b1dea5810a68f0f46f35e1797a6f5 not found: ID does not exist" Jul 07 14:53:45.818809 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:53:45.818787 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-ait-9e0aa07d-7588b756c-f529v"] Jul 07 14:53:45.822560 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:53:45.822538 2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-ait-9e0aa07d-7588b756c-f529v"] Jul 07 14:53:45.836028 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:53:45.836007 2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f995898a-20c5-4271-9ed8-c5d5c36b9a9a" path="/var/lib/kubelet/pods/f995898a-20c5-4271-9ed8-c5d5c36b9a9a/volumes" Jul 07 14:53:46.906452 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:53:46.906422 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:53:51.493026 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:53:51.492993 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:53:51.569106 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:53:51.569073 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:53:51.723040 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:53:51.723014 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:53:51.824357 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:53:51.824319 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:53:52.851145 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:53:52.851111 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:53:52.897447 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:53:52.897418 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-ait-4198bef9-6984845d96-lw6kh"] Jul 07 14:53:52.898063 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:53:52.898048 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f995898a-20c5-4271-9ed8-c5d5c36b9a9a" containerName="maas-api" Jul 07 14:53:52.898124 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:53:52.898067 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="f995898a-20c5-4271-9ed8-c5d5c36b9a9a" containerName="maas-api" Jul 07 14:53:52.898186 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:53:52.898175 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="f995898a-20c5-4271-9ed8-c5d5c36b9a9a" containerName="maas-api" Jul 07 14:53:52.902954 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:53:52.902934 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-4198bef9-6984845d96-lw6kh" Jul 07 14:53:52.905574 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:53:52.905553 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-ait-4198bef9-serving-cert\"" Jul 07 14:53:52.913968 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:53:52.913947 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-ait-4198bef9-6984845d96-lw6kh"] Jul 07 14:53:52.944560 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:53:52.944533 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:53:52.988355 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:53:52.988330 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/884c0921-7b8e-487b-8b5f-f94af3f885d5-maas-api-tls\") pod \"maas-api-e2e-ait-4198bef9-6984845d96-lw6kh\" (UID: \"884c0921-7b8e-487b-8b5f-f94af3f885d5\") " pod="opendatahub/maas-api-e2e-ait-4198bef9-6984845d96-lw6kh" Jul 07 14:53:52.988457 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:53:52.988369 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vc8qg\" (UniqueName: \"kubernetes.io/projected/884c0921-7b8e-487b-8b5f-f94af3f885d5-kube-api-access-vc8qg\") pod \"maas-api-e2e-ait-4198bef9-6984845d96-lw6kh\" (UID: \"884c0921-7b8e-487b-8b5f-f94af3f885d5\") " pod="opendatahub/maas-api-e2e-ait-4198bef9-6984845d96-lw6kh" Jul 07 14:53:52.988457 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:53:52.988411 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/884c0921-7b8e-487b-8b5f-f94af3f885d5-openshift-service-ca\") pod \"maas-api-e2e-ait-4198bef9-6984845d96-lw6kh\" (UID: \"884c0921-7b8e-487b-8b5f-f94af3f885d5\") " pod="opendatahub/maas-api-e2e-ait-4198bef9-6984845d96-lw6kh" Jul 07 14:53:53.089395 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:53:53.089367 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/884c0921-7b8e-487b-8b5f-f94af3f885d5-openshift-service-ca\") pod \"maas-api-e2e-ait-4198bef9-6984845d96-lw6kh\" (UID: \"884c0921-7b8e-487b-8b5f-f94af3f885d5\") " pod="opendatahub/maas-api-e2e-ait-4198bef9-6984845d96-lw6kh" Jul 07 14:53:53.089562 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:53:53.089481 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/884c0921-7b8e-487b-8b5f-f94af3f885d5-maas-api-tls\") pod \"maas-api-e2e-ait-4198bef9-6984845d96-lw6kh\" (UID: \"884c0921-7b8e-487b-8b5f-f94af3f885d5\") " pod="opendatahub/maas-api-e2e-ait-4198bef9-6984845d96-lw6kh" Jul 07 14:53:53.089609 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:53:53.089565 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vc8qg\" (UniqueName: \"kubernetes.io/projected/884c0921-7b8e-487b-8b5f-f94af3f885d5-kube-api-access-vc8qg\") pod \"maas-api-e2e-ait-4198bef9-6984845d96-lw6kh\" (UID: \"884c0921-7b8e-487b-8b5f-f94af3f885d5\") " pod="opendatahub/maas-api-e2e-ait-4198bef9-6984845d96-lw6kh" Jul 07 14:53:53.090006 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:53:53.089984 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/884c0921-7b8e-487b-8b5f-f94af3f885d5-openshift-service-ca\") pod \"maas-api-e2e-ait-4198bef9-6984845d96-lw6kh\" (UID: \"884c0921-7b8e-487b-8b5f-f94af3f885d5\") " pod="opendatahub/maas-api-e2e-ait-4198bef9-6984845d96-lw6kh" Jul 07 14:53:53.091649 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:53:53.091626 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/884c0921-7b8e-487b-8b5f-f94af3f885d5-maas-api-tls\") pod \"maas-api-e2e-ait-4198bef9-6984845d96-lw6kh\" (UID: \"884c0921-7b8e-487b-8b5f-f94af3f885d5\") " pod="opendatahub/maas-api-e2e-ait-4198bef9-6984845d96-lw6kh" Jul 07 14:53:53.098843 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:53:53.098820 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vc8qg\" (UniqueName: \"kubernetes.io/projected/884c0921-7b8e-487b-8b5f-f94af3f885d5-kube-api-access-vc8qg\") pod \"maas-api-e2e-ait-4198bef9-6984845d96-lw6kh\" (UID: \"884c0921-7b8e-487b-8b5f-f94af3f885d5\") " pod="opendatahub/maas-api-e2e-ait-4198bef9-6984845d96-lw6kh" Jul 07 14:53:53.117407 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:53:53.117345 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:53:53.214024 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:53:53.213992 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-4198bef9-6984845d96-lw6kh" Jul 07 14:53:53.348145 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:53:53.348119 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-ait-4198bef9-6984845d96-lw6kh"] Jul 07 14:53:53.350179 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:53:53.350151 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod884c0921_7b8e_487b_8b5f_f94af3f885d5.slice/crio-bdd9e2cc9ae9d1593feab1ff5c4ee1bb67f1732b53151d5617af1b4215796552 WatchSource:0}: Error finding container bdd9e2cc9ae9d1593feab1ff5c4ee1bb67f1732b53151d5617af1b4215796552: Status 404 returned error can't find the container with id bdd9e2cc9ae9d1593feab1ff5c4ee1bb67f1732b53151d5617af1b4215796552 Jul 07 14:53:53.489099 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:53:53.489024 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:53:53.585032 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:53:53.585008 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:53:53.856858 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:53:53.856817 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-4198bef9-6984845d96-lw6kh" event={"ID":"884c0921-7b8e-487b-8b5f-f94af3f885d5","Type":"ContainerStarted","Data":"509a81070d0612e37d6538f0b30e59e2f918aa28b85675ef1cc947a4adbc9317"} Jul 07 14:53:53.857224 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:53:53.856882 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-4198bef9-6984845d96-lw6kh" event={"ID":"884c0921-7b8e-487b-8b5f-f94af3f885d5","Type":"ContainerStarted","Data":"bdd9e2cc9ae9d1593feab1ff5c4ee1bb67f1732b53151d5617af1b4215796552"} Jul 07 14:53:53.857224 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:53:53.856911 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-ait-4198bef9-6984845d96-lw6kh" Jul 07 14:53:53.873680 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:53:53.873632 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-ait-4198bef9-6984845d96-lw6kh" podStartSLOduration=1.5085531890000001 podStartE2EDuration="1.873612835s" totalImagesPullingTime="365.059646ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 14:53:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:53:53.87312024 +0000 UTC m=+1670.582258623" watchObservedRunningTime="2026-07-07 14:53:53.873612835 +0000 UTC m=+1670.582751215" Jul 07 14:53:55.415900 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:53:55.415866 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:53:58.102029 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:53:58.101991 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:53:58.457212 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:53:58.457136 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-ait-4198bef9-6984845d96-lw6kh"] Jul 07 14:53:58.457371 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:53:58.457343 2557 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-ait-4198bef9-6984845d96-lw6kh" podUID="884c0921-7b8e-487b-8b5f-f94af3f885d5" containerName="maas-api" containerID="cri-o://509a81070d0612e37d6538f0b30e59e2f918aa28b85675ef1cc947a4adbc9317" gracePeriod=30 Jul 07 14:53:58.462966 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:53:58.462946 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-ait-4198bef9-6984845d96-lw6kh" Jul 07 14:53:58.482436 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:53:58.482394 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:53:58.704071 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:53:58.704046 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-4198bef9-6984845d96-lw6kh" Jul 07 14:53:58.842587 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:53:58.842561 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/884c0921-7b8e-487b-8b5f-f94af3f885d5-openshift-service-ca\") pod \"884c0921-7b8e-487b-8b5f-f94af3f885d5\" (UID: \"884c0921-7b8e-487b-8b5f-f94af3f885d5\") " Jul 07 14:53:58.842717 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:53:58.842619 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vc8qg\" (UniqueName: \"kubernetes.io/projected/884c0921-7b8e-487b-8b5f-f94af3f885d5-kube-api-access-vc8qg\") pod \"884c0921-7b8e-487b-8b5f-f94af3f885d5\" (UID: \"884c0921-7b8e-487b-8b5f-f94af3f885d5\") " Jul 07 14:53:58.842717 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:53:58.842705 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/884c0921-7b8e-487b-8b5f-f94af3f885d5-maas-api-tls\") pod \"884c0921-7b8e-487b-8b5f-f94af3f885d5\" (UID: \"884c0921-7b8e-487b-8b5f-f94af3f885d5\") " Jul 07 14:53:58.842904 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:53:58.842885 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/884c0921-7b8e-487b-8b5f-f94af3f885d5-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "884c0921-7b8e-487b-8b5f-f94af3f885d5" (UID: "884c0921-7b8e-487b-8b5f-f94af3f885d5"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 14:53:58.844580 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:53:58.844558 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/884c0921-7b8e-487b-8b5f-f94af3f885d5-kube-api-access-vc8qg" (OuterVolumeSpecName: "kube-api-access-vc8qg") pod "884c0921-7b8e-487b-8b5f-f94af3f885d5" (UID: "884c0921-7b8e-487b-8b5f-f94af3f885d5"). InnerVolumeSpecName "kube-api-access-vc8qg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 14:53:58.844687 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:53:58.844586 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/884c0921-7b8e-487b-8b5f-f94af3f885d5-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "884c0921-7b8e-487b-8b5f-f94af3f885d5" (UID: "884c0921-7b8e-487b-8b5f-f94af3f885d5"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 14:53:58.886296 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:53:58.886269 2557 generic.go:356] "Generic (PLEG): container finished" podID="884c0921-7b8e-487b-8b5f-f94af3f885d5" containerID="509a81070d0612e37d6538f0b30e59e2f918aa28b85675ef1cc947a4adbc9317" exitCode=0 Jul 07 14:53:58.886407 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:53:58.886309 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-4198bef9-6984845d96-lw6kh" Jul 07 14:53:58.886407 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:53:58.886358 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-4198bef9-6984845d96-lw6kh" event={"ID":"884c0921-7b8e-487b-8b5f-f94af3f885d5","Type":"ContainerDied","Data":"509a81070d0612e37d6538f0b30e59e2f918aa28b85675ef1cc947a4adbc9317"} Jul 07 14:53:58.886407 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:53:58.886406 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-4198bef9-6984845d96-lw6kh" event={"ID":"884c0921-7b8e-487b-8b5f-f94af3f885d5","Type":"ContainerDied","Data":"bdd9e2cc9ae9d1593feab1ff5c4ee1bb67f1732b53151d5617af1b4215796552"} Jul 07 14:53:58.886542 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:53:58.886429 2557 scope.go:117] "RemoveContainer" containerID="509a81070d0612e37d6538f0b30e59e2f918aa28b85675ef1cc947a4adbc9317" Jul 07 14:53:58.896667 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:53:58.896653 2557 scope.go:117] "RemoveContainer" containerID="509a81070d0612e37d6538f0b30e59e2f918aa28b85675ef1cc947a4adbc9317" Jul 07 14:53:58.896928 ip-10-0-134-42 kubenswrapper[2557]: E0707 14:53:58.896911 2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"509a81070d0612e37d6538f0b30e59e2f918aa28b85675ef1cc947a4adbc9317\": container with ID starting with 509a81070d0612e37d6538f0b30e59e2f918aa28b85675ef1cc947a4adbc9317 not found: ID does not exist" containerID="509a81070d0612e37d6538f0b30e59e2f918aa28b85675ef1cc947a4adbc9317" Jul 07 14:53:58.896977 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:53:58.896937 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"509a81070d0612e37d6538f0b30e59e2f918aa28b85675ef1cc947a4adbc9317"} err="failed to get container status \"509a81070d0612e37d6538f0b30e59e2f918aa28b85675ef1cc947a4adbc9317\": rpc error: code = NotFound desc = could not find container \"509a81070d0612e37d6538f0b30e59e2f918aa28b85675ef1cc947a4adbc9317\": container with ID starting with 509a81070d0612e37d6538f0b30e59e2f918aa28b85675ef1cc947a4adbc9317 not found: ID does not exist" Jul 07 14:53:58.911540 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:53:58.908685 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-ait-4198bef9-6984845d96-lw6kh"] Jul 07 14:53:58.915711 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:53:58.915689 2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-ait-4198bef9-6984845d96-lw6kh"] Jul 07 14:53:58.943405 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:53:58.943367 2557 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/884c0921-7b8e-487b-8b5f-f94af3f885d5-maas-api-tls\") on node \"ip-10-0-134-42.ec2.internal\" DevicePath \"\"" Jul 07 14:53:58.943405 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:53:58.943405 2557 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/884c0921-7b8e-487b-8b5f-f94af3f885d5-openshift-service-ca\") on node \"ip-10-0-134-42.ec2.internal\" DevicePath \"\"" Jul 07 14:53:58.943557 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:53:58.943416 2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vc8qg\" (UniqueName: \"kubernetes.io/projected/884c0921-7b8e-487b-8b5f-f94af3f885d5-kube-api-access-vc8qg\") on node \"ip-10-0-134-42.ec2.internal\" DevicePath \"\"" Jul 07 14:53:59.836644 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:53:59.836612 2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="884c0921-7b8e-487b-8b5f-f94af3f885d5" path="/var/lib/kubelet/pods/884c0921-7b8e-487b-8b5f-f94af3f885d5/volumes" Jul 07 14:54:04.434628 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:54:04.434594 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:54:04.555993 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:54:04.555954 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:54:11.026351 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:54:11.026314 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:54:11.126720 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:54:11.126672 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:54:11.313882 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:54:11.313792 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:54:12.459367 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:54:12.459314 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:54:12.533910 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:54:12.533882 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-derive-35928690-6755b5d677-x72fw"] Jul 07 14:54:12.534290 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:54:12.534278 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="884c0921-7b8e-487b-8b5f-f94af3f885d5" containerName="maas-api" Jul 07 14:54:12.534344 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:54:12.534292 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="884c0921-7b8e-487b-8b5f-f94af3f885d5" containerName="maas-api" Jul 07 14:54:12.534383 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:54:12.534352 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="884c0921-7b8e-487b-8b5f-f94af3f885d5" containerName="maas-api" Jul 07 14:54:12.538903 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:54:12.538882 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-derive-35928690-6755b5d677-x72fw" Jul 07 14:54:12.540792 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:54:12.540771 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-derive-35928690-serving-cert\"" Jul 07 14:54:12.548687 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:54:12.548667 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-derive-35928690-6755b5d677-x72fw"] Jul 07 14:54:12.550270 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:54:12.550243 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:54:12.640532 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:54:12.640494 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:54:12.658269 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:54:12.658228 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b1bd256a-af52-4478-9caf-6f52d6e90811-openshift-service-ca\") pod \"maas-api-e2e-derive-35928690-6755b5d677-x72fw\" (UID: \"b1bd256a-af52-4478-9caf-6f52d6e90811\") " pod="opendatahub/maas-api-e2e-derive-35928690-6755b5d677-x72fw" Jul 07 14:54:12.658392 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:54:12.658293 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p4lsr\" (UniqueName: \"kubernetes.io/projected/b1bd256a-af52-4478-9caf-6f52d6e90811-kube-api-access-p4lsr\") pod \"maas-api-e2e-derive-35928690-6755b5d677-x72fw\" (UID: \"b1bd256a-af52-4478-9caf-6f52d6e90811\") " pod="opendatahub/maas-api-e2e-derive-35928690-6755b5d677-x72fw" Jul 07 14:54:12.658392 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:54:12.658357 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b1bd256a-af52-4478-9caf-6f52d6e90811-maas-api-tls\") pod \"maas-api-e2e-derive-35928690-6755b5d677-x72fw\" (UID: \"b1bd256a-af52-4478-9caf-6f52d6e90811\") " pod="opendatahub/maas-api-e2e-derive-35928690-6755b5d677-x72fw" Jul 07 14:54:12.759751 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:54:12.759690 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b1bd256a-af52-4478-9caf-6f52d6e90811-maas-api-tls\") pod \"maas-api-e2e-derive-35928690-6755b5d677-x72fw\" (UID: \"b1bd256a-af52-4478-9caf-6f52d6e90811\") " pod="opendatahub/maas-api-e2e-derive-35928690-6755b5d677-x72fw" Jul 07 14:54:12.759873 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:54:12.759761 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b1bd256a-af52-4478-9caf-6f52d6e90811-openshift-service-ca\") pod \"maas-api-e2e-derive-35928690-6755b5d677-x72fw\" (UID: \"b1bd256a-af52-4478-9caf-6f52d6e90811\") " pod="opendatahub/maas-api-e2e-derive-35928690-6755b5d677-x72fw" Jul 07 14:54:12.759873 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:54:12.759792 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p4lsr\" (UniqueName: \"kubernetes.io/projected/b1bd256a-af52-4478-9caf-6f52d6e90811-kube-api-access-p4lsr\") pod \"maas-api-e2e-derive-35928690-6755b5d677-x72fw\" (UID: \"b1bd256a-af52-4478-9caf-6f52d6e90811\") " pod="opendatahub/maas-api-e2e-derive-35928690-6755b5d677-x72fw" Jul 07 14:54:12.759873 ip-10-0-134-42 kubenswrapper[2557]: E0707 14:54:12.759833 2557 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-derive-35928690-serving-cert: secret "maas-api-e2e-derive-35928690-serving-cert" not found Jul 07 14:54:12.760043 ip-10-0-134-42 kubenswrapper[2557]: E0707 14:54:12.759900 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b1bd256a-af52-4478-9caf-6f52d6e90811-maas-api-tls podName:b1bd256a-af52-4478-9caf-6f52d6e90811 nodeName:}" failed. No retries permitted until 2026-07-07 14:54:13.259885352 +0000 UTC m=+1689.969023710 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/b1bd256a-af52-4478-9caf-6f52d6e90811-maas-api-tls") pod "maas-api-e2e-derive-35928690-6755b5d677-x72fw" (UID: "b1bd256a-af52-4478-9caf-6f52d6e90811") : secret "maas-api-e2e-derive-35928690-serving-cert" not found Jul 07 14:54:12.760443 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:54:12.760422 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b1bd256a-af52-4478-9caf-6f52d6e90811-openshift-service-ca\") pod \"maas-api-e2e-derive-35928690-6755b5d677-x72fw\" (UID: \"b1bd256a-af52-4478-9caf-6f52d6e90811\") " pod="opendatahub/maas-api-e2e-derive-35928690-6755b5d677-x72fw" Jul 07 14:54:12.770868 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:54:12.770848 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p4lsr\" (UniqueName: \"kubernetes.io/projected/b1bd256a-af52-4478-9caf-6f52d6e90811-kube-api-access-p4lsr\") pod \"maas-api-e2e-derive-35928690-6755b5d677-x72fw\" (UID: \"b1bd256a-af52-4478-9caf-6f52d6e90811\") " pod="opendatahub/maas-api-e2e-derive-35928690-6755b5d677-x72fw" Jul 07 14:54:13.010358 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:54:13.010260 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:54:13.109752 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:54:13.109722 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:54:13.240895 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:54:13.240863 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:54:13.265099 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:54:13.265024 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b1bd256a-af52-4478-9caf-6f52d6e90811-maas-api-tls\") pod \"maas-api-e2e-derive-35928690-6755b5d677-x72fw\" (UID: \"b1bd256a-af52-4478-9caf-6f52d6e90811\") " pod="opendatahub/maas-api-e2e-derive-35928690-6755b5d677-x72fw" Jul 07 14:54:13.267373 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:54:13.267350 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b1bd256a-af52-4478-9caf-6f52d6e90811-maas-api-tls\") pod \"maas-api-e2e-derive-35928690-6755b5d677-x72fw\" (UID: \"b1bd256a-af52-4478-9caf-6f52d6e90811\") " pod="opendatahub/maas-api-e2e-derive-35928690-6755b5d677-x72fw" Jul 07 14:54:13.451990 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:54:13.451950 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-derive-35928690-6755b5d677-x72fw" Jul 07 14:54:13.589313 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:54:13.589290 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-derive-35928690-6755b5d677-x72fw"] Jul 07 14:54:13.590883 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:54:13.590857 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb1bd256a_af52_4478_9caf_6f52d6e90811.slice/crio-949abe15ff1793cafc0b32b2da30888263acb90c6bf8898ff17dbb5a76139064 WatchSource:0}: Error finding container 949abe15ff1793cafc0b32b2da30888263acb90c6bf8898ff17dbb5a76139064: Status 404 returned error can't find the container with id 949abe15ff1793cafc0b32b2da30888263acb90c6bf8898ff17dbb5a76139064 Jul 07 14:54:13.976128 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:54:13.976093 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-derive-35928690-6755b5d677-x72fw" event={"ID":"b1bd256a-af52-4478-9caf-6f52d6e90811","Type":"ContainerStarted","Data":"949abe15ff1793cafc0b32b2da30888263acb90c6bf8898ff17dbb5a76139064"} Jul 07 14:54:14.982803 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:54:14.982765 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-derive-35928690-6755b5d677-x72fw" event={"ID":"b1bd256a-af52-4478-9caf-6f52d6e90811","Type":"ContainerStarted","Data":"dbbc0de9098a63db53f19fffedb2dd7c3439347ce26355ac132060d3f18f6dfc"} Jul 07 14:54:14.983181 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:54:14.982832 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-derive-35928690-6755b5d677-x72fw" Jul 07 14:54:15.002973 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:54:15.002929 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-derive-35928690-6755b5d677-x72fw" podStartSLOduration=2.52269313 podStartE2EDuration="3.002915608s" totalImagesPullingTime="480.222478ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 14:54:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:54:15.000495217 +0000 UTC m=+1691.709633630" watchObservedRunningTime="2026-07-07 14:54:15.002915608 +0000 UTC m=+1691.712053988" Jul 07 14:54:16.252990 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:54:16.252960 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:54:16.380969 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:54:16.380935 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:54:17.046066 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:54:17.046034 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:54:17.320086 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:54:17.319988 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:54:17.426009 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:54:17.425974 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:54:17.570470 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:54:17.570399 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-derive-35928690-6755b5d677-x72fw"] Jul 07 14:54:17.570627 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:54:17.570585 2557 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-derive-35928690-6755b5d677-x72fw" podUID="b1bd256a-af52-4478-9caf-6f52d6e90811" containerName="maas-api" containerID="cri-o://dbbc0de9098a63db53f19fffedb2dd7c3439347ce26355ac132060d3f18f6dfc" gracePeriod=30 Jul 07 14:54:17.615549 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:54:17.615505 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:54:17.815841 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:54:17.815817 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-derive-35928690-6755b5d677-x72fw" Jul 07 14:54:17.901556 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:54:17.901474 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p4lsr\" (UniqueName: \"kubernetes.io/projected/b1bd256a-af52-4478-9caf-6f52d6e90811-kube-api-access-p4lsr\") pod \"b1bd256a-af52-4478-9caf-6f52d6e90811\" (UID: \"b1bd256a-af52-4478-9caf-6f52d6e90811\") " Jul 07 14:54:17.903365 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:54:17.903342 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b1bd256a-af52-4478-9caf-6f52d6e90811-kube-api-access-p4lsr" (OuterVolumeSpecName: "kube-api-access-p4lsr") pod "b1bd256a-af52-4478-9caf-6f52d6e90811" (UID: "b1bd256a-af52-4478-9caf-6f52d6e90811"). InnerVolumeSpecName "kube-api-access-p4lsr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 14:54:18.000976 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:54:18.000950 2557 generic.go:356] "Generic (PLEG): container finished" podID="b1bd256a-af52-4478-9caf-6f52d6e90811" containerID="dbbc0de9098a63db53f19fffedb2dd7c3439347ce26355ac132060d3f18f6dfc" exitCode=0 Jul 07 14:54:18.001085 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:54:18.000982 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-derive-35928690-6755b5d677-x72fw" event={"ID":"b1bd256a-af52-4478-9caf-6f52d6e90811","Type":"ContainerDied","Data":"dbbc0de9098a63db53f19fffedb2dd7c3439347ce26355ac132060d3f18f6dfc"} Jul 07 14:54:18.001085 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:54:18.001045 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-derive-35928690-6755b5d677-x72fw" event={"ID":"b1bd256a-af52-4478-9caf-6f52d6e90811","Type":"ContainerDied","Data":"949abe15ff1793cafc0b32b2da30888263acb90c6bf8898ff17dbb5a76139064"} Jul 07 14:54:18.001085 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:54:18.000994 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-derive-35928690-6755b5d677-x72fw" Jul 07 14:54:18.001197 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:54:18.001080 2557 scope.go:117] "RemoveContainer" containerID="dbbc0de9098a63db53f19fffedb2dd7c3439347ce26355ac132060d3f18f6dfc" Jul 07 14:54:18.001958 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:54:18.001937 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b1bd256a-af52-4478-9caf-6f52d6e90811-maas-api-tls\") pod \"b1bd256a-af52-4478-9caf-6f52d6e90811\" (UID: \"b1bd256a-af52-4478-9caf-6f52d6e90811\") " Jul 07 14:54:18.002050 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:54:18.001977 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b1bd256a-af52-4478-9caf-6f52d6e90811-openshift-service-ca\") pod \"b1bd256a-af52-4478-9caf-6f52d6e90811\" (UID: \"b1bd256a-af52-4478-9caf-6f52d6e90811\") " Jul 07 14:54:18.002267 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:54:18.002249 2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p4lsr\" (UniqueName: \"kubernetes.io/projected/b1bd256a-af52-4478-9caf-6f52d6e90811-kube-api-access-p4lsr\") on node \"ip-10-0-134-42.ec2.internal\" DevicePath \"\"" Jul 07 14:54:18.002385 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:54:18.002359 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b1bd256a-af52-4478-9caf-6f52d6e90811-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "b1bd256a-af52-4478-9caf-6f52d6e90811" (UID: "b1bd256a-af52-4478-9caf-6f52d6e90811"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 14:54:18.003876 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:54:18.003853 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b1bd256a-af52-4478-9caf-6f52d6e90811-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "b1bd256a-af52-4478-9caf-6f52d6e90811" (UID: "b1bd256a-af52-4478-9caf-6f52d6e90811"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 14:54:18.015173 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:54:18.015156 2557 scope.go:117] "RemoveContainer" containerID="dbbc0de9098a63db53f19fffedb2dd7c3439347ce26355ac132060d3f18f6dfc" Jul 07 14:54:18.015419 ip-10-0-134-42 kubenswrapper[2557]: E0707 14:54:18.015390 2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dbbc0de9098a63db53f19fffedb2dd7c3439347ce26355ac132060d3f18f6dfc\": container with ID starting with dbbc0de9098a63db53f19fffedb2dd7c3439347ce26355ac132060d3f18f6dfc not found: ID does not exist" containerID="dbbc0de9098a63db53f19fffedb2dd7c3439347ce26355ac132060d3f18f6dfc" Jul 07 14:54:18.015472 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:54:18.015431 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dbbc0de9098a63db53f19fffedb2dd7c3439347ce26355ac132060d3f18f6dfc"} err="failed to get container status \"dbbc0de9098a63db53f19fffedb2dd7c3439347ce26355ac132060d3f18f6dfc\": rpc error: code = NotFound desc = could not find container \"dbbc0de9098a63db53f19fffedb2dd7c3439347ce26355ac132060d3f18f6dfc\": container with ID starting with dbbc0de9098a63db53f19fffedb2dd7c3439347ce26355ac132060d3f18f6dfc not found: ID does not exist" Jul 07 14:54:18.102617 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:54:18.102597 2557 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b1bd256a-af52-4478-9caf-6f52d6e90811-maas-api-tls\") on node \"ip-10-0-134-42.ec2.internal\" DevicePath \"\"" Jul 07 14:54:18.102690 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:54:18.102621 2557 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b1bd256a-af52-4478-9caf-6f52d6e90811-openshift-service-ca\") on node \"ip-10-0-134-42.ec2.internal\" DevicePath \"\"" Jul 07 14:54:18.326736 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:54:18.326706 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-derive-35928690-6755b5d677-x72fw"] Jul 07 14:54:18.330811 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:54:18.330788 2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-derive-35928690-6755b5d677-x72fw"] Jul 07 14:54:19.842713 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:54:19.842681 2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b1bd256a-af52-4478-9caf-6f52d6e90811" path="/var/lib/kubelet/pods/b1bd256a-af52-4478-9caf-6f52d6e90811/volumes" Jul 07 14:54:20.855867 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:54:20.855829 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:54:23.049552 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:54:23.049503 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:54:23.820805 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:54:23.820769 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:54:23.912780 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:54:23.912750 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:54:24.438673 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:54:24.438637 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:54:24.529592 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:54:24.529562 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:54:26.209914 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:54:26.209837 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:54:32.340146 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:54:32.340112 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:54:32.519685 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:54:32.519644 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:54:32.647682 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:54:32.647599 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:54:33.205008 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:54:33.204963 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:54:59.227935 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:54:59.227874 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:55:05.284118 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:55:05.284078 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:55:05.385921 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:55:05.385882 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:55:21.331458 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:55:21.331403 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:55:27.448136 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:55:27.448100 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:55:27.561990 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:55:27.561956 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:55:27.690811 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:55:27.690775 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:55:38.174529 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:55:38.174477 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:55:38.995590 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:55:38.995552 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:55:45.047241 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:55:45.047195 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:55:45.116845 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:55:45.116812 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:55:45.201918 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:55:45.201881 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:55:45.334639 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:55:45.334610 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:55:45.462376 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:55:45.462352 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:55:46.204902 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:55:46.204855 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:55:46.302727 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:55:46.302696 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:55:46.403183 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:55:46.403152 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:55:48.010491 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:55:48.010458 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:55:54.238665 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:55:54.238634 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:55:54.310328 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:55:54.310295 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:55:54.939557 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:55:54.939496 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:56:01.076146 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:56:01.076062 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:56:01.179428 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:56:01.179393 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:56:01.312475 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:56:01.312433 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:56:01.694703 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:56:01.694663 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:56:01.828907 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:56:01.828867 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:56:04.561877 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:56:04.561834 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:56:04.623534 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:56:04.623481 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-mt-8dea50b1-6cdb997bd5-k52sl"] Jul 07 14:56:04.624158 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:56:04.624131 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b1bd256a-af52-4478-9caf-6f52d6e90811" containerName="maas-api" Jul 07 14:56:04.624158 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:56:04.624157 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="b1bd256a-af52-4478-9caf-6f52d6e90811" containerName="maas-api" Jul 07 14:56:04.624379 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:56:04.624258 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="b1bd256a-af52-4478-9caf-6f52d6e90811" containerName="maas-api" Jul 07 14:56:04.627997 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:56:04.627979 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-8dea50b1-6cdb997bd5-k52sl" Jul 07 14:56:04.630620 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:56:04.630597 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-mt-8dea50b1-serving-cert\"" Jul 07 14:56:04.639468 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:56:04.639444 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-mt-8dea50b1-6cdb997bd5-k52sl"] Jul 07 14:56:04.661451 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:56:04.661412 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:56:04.731882 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:56:04.731844 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ed439795-0351-495c-aca4-4d8dce748b08-maas-api-tls\") pod \"maas-api-e2e-mt-8dea50b1-6cdb997bd5-k52sl\" (UID: \"ed439795-0351-495c-aca4-4d8dce748b08\") " pod="opendatahub/maas-api-e2e-mt-8dea50b1-6cdb997bd5-k52sl" Jul 07 14:56:04.731882 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:56:04.731885 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ed439795-0351-495c-aca4-4d8dce748b08-openshift-service-ca\") pod \"maas-api-e2e-mt-8dea50b1-6cdb997bd5-k52sl\" (UID: \"ed439795-0351-495c-aca4-4d8dce748b08\") " pod="opendatahub/maas-api-e2e-mt-8dea50b1-6cdb997bd5-k52sl" Jul 07 14:56:04.732139 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:56:04.732081 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m4ljl\" (UniqueName: \"kubernetes.io/projected/ed439795-0351-495c-aca4-4d8dce748b08-kube-api-access-m4ljl\") pod \"maas-api-e2e-mt-8dea50b1-6cdb997bd5-k52sl\" (UID: \"ed439795-0351-495c-aca4-4d8dce748b08\") " pod="opendatahub/maas-api-e2e-mt-8dea50b1-6cdb997bd5-k52sl" Jul 07 14:56:04.749494 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:56:04.749458 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:56:04.832910 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:56:04.832879 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ed439795-0351-495c-aca4-4d8dce748b08-maas-api-tls\") pod \"maas-api-e2e-mt-8dea50b1-6cdb997bd5-k52sl\" (UID: \"ed439795-0351-495c-aca4-4d8dce748b08\") " pod="opendatahub/maas-api-e2e-mt-8dea50b1-6cdb997bd5-k52sl" Jul 07 14:56:04.832910 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:56:04.832912 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ed439795-0351-495c-aca4-4d8dce748b08-openshift-service-ca\") pod \"maas-api-e2e-mt-8dea50b1-6cdb997bd5-k52sl\" (UID: \"ed439795-0351-495c-aca4-4d8dce748b08\") " pod="opendatahub/maas-api-e2e-mt-8dea50b1-6cdb997bd5-k52sl" Jul 07 14:56:04.833117 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:56:04.833004 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m4ljl\" (UniqueName: \"kubernetes.io/projected/ed439795-0351-495c-aca4-4d8dce748b08-kube-api-access-m4ljl\") pod \"maas-api-e2e-mt-8dea50b1-6cdb997bd5-k52sl\" (UID: \"ed439795-0351-495c-aca4-4d8dce748b08\") " pod="opendatahub/maas-api-e2e-mt-8dea50b1-6cdb997bd5-k52sl" Jul 07 14:56:04.833624 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:56:04.833605 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ed439795-0351-495c-aca4-4d8dce748b08-openshift-service-ca\") pod \"maas-api-e2e-mt-8dea50b1-6cdb997bd5-k52sl\" (UID: \"ed439795-0351-495c-aca4-4d8dce748b08\") " pod="opendatahub/maas-api-e2e-mt-8dea50b1-6cdb997bd5-k52sl" Jul 07 14:56:04.835297 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:56:04.835271 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ed439795-0351-495c-aca4-4d8dce748b08-maas-api-tls\") pod \"maas-api-e2e-mt-8dea50b1-6cdb997bd5-k52sl\" (UID: \"ed439795-0351-495c-aca4-4d8dce748b08\") " pod="opendatahub/maas-api-e2e-mt-8dea50b1-6cdb997bd5-k52sl" Jul 07 14:56:04.843231 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:56:04.843201 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m4ljl\" (UniqueName: \"kubernetes.io/projected/ed439795-0351-495c-aca4-4d8dce748b08-kube-api-access-m4ljl\") pod \"maas-api-e2e-mt-8dea50b1-6cdb997bd5-k52sl\" (UID: \"ed439795-0351-495c-aca4-4d8dce748b08\") " pod="opendatahub/maas-api-e2e-mt-8dea50b1-6cdb997bd5-k52sl" Jul 07 14:56:04.861446 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:56:04.861421 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:56:04.938890 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:56:04.938858 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-8dea50b1-6cdb997bd5-k52sl" Jul 07 14:56:05.091469 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:56:05.091444 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-mt-8dea50b1-6cdb997bd5-k52sl"] Jul 07 14:56:05.093250 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:56:05.093223 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poded439795_0351_495c_aca4_4d8dce748b08.slice/crio-3d582d624fa6850196b6afdad71cc641768ad7cdf8669e38599a7f6fa6201fdc WatchSource:0}: Error finding container 3d582d624fa6850196b6afdad71cc641768ad7cdf8669e38599a7f6fa6201fdc: Status 404 returned error can't find the container with id 3d582d624fa6850196b6afdad71cc641768ad7cdf8669e38599a7f6fa6201fdc Jul 07 14:56:05.287653 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:56:05.287602 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:56:05.387265 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:56:05.387171 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:56:05.614204 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:56:05.614161 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-8dea50b1-6cdb997bd5-k52sl" event={"ID":"ed439795-0351-495c-aca4-4d8dce748b08","Type":"ContainerStarted","Data":"3d582d624fa6850196b6afdad71cc641768ad7cdf8669e38599a7f6fa6201fdc"} Jul 07 14:56:05.630825 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:56:05.630800 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-vd9g9_0fa80df4-c933-4758-b75f-3f70b7217c0d/console-operator/2.log" Jul 07 14:56:05.635848 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:56:05.635822 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-vd9g9_0fa80df4-c933-4758-b75f-3f70b7217c0d/console-operator/2.log" Jul 07 14:56:05.636033 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:56:05.636018 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-scfg4_90d2dce1-be42-4595-b9c4-74a024eef0e9/ovn-acl-logging/0.log" Jul 07 14:56:05.642832 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:56:05.642813 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-scfg4_90d2dce1-be42-4595-b9c4-74a024eef0e9/ovn-acl-logging/0.log" Jul 07 14:56:06.621374 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:56:06.621336 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-8dea50b1-6cdb997bd5-k52sl" event={"ID":"ed439795-0351-495c-aca4-4d8dce748b08","Type":"ContainerStarted","Data":"fb7f94ae339dfd9f50c0bf0357f01043a027b6e6c8deed8eea2f36eb25e686d1"} Jul 07 14:56:06.621873 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:56:06.621449 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-mt-8dea50b1-6cdb997bd5-k52sl" Jul 07 14:56:06.656553 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:56:06.656489 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-mt-8dea50b1-6cdb997bd5-k52sl" podStartSLOduration=2.114272581 podStartE2EDuration="2.656474578s" totalImagesPullingTime="542.201997ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 14:56:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:56:06.654762673 +0000 UTC m=+1803.363901070" watchObservedRunningTime="2026-07-07 14:56:06.656474578 +0000 UTC m=+1803.365612956" Jul 07 14:56:08.973471 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:56:08.973428 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:56:09.553028 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:56:09.552991 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:56:09.649953 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:56:09.649920 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:56:09.983157 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:56:09.983123 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:56:12.632558 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:56:12.632509 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-mt-8dea50b1-6cdb997bd5-k52sl" Jul 07 14:56:16.408125 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:56:16.408095 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:56:16.503359 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:56:16.503334 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:56:16.798867 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:56:16.798780 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:56:17.156804 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:56:17.156773 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-mt-8dea50b1-6cdb997bd5-k52sl"] Jul 07 14:56:17.156988 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:56:17.156952 2557 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-mt-8dea50b1-6cdb997bd5-k52sl" podUID="ed439795-0351-495c-aca4-4d8dce748b08" containerName="maas-api" containerID="cri-o://fb7f94ae339dfd9f50c0bf0357f01043a027b6e6c8deed8eea2f36eb25e686d1" gracePeriod=30 Jul 07 14:56:17.213540 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:56:17.213479 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:56:17.309793 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:56:17.309760 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:56:17.423962 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:56:17.423935 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-8dea50b1-6cdb997bd5-k52sl" Jul 07 14:56:17.447265 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:56:17.447242 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ed439795-0351-495c-aca4-4d8dce748b08-openshift-service-ca\") pod \"ed439795-0351-495c-aca4-4d8dce748b08\" (UID: \"ed439795-0351-495c-aca4-4d8dce748b08\") " Jul 07 14:56:17.447400 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:56:17.447306 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ed439795-0351-495c-aca4-4d8dce748b08-maas-api-tls\") pod \"ed439795-0351-495c-aca4-4d8dce748b08\" (UID: \"ed439795-0351-495c-aca4-4d8dce748b08\") " Jul 07 14:56:17.447400 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:56:17.447362 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-m4ljl\" (UniqueName: \"kubernetes.io/projected/ed439795-0351-495c-aca4-4d8dce748b08-kube-api-access-m4ljl\") pod \"ed439795-0351-495c-aca4-4d8dce748b08\" (UID: \"ed439795-0351-495c-aca4-4d8dce748b08\") " Jul 07 14:56:17.447592 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:56:17.447569 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ed439795-0351-495c-aca4-4d8dce748b08-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "ed439795-0351-495c-aca4-4d8dce748b08" (UID: "ed439795-0351-495c-aca4-4d8dce748b08"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 14:56:17.447666 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:56:17.447653 2557 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ed439795-0351-495c-aca4-4d8dce748b08-openshift-service-ca\") on node \"ip-10-0-134-42.ec2.internal\" DevicePath \"\"" Jul 07 14:56:17.449222 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:56:17.449200 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ed439795-0351-495c-aca4-4d8dce748b08-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "ed439795-0351-495c-aca4-4d8dce748b08" (UID: "ed439795-0351-495c-aca4-4d8dce748b08"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 14:56:17.449341 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:56:17.449312 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ed439795-0351-495c-aca4-4d8dce748b08-kube-api-access-m4ljl" (OuterVolumeSpecName: "kube-api-access-m4ljl") pod "ed439795-0351-495c-aca4-4d8dce748b08" (UID: "ed439795-0351-495c-aca4-4d8dce748b08"). InnerVolumeSpecName "kube-api-access-m4ljl". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 14:56:17.548042 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:56:17.548018 2557 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ed439795-0351-495c-aca4-4d8dce748b08-maas-api-tls\") on node \"ip-10-0-134-42.ec2.internal\" DevicePath \"\"" Jul 07 14:56:17.548042 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:56:17.548037 2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m4ljl\" (UniqueName: \"kubernetes.io/projected/ed439795-0351-495c-aca4-4d8dce748b08-kube-api-access-m4ljl\") on node \"ip-10-0-134-42.ec2.internal\" DevicePath \"\"" Jul 07 14:56:17.684779 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:56:17.684714 2557 generic.go:356] "Generic (PLEG): container finished" podID="ed439795-0351-495c-aca4-4d8dce748b08" containerID="fb7f94ae339dfd9f50c0bf0357f01043a027b6e6c8deed8eea2f36eb25e686d1" exitCode=0 Jul 07 14:56:17.684779 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:56:17.684762 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-8dea50b1-6cdb997bd5-k52sl" Jul 07 14:56:17.684923 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:56:17.684791 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-8dea50b1-6cdb997bd5-k52sl" event={"ID":"ed439795-0351-495c-aca4-4d8dce748b08","Type":"ContainerDied","Data":"fb7f94ae339dfd9f50c0bf0357f01043a027b6e6c8deed8eea2f36eb25e686d1"} Jul 07 14:56:17.684923 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:56:17.684868 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-8dea50b1-6cdb997bd5-k52sl" event={"ID":"ed439795-0351-495c-aca4-4d8dce748b08","Type":"ContainerDied","Data":"3d582d624fa6850196b6afdad71cc641768ad7cdf8669e38599a7f6fa6201fdc"} Jul 07 14:56:17.684923 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:56:17.684891 2557 scope.go:117] "RemoveContainer" containerID="fb7f94ae339dfd9f50c0bf0357f01043a027b6e6c8deed8eea2f36eb25e686d1" Jul 07 14:56:17.694570 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:56:17.694549 2557 scope.go:117] "RemoveContainer" containerID="fb7f94ae339dfd9f50c0bf0357f01043a027b6e6c8deed8eea2f36eb25e686d1" Jul 07 14:56:17.694875 ip-10-0-134-42 kubenswrapper[2557]: E0707 14:56:17.694847 2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fb7f94ae339dfd9f50c0bf0357f01043a027b6e6c8deed8eea2f36eb25e686d1\": container with ID starting with fb7f94ae339dfd9f50c0bf0357f01043a027b6e6c8deed8eea2f36eb25e686d1 not found: ID does not exist" containerID="fb7f94ae339dfd9f50c0bf0357f01043a027b6e6c8deed8eea2f36eb25e686d1" Jul 07 14:56:17.694956 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:56:17.694876 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fb7f94ae339dfd9f50c0bf0357f01043a027b6e6c8deed8eea2f36eb25e686d1"} err="failed to get container status \"fb7f94ae339dfd9f50c0bf0357f01043a027b6e6c8deed8eea2f36eb25e686d1\": rpc error: code = NotFound desc = could not find container \"fb7f94ae339dfd9f50c0bf0357f01043a027b6e6c8deed8eea2f36eb25e686d1\": container with ID starting with fb7f94ae339dfd9f50c0bf0357f01043a027b6e6c8deed8eea2f36eb25e686d1 not found: ID does not exist" Jul 07 14:56:17.709443 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:56:17.709419 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-mt-8dea50b1-6cdb997bd5-k52sl"] Jul 07 14:56:17.712794 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:56:17.712773 2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-mt-8dea50b1-6cdb997bd5-k52sl"] Jul 07 14:56:17.836539 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:56:17.836488 2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ed439795-0351-495c-aca4-4d8dce748b08" path="/var/lib/kubelet/pods/ed439795-0351-495c-aca4-4d8dce748b08/volumes" Jul 07 14:56:23.859223 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:56:23.859191 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:56:23.960295 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:56:23.960262 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:56:24.129590 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:56:24.129494 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:56:24.377437 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:56:24.377402 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:56:24.483259 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:56:24.483178 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:56:24.582971 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:56:24.582940 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:56:24.694463 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:56:24.694427 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:56:30.662205 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:56:30.662164 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:56:30.729892 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:56:30.729862 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:56:30.838583 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:56:30.838548 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:56:31.000216 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:56:31.000118 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:56:31.102171 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:56:31.102131 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:56:37.212388 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:56:37.212356 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:56:37.316415 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:56:37.316388 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:56:43.986791 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:56:43.986755 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:56:44.381655 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:56:44.381619 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:56:44.491466 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:56:44.491430 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:56:44.592420 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:56:44.592390 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:56:47.447009 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:56:47.446969 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:56:47.513313 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:56:47.513284 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-mt-4afb43ce-f489d75d9-sg5kv"] Jul 07 14:56:47.513946 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:56:47.513926 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ed439795-0351-495c-aca4-4d8dce748b08" containerName="maas-api" Jul 07 14:56:47.514019 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:56:47.513951 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="ed439795-0351-495c-aca4-4d8dce748b08" containerName="maas-api" Jul 07 14:56:47.514067 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:56:47.514055 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="ed439795-0351-495c-aca4-4d8dce748b08" containerName="maas-api" Jul 07 14:56:47.517407 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:56:47.517387 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-4afb43ce-f489d75d9-sg5kv" Jul 07 14:56:47.521224 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:56:47.521205 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-mt-4afb43ce-serving-cert\"" Jul 07 14:56:47.527701 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:56:47.527667 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-mt-4afb43ce-f489d75d9-sg5kv"] Jul 07 14:56:47.530218 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:56:47.530195 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:56:47.601893 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:56:47.601869 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:56:47.609230 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:56:47.609203 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/03220e17-840a-4aee-a7de-e226c06e7ec6-maas-api-tls\") pod \"maas-api-e2e-mt-4afb43ce-f489d75d9-sg5kv\" (UID: \"03220e17-840a-4aee-a7de-e226c06e7ec6\") " pod="opendatahub/maas-api-e2e-mt-4afb43ce-f489d75d9-sg5kv" Jul 07 14:56:47.609230 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:56:47.609299 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/03220e17-840a-4aee-a7de-e226c06e7ec6-openshift-service-ca\") pod \"maas-api-e2e-mt-4afb43ce-f489d75d9-sg5kv\" (UID: \"03220e17-840a-4aee-a7de-e226c06e7ec6\") " pod="opendatahub/maas-api-e2e-mt-4afb43ce-f489d75d9-sg5kv" Jul 07 14:56:47.609601 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:56:47.609349 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-79xgn\" (UniqueName: \"kubernetes.io/projected/03220e17-840a-4aee-a7de-e226c06e7ec6-kube-api-access-79xgn\") pod \"maas-api-e2e-mt-4afb43ce-f489d75d9-sg5kv\" (UID: \"03220e17-840a-4aee-a7de-e226c06e7ec6\") " pod="opendatahub/maas-api-e2e-mt-4afb43ce-f489d75d9-sg5kv" Jul 07 14:56:47.710561 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:56:47.710476 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/03220e17-840a-4aee-a7de-e226c06e7ec6-openshift-service-ca\") pod \"maas-api-e2e-mt-4afb43ce-f489d75d9-sg5kv\" (UID: \"03220e17-840a-4aee-a7de-e226c06e7ec6\") " pod="opendatahub/maas-api-e2e-mt-4afb43ce-f489d75d9-sg5kv" Jul 07 14:56:47.710561 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:56:47.710555 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-79xgn\" (UniqueName: \"kubernetes.io/projected/03220e17-840a-4aee-a7de-e226c06e7ec6-kube-api-access-79xgn\") pod \"maas-api-e2e-mt-4afb43ce-f489d75d9-sg5kv\" (UID: \"03220e17-840a-4aee-a7de-e226c06e7ec6\") " pod="opendatahub/maas-api-e2e-mt-4afb43ce-f489d75d9-sg5kv" Jul 07 14:56:47.710729 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:56:47.710693 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/03220e17-840a-4aee-a7de-e226c06e7ec6-maas-api-tls\") pod \"maas-api-e2e-mt-4afb43ce-f489d75d9-sg5kv\" (UID: \"03220e17-840a-4aee-a7de-e226c06e7ec6\") " pod="opendatahub/maas-api-e2e-mt-4afb43ce-f489d75d9-sg5kv" Jul 07 14:56:47.710821 ip-10-0-134-42 kubenswrapper[2557]: E0707 14:56:47.710805 2557 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-mt-4afb43ce-serving-cert: secret "maas-api-e2e-mt-4afb43ce-serving-cert" not found Jul 07 14:56:47.710892 ip-10-0-134-42 kubenswrapper[2557]: E0707 14:56:47.710872 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/03220e17-840a-4aee-a7de-e226c06e7ec6-maas-api-tls podName:03220e17-840a-4aee-a7de-e226c06e7ec6 nodeName:}" failed. No retries permitted until 2026-07-07 14:56:48.210852993 +0000 UTC m=+1844.919991359 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/03220e17-840a-4aee-a7de-e226c06e7ec6-maas-api-tls") pod "maas-api-e2e-mt-4afb43ce-f489d75d9-sg5kv" (UID: "03220e17-840a-4aee-a7de-e226c06e7ec6") : secret "maas-api-e2e-mt-4afb43ce-serving-cert" not found Jul 07 14:56:47.711082 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:56:47.711047 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/03220e17-840a-4aee-a7de-e226c06e7ec6-openshift-service-ca\") pod \"maas-api-e2e-mt-4afb43ce-f489d75d9-sg5kv\" (UID: \"03220e17-840a-4aee-a7de-e226c06e7ec6\") " pod="opendatahub/maas-api-e2e-mt-4afb43ce-f489d75d9-sg5kv" Jul 07 14:56:47.719854 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:56:47.719832 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-79xgn\" (UniqueName: \"kubernetes.io/projected/03220e17-840a-4aee-a7de-e226c06e7ec6-kube-api-access-79xgn\") pod \"maas-api-e2e-mt-4afb43ce-f489d75d9-sg5kv\" (UID: \"03220e17-840a-4aee-a7de-e226c06e7ec6\") " pod="opendatahub/maas-api-e2e-mt-4afb43ce-f489d75d9-sg5kv" Jul 07 14:56:47.778552 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:56:47.778528 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:56:48.076915 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:56:48.076888 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:56:48.179484 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:56:48.179449 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:56:48.215347 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:56:48.215313 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/03220e17-840a-4aee-a7de-e226c06e7ec6-maas-api-tls\") pod \"maas-api-e2e-mt-4afb43ce-f489d75d9-sg5kv\" (UID: \"03220e17-840a-4aee-a7de-e226c06e7ec6\") " pod="opendatahub/maas-api-e2e-mt-4afb43ce-f489d75d9-sg5kv" Jul 07 14:56:48.218259 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:56:48.218235 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/03220e17-840a-4aee-a7de-e226c06e7ec6-maas-api-tls\") pod \"maas-api-e2e-mt-4afb43ce-f489d75d9-sg5kv\" (UID: \"03220e17-840a-4aee-a7de-e226c06e7ec6\") " pod="opendatahub/maas-api-e2e-mt-4afb43ce-f489d75d9-sg5kv" Jul 07 14:56:48.428970 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:56:48.428887 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-4afb43ce-f489d75d9-sg5kv" Jul 07 14:56:48.569496 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:56:48.569467 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-mt-4afb43ce-f489d75d9-sg5kv"] Jul 07 14:56:48.571398 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:56:48.571362 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod03220e17_840a_4aee_a7de_e226c06e7ec6.slice/crio-80bab7e1865e67d7a39754eec9b2ac552eb37bb898ff8eef6381c1e31485d02f WatchSource:0}: Error finding container 80bab7e1865e67d7a39754eec9b2ac552eb37bb898ff8eef6381c1e31485d02f: Status 404 returned error can't find the container with id 80bab7e1865e67d7a39754eec9b2ac552eb37bb898ff8eef6381c1e31485d02f Jul 07 14:56:48.867095 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:56:48.867059 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-4afb43ce-f489d75d9-sg5kv" event={"ID":"03220e17-840a-4aee-a7de-e226c06e7ec6","Type":"ContainerStarted","Data":"80bab7e1865e67d7a39754eec9b2ac552eb37bb898ff8eef6381c1e31485d02f"} Jul 07 14:56:49.873633 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:56:49.873602 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-4afb43ce-f489d75d9-sg5kv" event={"ID":"03220e17-840a-4aee-a7de-e226c06e7ec6","Type":"ContainerStarted","Data":"1eedede00ab65e9956168e48c5dc3b25920239d560f0750a76194c768864b162"} Jul 07 14:56:49.874029 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:56:49.873728 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-mt-4afb43ce-f489d75d9-sg5kv" Jul 07 14:56:49.893649 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:56:49.893599 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-mt-4afb43ce-f489d75d9-sg5kv" podStartSLOduration=2.4769342500000002 podStartE2EDuration="2.893586493s" totalImagesPullingTime="416.652243ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 14:56:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:56:49.890972913 +0000 UTC m=+1846.600111290" watchObservedRunningTime="2026-07-07 14:56:49.893586493 +0000 UTC m=+1846.602724872" Jul 07 14:56:51.842714 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:56:51.842686 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:56:55.657692 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:56:55.657652 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:56:55.884016 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:56:55.883987 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-mt-4afb43ce-f489d75d9-sg5kv" Jul 07 14:57:14.030876 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:57:14.030843 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:57:14.498138 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:57:14.498105 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-mt-4afb43ce-f489d75d9-sg5kv"] Jul 07 14:57:14.498372 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:57:14.498346 2557 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-mt-4afb43ce-f489d75d9-sg5kv" podUID="03220e17-840a-4aee-a7de-e226c06e7ec6" containerName="maas-api" containerID="cri-o://1eedede00ab65e9956168e48c5dc3b25920239d560f0750a76194c768864b162" gracePeriod=30 Jul 07 14:57:14.524126 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:57:14.524103 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:57:14.752414 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:57:14.752360 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-4afb43ce-f489d75d9-sg5kv" Jul 07 14:57:14.845744 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:57:14.845719 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/03220e17-840a-4aee-a7de-e226c06e7ec6-maas-api-tls\") pod \"03220e17-840a-4aee-a7de-e226c06e7ec6\" (UID: \"03220e17-840a-4aee-a7de-e226c06e7ec6\") " Jul 07 14:57:14.845868 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:57:14.845779 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/03220e17-840a-4aee-a7de-e226c06e7ec6-openshift-service-ca\") pod \"03220e17-840a-4aee-a7de-e226c06e7ec6\" (UID: \"03220e17-840a-4aee-a7de-e226c06e7ec6\") " Jul 07 14:57:14.845868 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:57:14.845802 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-79xgn\" (UniqueName: \"kubernetes.io/projected/03220e17-840a-4aee-a7de-e226c06e7ec6-kube-api-access-79xgn\") pod \"03220e17-840a-4aee-a7de-e226c06e7ec6\" (UID: \"03220e17-840a-4aee-a7de-e226c06e7ec6\") " Jul 07 14:57:14.846164 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:57:14.846139 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/03220e17-840a-4aee-a7de-e226c06e7ec6-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "03220e17-840a-4aee-a7de-e226c06e7ec6" (UID: "03220e17-840a-4aee-a7de-e226c06e7ec6"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 14:57:14.847840 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:57:14.847813 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/03220e17-840a-4aee-a7de-e226c06e7ec6-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "03220e17-840a-4aee-a7de-e226c06e7ec6" (UID: "03220e17-840a-4aee-a7de-e226c06e7ec6"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 14:57:14.847941 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:57:14.847888 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/03220e17-840a-4aee-a7de-e226c06e7ec6-kube-api-access-79xgn" (OuterVolumeSpecName: "kube-api-access-79xgn") pod "03220e17-840a-4aee-a7de-e226c06e7ec6" (UID: "03220e17-840a-4aee-a7de-e226c06e7ec6"). InnerVolumeSpecName "kube-api-access-79xgn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 14:57:14.946700 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:57:14.946676 2557 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/03220e17-840a-4aee-a7de-e226c06e7ec6-maas-api-tls\") on node \"ip-10-0-134-42.ec2.internal\" DevicePath \"\"" Jul 07 14:57:14.946700 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:57:14.946698 2557 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/03220e17-840a-4aee-a7de-e226c06e7ec6-openshift-service-ca\") on node \"ip-10-0-134-42.ec2.internal\" DevicePath \"\"" Jul 07 14:57:14.946829 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:57:14.946709 2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-79xgn\" (UniqueName: \"kubernetes.io/projected/03220e17-840a-4aee-a7de-e226c06e7ec6-kube-api-access-79xgn\") on node \"ip-10-0-134-42.ec2.internal\" DevicePath \"\"" Jul 07 14:57:15.015203 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:57:15.015144 2557 generic.go:356] "Generic (PLEG): container finished" podID="03220e17-840a-4aee-a7de-e226c06e7ec6" containerID="1eedede00ab65e9956168e48c5dc3b25920239d560f0750a76194c768864b162" exitCode=0 Jul 07 14:57:15.015203 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:57:15.015186 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-4afb43ce-f489d75d9-sg5kv" Jul 07 14:57:15.015319 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:57:15.015226 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-4afb43ce-f489d75d9-sg5kv" event={"ID":"03220e17-840a-4aee-a7de-e226c06e7ec6","Type":"ContainerDied","Data":"1eedede00ab65e9956168e48c5dc3b25920239d560f0750a76194c768864b162"} Jul 07 14:57:15.015319 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:57:15.015277 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-4afb43ce-f489d75d9-sg5kv" event={"ID":"03220e17-840a-4aee-a7de-e226c06e7ec6","Type":"ContainerDied","Data":"80bab7e1865e67d7a39754eec9b2ac552eb37bb898ff8eef6381c1e31485d02f"} Jul 07 14:57:15.015319 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:57:15.015304 2557 scope.go:117] "RemoveContainer" containerID="1eedede00ab65e9956168e48c5dc3b25920239d560f0750a76194c768864b162" Jul 07 14:57:15.024745 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:57:15.024729 2557 scope.go:117] "RemoveContainer" containerID="1eedede00ab65e9956168e48c5dc3b25920239d560f0750a76194c768864b162" Jul 07 14:57:15.024976 ip-10-0-134-42 kubenswrapper[2557]: E0707 14:57:15.024956 2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1eedede00ab65e9956168e48c5dc3b25920239d560f0750a76194c768864b162\": container with ID starting with 1eedede00ab65e9956168e48c5dc3b25920239d560f0750a76194c768864b162 not found: ID does not exist" containerID="1eedede00ab65e9956168e48c5dc3b25920239d560f0750a76194c768864b162" Jul 07 14:57:15.025019 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:57:15.024984 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1eedede00ab65e9956168e48c5dc3b25920239d560f0750a76194c768864b162"} err="failed to get container status \"1eedede00ab65e9956168e48c5dc3b25920239d560f0750a76194c768864b162\": rpc error: code = NotFound desc = could not find container \"1eedede00ab65e9956168e48c5dc3b25920239d560f0750a76194c768864b162\": container with ID starting with 1eedede00ab65e9956168e48c5dc3b25920239d560f0750a76194c768864b162 not found: ID does not exist" Jul 07 14:57:15.037269 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:57:15.037246 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-mt-4afb43ce-f489d75d9-sg5kv"] Jul 07 14:57:15.042687 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:57:15.042667 2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-mt-4afb43ce-f489d75d9-sg5kv"] Jul 07 14:57:15.835584 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:57:15.835555 2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="03220e17-840a-4aee-a7de-e226c06e7ec6" path="/var/lib/kubelet/pods/03220e17-840a-4aee-a7de-e226c06e7ec6/volumes" Jul 07 14:57:20.107771 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:57:20.107736 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:57:25.183361 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:57:25.183323 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:57:46.573068 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:57:46.572984 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:57:48.181241 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:57:48.181200 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:57:54.655473 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:57:54.655438 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:57:55.843789 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:57:55.843756 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:57:56.248486 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:57:56.248396 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:57:56.348643 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:57:56.348614 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:57:56.450656 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:57:56.450626 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:57:57.224165 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:57:57.224119 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:57:57.338691 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:57:57.338657 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:57:57.471875 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:57:57.471836 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:57:58.910164 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:57:58.910128 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:58:05.351367 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:58:05.351335 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:58:05.450231 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:58:05.450197 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:58:06.331346 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:58:06.331309 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:58:12.798912 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:58:12.798871 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:58:12.958189 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:58:12.958153 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:58:13.075446 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:58:13.075413 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:58:23.607706 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:58:23.607671 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:58:23.894614 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:58:23.894535 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:58:26.411853 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:58:26.411814 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:58:26.479094 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:58:26.479060 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:58:44.960656 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:58:44.960619 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:58:51.523941 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:58:51.523901 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:58:51.596779 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:58:51.596749 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:58:51.917620 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:58:51.917593 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:58:52.260993 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:58:52.260900 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:58:52.364782 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:58:52.364751 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:58:55.081947 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:58:55.081905 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:58:55.161798 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:58:55.161771 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-inf-a-17aa9c-665f8c88f9-jwfbt"] Jul 07 14:58:55.162177 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:58:55.162164 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="03220e17-840a-4aee-a7de-e226c06e7ec6" containerName="maas-api" Jul 07 14:58:55.162221 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:58:55.162180 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="03220e17-840a-4aee-a7de-e226c06e7ec6" containerName="maas-api" Jul 07 14:58:55.162255 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:58:55.162249 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="03220e17-840a-4aee-a7de-e226c06e7ec6" containerName="maas-api" Jul 07 14:58:55.165457 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:58:55.165440 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-a-17aa9c-665f8c88f9-jwfbt" Jul 07 14:58:55.169665 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:58:55.169644 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-inf-a-17aa9c-serving-cert\"" Jul 07 14:58:55.180138 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:58:55.180116 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-inf-a-17aa9c-665f8c88f9-jwfbt"] Jul 07 14:58:55.189846 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:58:55.189821 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:58:55.225050 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:58:55.225024 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5mmxc\" (UniqueName: \"kubernetes.io/projected/87bc723f-df5f-4e4b-892d-8fbbbf304e82-kube-api-access-5mmxc\") pod \"maas-api-e2e-inf-a-17aa9c-665f8c88f9-jwfbt\" (UID: \"87bc723f-df5f-4e4b-892d-8fbbbf304e82\") " pod="opendatahub/maas-api-e2e-inf-a-17aa9c-665f8c88f9-jwfbt" Jul 07 14:58:55.225158 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:58:55.225056 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/87bc723f-df5f-4e4b-892d-8fbbbf304e82-maas-api-tls\") pod \"maas-api-e2e-inf-a-17aa9c-665f8c88f9-jwfbt\" (UID: \"87bc723f-df5f-4e4b-892d-8fbbbf304e82\") " pod="opendatahub/maas-api-e2e-inf-a-17aa9c-665f8c88f9-jwfbt" Jul 07 14:58:55.225158 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:58:55.225092 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/87bc723f-df5f-4e4b-892d-8fbbbf304e82-openshift-service-ca\") pod \"maas-api-e2e-inf-a-17aa9c-665f8c88f9-jwfbt\" (UID: \"87bc723f-df5f-4e4b-892d-8fbbbf304e82\") " pod="opendatahub/maas-api-e2e-inf-a-17aa9c-665f8c88f9-jwfbt" Jul 07 14:58:55.288509 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:58:55.288481 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:58:55.326402 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:58:55.326380 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5mmxc\" (UniqueName: \"kubernetes.io/projected/87bc723f-df5f-4e4b-892d-8fbbbf304e82-kube-api-access-5mmxc\") pod \"maas-api-e2e-inf-a-17aa9c-665f8c88f9-jwfbt\" (UID: \"87bc723f-df5f-4e4b-892d-8fbbbf304e82\") " pod="opendatahub/maas-api-e2e-inf-a-17aa9c-665f8c88f9-jwfbt" Jul 07 14:58:55.326513 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:58:55.326412 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/87bc723f-df5f-4e4b-892d-8fbbbf304e82-maas-api-tls\") pod \"maas-api-e2e-inf-a-17aa9c-665f8c88f9-jwfbt\" (UID: \"87bc723f-df5f-4e4b-892d-8fbbbf304e82\") " pod="opendatahub/maas-api-e2e-inf-a-17aa9c-665f8c88f9-jwfbt" Jul 07 14:58:55.326513 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:58:55.326454 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/87bc723f-df5f-4e4b-892d-8fbbbf304e82-openshift-service-ca\") pod \"maas-api-e2e-inf-a-17aa9c-665f8c88f9-jwfbt\" (UID: \"87bc723f-df5f-4e4b-892d-8fbbbf304e82\") " pod="opendatahub/maas-api-e2e-inf-a-17aa9c-665f8c88f9-jwfbt" Jul 07 14:58:55.326626 ip-10-0-134-42 kubenswrapper[2557]: E0707 14:58:55.326608 2557 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-inf-a-17aa9c-serving-cert: secret "maas-api-e2e-inf-a-17aa9c-serving-cert" not found Jul 07 14:58:55.326707 ip-10-0-134-42 kubenswrapper[2557]: E0707 14:58:55.326698 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/87bc723f-df5f-4e4b-892d-8fbbbf304e82-maas-api-tls podName:87bc723f-df5f-4e4b-892d-8fbbbf304e82 nodeName:}" failed. No retries permitted until 2026-07-07 14:58:55.826683329 +0000 UTC m=+1972.535821692 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/87bc723f-df5f-4e4b-892d-8fbbbf304e82-maas-api-tls") pod "maas-api-e2e-inf-a-17aa9c-665f8c88f9-jwfbt" (UID: "87bc723f-df5f-4e4b-892d-8fbbbf304e82") : secret "maas-api-e2e-inf-a-17aa9c-serving-cert" not found Jul 07 14:58:55.327091 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:58:55.327074 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/87bc723f-df5f-4e4b-892d-8fbbbf304e82-openshift-service-ca\") pod \"maas-api-e2e-inf-a-17aa9c-665f8c88f9-jwfbt\" (UID: \"87bc723f-df5f-4e4b-892d-8fbbbf304e82\") " pod="opendatahub/maas-api-e2e-inf-a-17aa9c-665f8c88f9-jwfbt" Jul 07 14:58:55.334939 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:58:55.334881 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5mmxc\" (UniqueName: \"kubernetes.io/projected/87bc723f-df5f-4e4b-892d-8fbbbf304e82-kube-api-access-5mmxc\") pod \"maas-api-e2e-inf-a-17aa9c-665f8c88f9-jwfbt\" (UID: \"87bc723f-df5f-4e4b-892d-8fbbbf304e82\") " pod="opendatahub/maas-api-e2e-inf-a-17aa9c-665f8c88f9-jwfbt" Jul 07 14:58:55.490121 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:58:55.490100 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:58:55.775118 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:58:55.775036 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:58:55.830410 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:58:55.830376 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/87bc723f-df5f-4e4b-892d-8fbbbf304e82-maas-api-tls\") pod \"maas-api-e2e-inf-a-17aa9c-665f8c88f9-jwfbt\" (UID: \"87bc723f-df5f-4e4b-892d-8fbbbf304e82\") " pod="opendatahub/maas-api-e2e-inf-a-17aa9c-665f8c88f9-jwfbt" Jul 07 14:58:55.832947 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:58:55.832912 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/87bc723f-df5f-4e4b-892d-8fbbbf304e82-maas-api-tls\") pod \"maas-api-e2e-inf-a-17aa9c-665f8c88f9-jwfbt\" (UID: \"87bc723f-df5f-4e4b-892d-8fbbbf304e82\") " pod="opendatahub/maas-api-e2e-inf-a-17aa9c-665f8c88f9-jwfbt" Jul 07 14:58:55.873581 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:58:55.873550 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:58:56.075552 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:58:56.075498 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-a-17aa9c-665f8c88f9-jwfbt" Jul 07 14:58:56.212713 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:58:56.212686 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-inf-a-17aa9c-665f8c88f9-jwfbt"] Jul 07 14:58:56.215084 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:58:56.215057 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod87bc723f_df5f_4e4b_892d_8fbbbf304e82.slice/crio-0babdaf09d90a8eaa245ab872c7ed090325f5eb147aad6366d554ed64d9dcfdb WatchSource:0}: Error finding container 0babdaf09d90a8eaa245ab872c7ed090325f5eb147aad6366d554ed64d9dcfdb: Status 404 returned error can't find the container with id 0babdaf09d90a8eaa245ab872c7ed090325f5eb147aad6366d554ed64d9dcfdb Jul 07 14:58:56.216790 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:58:56.216773 2557 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 07 14:58:56.607484 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:58:56.607456 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-a-17aa9c-665f8c88f9-jwfbt" event={"ID":"87bc723f-df5f-4e4b-892d-8fbbbf304e82","Type":"ContainerStarted","Data":"0babdaf09d90a8eaa245ab872c7ed090325f5eb147aad6366d554ed64d9dcfdb"} Jul 07 14:58:57.614128 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:58:57.614092 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-a-17aa9c-665f8c88f9-jwfbt" event={"ID":"87bc723f-df5f-4e4b-892d-8fbbbf304e82","Type":"ContainerStarted","Data":"27e78bf3f54001b9f2c3543a9dea78ed53d561569b7216f2e0941809d76bcfeb"} Jul 07 14:58:57.614535 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:58:57.614238 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-inf-a-17aa9c-665f8c88f9-jwfbt" Jul 07 14:58:57.637180 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:58:57.637139 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-inf-a-17aa9c-665f8c88f9-jwfbt" podStartSLOduration=2.153760669 podStartE2EDuration="2.637126751s" totalImagesPullingTime="483.366082ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 14:58:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:58:57.634901958 +0000 UTC m=+1974.344040338" watchObservedRunningTime="2026-07-07 14:58:57.637126751 +0000 UTC m=+1974.346265131" Jul 07 14:58:59.629888 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:58:59.629854 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:59:00.022326 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:59:00.022245 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:59:00.434064 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:59:00.434021 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:59:00.600836 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:59:00.600810 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:59:03.125320 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:59:03.125277 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:59:03.227079 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:59:03.227027 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:59:03.296858 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:59:03.296819 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-inf-b-54f2c7-89cd6645-85vnz"] Jul 07 14:59:03.300946 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:59:03.300926 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-b-54f2c7-89cd6645-85vnz" Jul 07 14:59:03.303397 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:59:03.303366 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-inf-b-54f2c7-serving-cert\"" Jul 07 14:59:03.313644 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:59:03.313619 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-inf-b-54f2c7-89cd6645-85vnz"] Jul 07 14:59:03.344833 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:59:03.344804 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:59:03.402777 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:59:03.402704 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/09b684b8-991c-4478-9c29-c24a57d0e259-maas-api-tls\") pod \"maas-api-e2e-inf-b-54f2c7-89cd6645-85vnz\" (UID: \"09b684b8-991c-4478-9c29-c24a57d0e259\") " pod="opendatahub/maas-api-e2e-inf-b-54f2c7-89cd6645-85vnz" Jul 07 14:59:03.402777 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:59:03.402763 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5wrkm\" (UniqueName: \"kubernetes.io/projected/09b684b8-991c-4478-9c29-c24a57d0e259-kube-api-access-5wrkm\") pod \"maas-api-e2e-inf-b-54f2c7-89cd6645-85vnz\" (UID: \"09b684b8-991c-4478-9c29-c24a57d0e259\") " pod="opendatahub/maas-api-e2e-inf-b-54f2c7-89cd6645-85vnz" Jul 07 14:59:03.402940 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:59:03.402888 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/09b684b8-991c-4478-9c29-c24a57d0e259-openshift-service-ca\") pod \"maas-api-e2e-inf-b-54f2c7-89cd6645-85vnz\" (UID: \"09b684b8-991c-4478-9c29-c24a57d0e259\") " pod="opendatahub/maas-api-e2e-inf-b-54f2c7-89cd6645-85vnz" Jul 07 14:59:03.504533 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:59:03.504486 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5wrkm\" (UniqueName: \"kubernetes.io/projected/09b684b8-991c-4478-9c29-c24a57d0e259-kube-api-access-5wrkm\") pod \"maas-api-e2e-inf-b-54f2c7-89cd6645-85vnz\" (UID: \"09b684b8-991c-4478-9c29-c24a57d0e259\") " pod="opendatahub/maas-api-e2e-inf-b-54f2c7-89cd6645-85vnz" Jul 07 14:59:03.504705 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:59:03.504627 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/09b684b8-991c-4478-9c29-c24a57d0e259-openshift-service-ca\") pod \"maas-api-e2e-inf-b-54f2c7-89cd6645-85vnz\" (UID: \"09b684b8-991c-4478-9c29-c24a57d0e259\") " pod="opendatahub/maas-api-e2e-inf-b-54f2c7-89cd6645-85vnz" Jul 07 14:59:03.504705 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:59:03.504689 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/09b684b8-991c-4478-9c29-c24a57d0e259-maas-api-tls\") pod \"maas-api-e2e-inf-b-54f2c7-89cd6645-85vnz\" (UID: \"09b684b8-991c-4478-9c29-c24a57d0e259\") " pod="opendatahub/maas-api-e2e-inf-b-54f2c7-89cd6645-85vnz" Jul 07 14:59:03.504838 ip-10-0-134-42 kubenswrapper[2557]: E0707 14:59:03.504788 2557 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-inf-b-54f2c7-serving-cert: secret "maas-api-e2e-inf-b-54f2c7-serving-cert" not found Jul 07 14:59:03.504880 ip-10-0-134-42 kubenswrapper[2557]: E0707 14:59:03.504840 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/09b684b8-991c-4478-9c29-c24a57d0e259-maas-api-tls podName:09b684b8-991c-4478-9c29-c24a57d0e259 nodeName:}" failed. No retries permitted until 2026-07-07 14:59:04.004825463 +0000 UTC m=+1980.713963822 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/09b684b8-991c-4478-9c29-c24a57d0e259-maas-api-tls") pod "maas-api-e2e-inf-b-54f2c7-89cd6645-85vnz" (UID: "09b684b8-991c-4478-9c29-c24a57d0e259") : secret "maas-api-e2e-inf-b-54f2c7-serving-cert" not found Jul 07 14:59:03.505288 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:59:03.505266 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/09b684b8-991c-4478-9c29-c24a57d0e259-openshift-service-ca\") pod \"maas-api-e2e-inf-b-54f2c7-89cd6645-85vnz\" (UID: \"09b684b8-991c-4478-9c29-c24a57d0e259\") " pod="opendatahub/maas-api-e2e-inf-b-54f2c7-89cd6645-85vnz" Jul 07 14:59:03.513180 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:59:03.513157 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5wrkm\" (UniqueName: \"kubernetes.io/projected/09b684b8-991c-4478-9c29-c24a57d0e259-kube-api-access-5wrkm\") pod \"maas-api-e2e-inf-b-54f2c7-89cd6645-85vnz\" (UID: \"09b684b8-991c-4478-9c29-c24a57d0e259\") " pod="opendatahub/maas-api-e2e-inf-b-54f2c7-89cd6645-85vnz" Jul 07 14:59:03.626197 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:59:03.626161 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-inf-a-17aa9c-665f8c88f9-jwfbt" Jul 07 14:59:03.765869 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:59:03.765775 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:59:03.863765 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:59:03.863733 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:59:04.009119 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:59:04.009092 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/09b684b8-991c-4478-9c29-c24a57d0e259-maas-api-tls\") pod \"maas-api-e2e-inf-b-54f2c7-89cd6645-85vnz\" (UID: \"09b684b8-991c-4478-9c29-c24a57d0e259\") " pod="opendatahub/maas-api-e2e-inf-b-54f2c7-89cd6645-85vnz" Jul 07 14:59:04.011768 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:59:04.011736 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/09b684b8-991c-4478-9c29-c24a57d0e259-maas-api-tls\") pod \"maas-api-e2e-inf-b-54f2c7-89cd6645-85vnz\" (UID: \"09b684b8-991c-4478-9c29-c24a57d0e259\") " pod="opendatahub/maas-api-e2e-inf-b-54f2c7-89cd6645-85vnz" Jul 07 14:59:04.212646 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:59:04.212619 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-b-54f2c7-89cd6645-85vnz" Jul 07 14:59:04.353301 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:59:04.353272 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-inf-b-54f2c7-89cd6645-85vnz"] Jul 07 14:59:04.354744 ip-10-0-134-42 kubenswrapper[2557]: W0707 14:59:04.354708 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod09b684b8_991c_4478_9c29_c24a57d0e259.slice/crio-771e1642eebf069354b75f3af28855791ac1e91b5b8bcb955743bc181ca4878c WatchSource:0}: Error finding container 771e1642eebf069354b75f3af28855791ac1e91b5b8bcb955743bc181ca4878c: Status 404 returned error can't find the container with id 771e1642eebf069354b75f3af28855791ac1e91b5b8bcb955743bc181ca4878c Jul 07 14:59:04.654970 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:59:04.654934 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-b-54f2c7-89cd6645-85vnz" event={"ID":"09b684b8-991c-4478-9c29-c24a57d0e259","Type":"ContainerStarted","Data":"771e1642eebf069354b75f3af28855791ac1e91b5b8bcb955743bc181ca4878c"} Jul 07 14:59:05.662009 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:59:05.661974 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-b-54f2c7-89cd6645-85vnz" event={"ID":"09b684b8-991c-4478-9c29-c24a57d0e259","Type":"ContainerStarted","Data":"59fb53f206e502cb512dcf72b75a813f691046866c52f3201307ff53d582ee2f"} Jul 07 14:59:05.662394 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:59:05.662067 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-inf-b-54f2c7-89cd6645-85vnz" Jul 07 14:59:05.683497 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:59:05.683455 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-inf-b-54f2c7-89cd6645-85vnz" podStartSLOduration=2.3735656560000002 podStartE2EDuration="2.683439905s" totalImagesPullingTime="309.874249ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 14:59:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:59:05.681194195 +0000 UTC m=+1982.390332576" watchObservedRunningTime="2026-07-07 14:59:05.683439905 +0000 UTC m=+1982.392578287" Jul 07 14:59:07.716039 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:59:07.716007 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:59:11.673119 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:59:11.673088 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-inf-b-54f2c7-89cd6645-85vnz" Jul 07 14:59:16.516586 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:59:16.516554 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:59:16.615163 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:59:16.615123 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:59:21.774674 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:59:21.774632 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:59:25.433932 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:59:25.433886 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:59:25.532586 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:59:25.532552 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:59:41.671464 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:59:41.671427 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 14:59:59.265628 ip-10-0-134-42 kubenswrapper[2557]: I0707 14:59:59.265575 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 15:00:00.139235 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:00:00.139196 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-key-cleanup-29723940-d5h4k"] Jul 07 15:00:00.142682 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:00:00.142661 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-key-cleanup-e2e-inf-b-54f2c7-29723940-r9gkj"] Jul 07 15:00:00.142815 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:00:00.142768 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29723940-d5h4k" Jul 07 15:00:00.146489 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:00:00.146474 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-key-cleanup-e2e-inf-a-17aa9c-29723940-m2hzf"] Jul 07 15:00:00.146588 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:00:00.146576 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-e2e-inf-b-54f2c7-29723940-r9gkj" Jul 07 15:00:00.150125 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:00:00.150100 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-e2e-inf-a-17aa9c-29723940-m2hzf" Jul 07 15:00:00.151273 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:00:00.151253 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-e2e-inf-a-17aa9c-29723940-m2hzf"] Jul 07 15:00:00.153978 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:00:00.153957 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-29723940-d5h4k"] Jul 07 15:00:00.169030 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:00:00.168988 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-e2e-inf-b-54f2c7-29723940-r9gkj"] Jul 07 15:00:00.216219 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:00:00.216195 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9xb4l\" (UniqueName: \"kubernetes.io/projected/eff497ca-5cde-48bf-9101-43be457e22b4-kube-api-access-9xb4l\") pod \"maas-api-key-cleanup-e2e-inf-b-54f2c7-29723940-r9gkj\" (UID: \"eff497ca-5cde-48bf-9101-43be457e22b4\") " pod="opendatahub/maas-api-key-cleanup-e2e-inf-b-54f2c7-29723940-r9gkj" Jul 07 15:00:00.216323 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:00:00.216240 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jnsdw\" (UniqueName: \"kubernetes.io/projected/46fa2d39-183b-44f2-ac8b-216630a0fb60-kube-api-access-jnsdw\") pod \"maas-api-key-cleanup-e2e-inf-a-17aa9c-29723940-m2hzf\" (UID: \"46fa2d39-183b-44f2-ac8b-216630a0fb60\") " pod="opendatahub/maas-api-key-cleanup-e2e-inf-a-17aa9c-29723940-m2hzf" Jul 07 15:00:00.216323 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:00:00.216316 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pgbtn\" (UniqueName: \"kubernetes.io/projected/338a9d19-9258-4d24-b1ed-bcf984664ed3-kube-api-access-pgbtn\") pod \"maas-api-key-cleanup-29723940-d5h4k\" (UID: \"338a9d19-9258-4d24-b1ed-bcf984664ed3\") " pod="opendatahub/maas-api-key-cleanup-29723940-d5h4k" Jul 07 15:00:00.317449 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:00:00.317424 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pgbtn\" (UniqueName: \"kubernetes.io/projected/338a9d19-9258-4d24-b1ed-bcf984664ed3-kube-api-access-pgbtn\") pod \"maas-api-key-cleanup-29723940-d5h4k\" (UID: \"338a9d19-9258-4d24-b1ed-bcf984664ed3\") " pod="opendatahub/maas-api-key-cleanup-29723940-d5h4k" Jul 07 15:00:00.317756 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:00:00.317464 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9xb4l\" (UniqueName: \"kubernetes.io/projected/eff497ca-5cde-48bf-9101-43be457e22b4-kube-api-access-9xb4l\") pod \"maas-api-key-cleanup-e2e-inf-b-54f2c7-29723940-r9gkj\" (UID: \"eff497ca-5cde-48bf-9101-43be457e22b4\") " pod="opendatahub/maas-api-key-cleanup-e2e-inf-b-54f2c7-29723940-r9gkj" Jul 07 15:00:00.317756 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:00:00.317506 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jnsdw\" (UniqueName: \"kubernetes.io/projected/46fa2d39-183b-44f2-ac8b-216630a0fb60-kube-api-access-jnsdw\") pod \"maas-api-key-cleanup-e2e-inf-a-17aa9c-29723940-m2hzf\" (UID: \"46fa2d39-183b-44f2-ac8b-216630a0fb60\") " pod="opendatahub/maas-api-key-cleanup-e2e-inf-a-17aa9c-29723940-m2hzf" Jul 07 15:00:00.325785 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:00:00.325765 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9xb4l\" (UniqueName: \"kubernetes.io/projected/eff497ca-5cde-48bf-9101-43be457e22b4-kube-api-access-9xb4l\") pod \"maas-api-key-cleanup-e2e-inf-b-54f2c7-29723940-r9gkj\" (UID: \"eff497ca-5cde-48bf-9101-43be457e22b4\") " pod="opendatahub/maas-api-key-cleanup-e2e-inf-b-54f2c7-29723940-r9gkj" Jul 07 15:00:00.325879 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:00:00.325827 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pgbtn\" (UniqueName: \"kubernetes.io/projected/338a9d19-9258-4d24-b1ed-bcf984664ed3-kube-api-access-pgbtn\") pod \"maas-api-key-cleanup-29723940-d5h4k\" (UID: \"338a9d19-9258-4d24-b1ed-bcf984664ed3\") " pod="opendatahub/maas-api-key-cleanup-29723940-d5h4k" Jul 07 15:00:00.325879 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:00:00.325871 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jnsdw\" (UniqueName: \"kubernetes.io/projected/46fa2d39-183b-44f2-ac8b-216630a0fb60-kube-api-access-jnsdw\") pod \"maas-api-key-cleanup-e2e-inf-a-17aa9c-29723940-m2hzf\" (UID: \"46fa2d39-183b-44f2-ac8b-216630a0fb60\") " pod="opendatahub/maas-api-key-cleanup-e2e-inf-a-17aa9c-29723940-m2hzf" Jul 07 15:00:00.456826 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:00:00.456761 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29723940-d5h4k" Jul 07 15:00:00.464802 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:00:00.464783 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-e2e-inf-b-54f2c7-29723940-r9gkj" Jul 07 15:00:00.473697 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:00:00.473667 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-e2e-inf-a-17aa9c-29723940-m2hzf" Jul 07 15:00:00.622965 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:00:00.622934 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-29723940-d5h4k"] Jul 07 15:00:00.623555 ip-10-0-134-42 kubenswrapper[2557]: W0707 15:00:00.623508 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod338a9d19_9258_4d24_b1ed_bcf984664ed3.slice/crio-13498acd9505b69f126b55e8ed7c6c40541584dba5fdadb48a921d9de7530eaf WatchSource:0}: Error finding container 13498acd9505b69f126b55e8ed7c6c40541584dba5fdadb48a921d9de7530eaf: Status 404 returned error can't find the container with id 13498acd9505b69f126b55e8ed7c6c40541584dba5fdadb48a921d9de7530eaf Jul 07 15:00:00.857133 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:00:00.857108 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-e2e-inf-b-54f2c7-29723940-r9gkj"] Jul 07 15:00:00.858375 ip-10-0-134-42 kubenswrapper[2557]: W0707 15:00:00.858346 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeff497ca_5cde_48bf_9101_43be457e22b4.slice/crio-e9e6fecb78857b9377a9699caadb82edc885004924c8d027f1f58275ab4a187e WatchSource:0}: Error finding container e9e6fecb78857b9377a9699caadb82edc885004924c8d027f1f58275ab4a187e: Status 404 returned error can't find the container with id e9e6fecb78857b9377a9699caadb82edc885004924c8d027f1f58275ab4a187e Jul 07 15:00:00.859801 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:00:00.859780 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-e2e-inf-a-17aa9c-29723940-m2hzf"] Jul 07 15:00:00.861959 ip-10-0-134-42 kubenswrapper[2557]: W0707 15:00:00.861928 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod46fa2d39_183b_44f2_ac8b_216630a0fb60.slice/crio-ab71af0cecaa4e0b544e7e429a3de4c3686babf0f01c9a63417ad52d611acf76 WatchSource:0}: Error finding container ab71af0cecaa4e0b544e7e429a3de4c3686babf0f01c9a63417ad52d611acf76: Status 404 returned error can't find the container with id ab71af0cecaa4e0b544e7e429a3de4c3686babf0f01c9a63417ad52d611acf76 Jul 07 15:00:00.985215 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:00:00.985183 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-e2e-inf-b-54f2c7-29723940-r9gkj" event={"ID":"eff497ca-5cde-48bf-9101-43be457e22b4","Type":"ContainerStarted","Data":"e9e6fecb78857b9377a9699caadb82edc885004924c8d027f1f58275ab4a187e"} Jul 07 15:00:00.986593 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:00:00.986571 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29723940-d5h4k" event={"ID":"338a9d19-9258-4d24-b1ed-bcf984664ed3","Type":"ContainerStarted","Data":"dce8a04f9a7846f36141a630302fc98d5bf1d6ba61d98c45618025f011a066d1"} Jul 07 15:00:00.986706 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:00:00.986620 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29723940-d5h4k" event={"ID":"338a9d19-9258-4d24-b1ed-bcf984664ed3","Type":"ContainerStarted","Data":"13498acd9505b69f126b55e8ed7c6c40541584dba5fdadb48a921d9de7530eaf"} Jul 07 15:00:00.987773 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:00:00.987752 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-e2e-inf-a-17aa9c-29723940-m2hzf" event={"ID":"46fa2d39-183b-44f2-ac8b-216630a0fb60","Type":"ContainerStarted","Data":"ab71af0cecaa4e0b544e7e429a3de4c3686babf0f01c9a63417ad52d611acf76"} Jul 07 15:00:01.003421 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:00:01.003374 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-key-cleanup-29723940-d5h4k" podStartSLOduration=1.003360645 podStartE2EDuration="1.003360645s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:00:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:00:01.001255672 +0000 UTC m=+2037.710394051" watchObservedRunningTime="2026-07-07 15:00:01.003360645 +0000 UTC m=+2037.712499025" Jul 07 15:00:01.995249 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:00:01.995221 2557 generic.go:356] "Generic (PLEG): container finished" podID="eff497ca-5cde-48bf-9101-43be457e22b4" containerID="17e3a097fad9516d4ba9a965fcded096b892ee8c54de730259db7b3083bdf6db" exitCode=0 Jul 07 15:00:01.995643 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:00:01.995307 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-e2e-inf-b-54f2c7-29723940-r9gkj" event={"ID":"eff497ca-5cde-48bf-9101-43be457e22b4","Type":"ContainerDied","Data":"17e3a097fad9516d4ba9a965fcded096b892ee8c54de730259db7b3083bdf6db"} Jul 07 15:00:01.996894 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:00:01.996867 2557 generic.go:356] "Generic (PLEG): container finished" podID="338a9d19-9258-4d24-b1ed-bcf984664ed3" containerID="dce8a04f9a7846f36141a630302fc98d5bf1d6ba61d98c45618025f011a066d1" exitCode=0 Jul 07 15:00:01.996977 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:00:01.996944 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29723940-d5h4k" event={"ID":"338a9d19-9258-4d24-b1ed-bcf984664ed3","Type":"ContainerDied","Data":"dce8a04f9a7846f36141a630302fc98d5bf1d6ba61d98c45618025f011a066d1"} Jul 07 15:00:01.998370 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:00:01.998350 2557 generic.go:356] "Generic (PLEG): container finished" podID="46fa2d39-183b-44f2-ac8b-216630a0fb60" containerID="82b357bdddac4de299d92355f2e7d3e5d924f861e8cd13c75415e4347cebf6e1" exitCode=0 Jul 07 15:00:01.998451 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:00:01.998411 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-e2e-inf-a-17aa9c-29723940-m2hzf" event={"ID":"46fa2d39-183b-44f2-ac8b-216630a0fb60","Type":"ContainerDied","Data":"82b357bdddac4de299d92355f2e7d3e5d924f861e8cd13c75415e4347cebf6e1"} Jul 07 15:00:03.174193 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:00:03.174170 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-e2e-inf-b-54f2c7-29723940-r9gkj" Jul 07 15:00:03.217840 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:00:03.217817 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29723940-d5h4k" Jul 07 15:00:03.220751 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:00:03.220732 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-e2e-inf-a-17aa9c-29723940-m2hzf" Jul 07 15:00:03.245502 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:00:03.245480 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9xb4l\" (UniqueName: \"kubernetes.io/projected/eff497ca-5cde-48bf-9101-43be457e22b4-kube-api-access-9xb4l\") pod \"eff497ca-5cde-48bf-9101-43be457e22b4\" (UID: \"eff497ca-5cde-48bf-9101-43be457e22b4\") " Jul 07 15:00:03.247411 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:00:03.247391 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eff497ca-5cde-48bf-9101-43be457e22b4-kube-api-access-9xb4l" (OuterVolumeSpecName: "kube-api-access-9xb4l") pod "eff497ca-5cde-48bf-9101-43be457e22b4" (UID: "eff497ca-5cde-48bf-9101-43be457e22b4"). InnerVolumeSpecName "kube-api-access-9xb4l". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 15:00:03.346765 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:00:03.346738 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jnsdw\" (UniqueName: \"kubernetes.io/projected/46fa2d39-183b-44f2-ac8b-216630a0fb60-kube-api-access-jnsdw\") pod \"46fa2d39-183b-44f2-ac8b-216630a0fb60\" (UID: \"46fa2d39-183b-44f2-ac8b-216630a0fb60\") " Jul 07 15:00:03.346913 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:00:03.346799 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pgbtn\" (UniqueName: \"kubernetes.io/projected/338a9d19-9258-4d24-b1ed-bcf984664ed3-kube-api-access-pgbtn\") pod \"338a9d19-9258-4d24-b1ed-bcf984664ed3\" (UID: \"338a9d19-9258-4d24-b1ed-bcf984664ed3\") " Jul 07 15:00:03.346986 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:00:03.346975 2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9xb4l\" (UniqueName: \"kubernetes.io/projected/eff497ca-5cde-48bf-9101-43be457e22b4-kube-api-access-9xb4l\") on node \"ip-10-0-134-42.ec2.internal\" DevicePath \"\"" Jul 07 15:00:03.348790 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:00:03.348764 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/46fa2d39-183b-44f2-ac8b-216630a0fb60-kube-api-access-jnsdw" (OuterVolumeSpecName: "kube-api-access-jnsdw") pod "46fa2d39-183b-44f2-ac8b-216630a0fb60" (UID: "46fa2d39-183b-44f2-ac8b-216630a0fb60"). InnerVolumeSpecName "kube-api-access-jnsdw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 15:00:03.348902 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:00:03.348794 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/338a9d19-9258-4d24-b1ed-bcf984664ed3-kube-api-access-pgbtn" (OuterVolumeSpecName: "kube-api-access-pgbtn") pod "338a9d19-9258-4d24-b1ed-bcf984664ed3" (UID: "338a9d19-9258-4d24-b1ed-bcf984664ed3"). InnerVolumeSpecName "kube-api-access-pgbtn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 15:00:03.448271 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:00:03.448240 2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jnsdw\" (UniqueName: \"kubernetes.io/projected/46fa2d39-183b-44f2-ac8b-216630a0fb60-kube-api-access-jnsdw\") on node \"ip-10-0-134-42.ec2.internal\" DevicePath \"\"" Jul 07 15:00:03.448271 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:00:03.448267 2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pgbtn\" (UniqueName: \"kubernetes.io/projected/338a9d19-9258-4d24-b1ed-bcf984664ed3-kube-api-access-pgbtn\") on node \"ip-10-0-134-42.ec2.internal\" DevicePath \"\"" Jul 07 15:00:04.011702 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:00:04.011674 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29723940-d5h4k" Jul 07 15:00:04.011836 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:00:04.011712 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29723940-d5h4k" event={"ID":"338a9d19-9258-4d24-b1ed-bcf984664ed3","Type":"ContainerDied","Data":"13498acd9505b69f126b55e8ed7c6c40541584dba5fdadb48a921d9de7530eaf"} Jul 07 15:00:04.011836 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:00:04.011755 2557 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="13498acd9505b69f126b55e8ed7c6c40541584dba5fdadb48a921d9de7530eaf" Jul 07 15:00:04.013067 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:00:04.013049 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-e2e-inf-a-17aa9c-29723940-m2hzf" Jul 07 15:00:04.013189 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:00:04.013095 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-e2e-inf-a-17aa9c-29723940-m2hzf" event={"ID":"46fa2d39-183b-44f2-ac8b-216630a0fb60","Type":"ContainerDied","Data":"ab71af0cecaa4e0b544e7e429a3de4c3686babf0f01c9a63417ad52d611acf76"} Jul 07 15:00:04.013189 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:00:04.013141 2557 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ab71af0cecaa4e0b544e7e429a3de4c3686babf0f01c9a63417ad52d611acf76" Jul 07 15:00:04.014559 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:00:04.014544 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-e2e-inf-b-54f2c7-29723940-r9gkj" Jul 07 15:00:04.014672 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:00:04.014553 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-e2e-inf-b-54f2c7-29723940-r9gkj" event={"ID":"eff497ca-5cde-48bf-9101-43be457e22b4","Type":"ContainerDied","Data":"e9e6fecb78857b9377a9699caadb82edc885004924c8d027f1f58275ab4a187e"} Jul 07 15:00:04.014672 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:00:04.014589 2557 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e9e6fecb78857b9377a9699caadb82edc885004924c8d027f1f58275ab4a187e" Jul 07 15:00:06.641575 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:00:06.641539 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 15:00:07.109401 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:00:07.109373 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 15:00:07.210274 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:00:07.210248 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 15:00:07.651854 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:00:07.651820 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 15:00:07.921781 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:00:07.921710 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 15:00:08.297816 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:00:08.297730 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 15:00:08.588847 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:00:08.588818 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-inf-a-17aa9c-665f8c88f9-jwfbt"] Jul 07 15:00:08.589031 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:00:08.589011 2557 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-inf-a-17aa9c-665f8c88f9-jwfbt" podUID="87bc723f-df5f-4e4b-892d-8fbbbf304e82" containerName="maas-api" containerID="cri-o://27e78bf3f54001b9f2c3543a9dea78ed53d561569b7216f2e0941809d76bcfeb" gracePeriod=30 Jul 07 15:00:08.620348 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:00:08.620315 2557 prober.go:120] "Probe failed" probeType="Readiness" pod="opendatahub/maas-api-e2e-inf-a-17aa9c-665f8c88f9-jwfbt" podUID="87bc723f-df5f-4e4b-892d-8fbbbf304e82" containerName="maas-api" probeResult="failure" output="Get \"https://10.132.0.67:8443/health\": dial tcp 10.132.0.67:8443: connect: connection refused" Jul 07 15:00:08.620708 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:00:08.620684 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-key-cleanup-e2e-inf-a-17aa9c-29723940-m2hzf"] Jul 07 15:00:08.623805 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:00:08.623780 2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-key-cleanup-e2e-inf-a-17aa9c-29723940-m2hzf"] Jul 07 15:00:08.841112 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:00:08.841050 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-a-17aa9c-665f8c88f9-jwfbt" Jul 07 15:00:08.845092 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:00:08.845068 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 15:00:08.944289 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:00:08.944260 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 15:00:09.003819 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:00:09.003791 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/87bc723f-df5f-4e4b-892d-8fbbbf304e82-openshift-service-ca\") pod \"87bc723f-df5f-4e4b-892d-8fbbbf304e82\" (UID: \"87bc723f-df5f-4e4b-892d-8fbbbf304e82\") " Jul 07 15:00:09.003944 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:00:09.003835 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5mmxc\" (UniqueName: \"kubernetes.io/projected/87bc723f-df5f-4e4b-892d-8fbbbf304e82-kube-api-access-5mmxc\") pod \"87bc723f-df5f-4e4b-892d-8fbbbf304e82\" (UID: \"87bc723f-df5f-4e4b-892d-8fbbbf304e82\") " Jul 07 15:00:09.003944 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:00:09.003855 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/87bc723f-df5f-4e4b-892d-8fbbbf304e82-maas-api-tls\") pod \"87bc723f-df5f-4e4b-892d-8fbbbf304e82\" (UID: \"87bc723f-df5f-4e4b-892d-8fbbbf304e82\") " Jul 07 15:00:09.004227 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:00:09.004199 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/87bc723f-df5f-4e4b-892d-8fbbbf304e82-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "87bc723f-df5f-4e4b-892d-8fbbbf304e82" (UID: "87bc723f-df5f-4e4b-892d-8fbbbf304e82"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 15:00:09.006035 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:00:09.006014 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/87bc723f-df5f-4e4b-892d-8fbbbf304e82-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "87bc723f-df5f-4e4b-892d-8fbbbf304e82" (UID: "87bc723f-df5f-4e4b-892d-8fbbbf304e82"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 15:00:09.006100 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:00:09.006071 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/87bc723f-df5f-4e4b-892d-8fbbbf304e82-kube-api-access-5mmxc" (OuterVolumeSpecName: "kube-api-access-5mmxc") pod "87bc723f-df5f-4e4b-892d-8fbbbf304e82" (UID: "87bc723f-df5f-4e4b-892d-8fbbbf304e82"). InnerVolumeSpecName "kube-api-access-5mmxc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 15:00:09.043758 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:00:09.043734 2557 generic.go:356] "Generic (PLEG): container finished" podID="87bc723f-df5f-4e4b-892d-8fbbbf304e82" containerID="27e78bf3f54001b9f2c3543a9dea78ed53d561569b7216f2e0941809d76bcfeb" exitCode=0 Jul 07 15:00:09.043841 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:00:09.043783 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-a-17aa9c-665f8c88f9-jwfbt" Jul 07 15:00:09.043841 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:00:09.043819 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-a-17aa9c-665f8c88f9-jwfbt" event={"ID":"87bc723f-df5f-4e4b-892d-8fbbbf304e82","Type":"ContainerDied","Data":"27e78bf3f54001b9f2c3543a9dea78ed53d561569b7216f2e0941809d76bcfeb"} Jul 07 15:00:09.043910 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:00:09.043888 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-a-17aa9c-665f8c88f9-jwfbt" event={"ID":"87bc723f-df5f-4e4b-892d-8fbbbf304e82","Type":"ContainerDied","Data":"0babdaf09d90a8eaa245ab872c7ed090325f5eb147aad6366d554ed64d9dcfdb"} Jul 07 15:00:09.043944 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:00:09.043909 2557 scope.go:117] "RemoveContainer" containerID="27e78bf3f54001b9f2c3543a9dea78ed53d561569b7216f2e0941809d76bcfeb" Jul 07 15:00:09.056232 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:00:09.056208 2557 scope.go:117] "RemoveContainer" containerID="27e78bf3f54001b9f2c3543a9dea78ed53d561569b7216f2e0941809d76bcfeb" Jul 07 15:00:09.056502 ip-10-0-134-42 kubenswrapper[2557]: E0707 15:00:09.056484 2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"27e78bf3f54001b9f2c3543a9dea78ed53d561569b7216f2e0941809d76bcfeb\": container with ID starting with 27e78bf3f54001b9f2c3543a9dea78ed53d561569b7216f2e0941809d76bcfeb not found: ID does not exist" containerID="27e78bf3f54001b9f2c3543a9dea78ed53d561569b7216f2e0941809d76bcfeb" Jul 07 15:00:09.056598 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:00:09.056529 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"27e78bf3f54001b9f2c3543a9dea78ed53d561569b7216f2e0941809d76bcfeb"} err="failed to get container status \"27e78bf3f54001b9f2c3543a9dea78ed53d561569b7216f2e0941809d76bcfeb\": rpc error: code = NotFound desc = could not find container \"27e78bf3f54001b9f2c3543a9dea78ed53d561569b7216f2e0941809d76bcfeb\": container with ID starting with 27e78bf3f54001b9f2c3543a9dea78ed53d561569b7216f2e0941809d76bcfeb not found: ID does not exist" Jul 07 15:00:09.081923 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:00:09.081901 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-inf-a-17aa9c-665f8c88f9-jwfbt"] Jul 07 15:00:09.092119 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:00:09.092064 2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-inf-a-17aa9c-665f8c88f9-jwfbt"] Jul 07 15:00:09.105453 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:00:09.105434 2557 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/87bc723f-df5f-4e4b-892d-8fbbbf304e82-openshift-service-ca\") on node \"ip-10-0-134-42.ec2.internal\" DevicePath \"\"" Jul 07 15:00:09.105564 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:00:09.105456 2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5mmxc\" (UniqueName: \"kubernetes.io/projected/87bc723f-df5f-4e4b-892d-8fbbbf304e82-kube-api-access-5mmxc\") on node \"ip-10-0-134-42.ec2.internal\" DevicePath \"\"" Jul 07 15:00:09.105564 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:00:09.105469 2557 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/87bc723f-df5f-4e4b-892d-8fbbbf304e82-maas-api-tls\") on node \"ip-10-0-134-42.ec2.internal\" DevicePath \"\"" Jul 07 15:00:09.836352 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:00:09.836321 2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="46fa2d39-183b-44f2-ac8b-216630a0fb60" path="/var/lib/kubelet/pods/46fa2d39-183b-44f2-ac8b-216630a0fb60/volumes" Jul 07 15:00:09.836714 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:00:09.836700 2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="87bc723f-df5f-4e4b-892d-8fbbbf304e82" path="/var/lib/kubelet/pods/87bc723f-df5f-4e4b-892d-8fbbbf304e82/volumes" Jul 07 15:00:13.372833 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:00:13.372805 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 15:00:50.370351 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:00:50.370272 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 15:00:50.697984 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:00:50.697877 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 15:00:50.954466 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:00:50.954382 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 15:00:51.137542 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:00:51.137484 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 15:00:51.558096 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:00:51.558063 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 15:00:51.657259 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:00:51.657227 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 15:00:51.781821 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:00:51.781784 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-inf-b-54f2c7-89cd6645-85vnz"] Jul 07 15:00:51.782113 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:00:51.782061 2557 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-inf-b-54f2c7-89cd6645-85vnz" podUID="09b684b8-991c-4478-9c29-c24a57d0e259" containerName="maas-api" containerID="cri-o://59fb53f206e502cb512dcf72b75a813f691046866c52f3201307ff53d582ee2f" gracePeriod=30 Jul 07 15:00:51.804617 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:00:51.804591 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-key-cleanup-e2e-inf-b-54f2c7-29723940-r9gkj"] Jul 07 15:00:51.813741 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:00:51.813685 2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-key-cleanup-e2e-inf-b-54f2c7-29723940-r9gkj"] Jul 07 15:00:51.817172 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:00:51.817151 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 15:00:51.836027 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:00:51.836001 2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="eff497ca-5cde-48bf-9101-43be457e22b4" path="/var/lib/kubelet/pods/eff497ca-5cde-48bf-9101-43be457e22b4/volumes" Jul 07 15:00:52.032200 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:00:52.032176 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-b-54f2c7-89cd6645-85vnz" Jul 07 15:00:52.187957 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:00:52.187922 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/09b684b8-991c-4478-9c29-c24a57d0e259-maas-api-tls\") pod \"09b684b8-991c-4478-9c29-c24a57d0e259\" (UID: \"09b684b8-991c-4478-9c29-c24a57d0e259\") " Jul 07 15:00:52.188134 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:00:52.188025 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/09b684b8-991c-4478-9c29-c24a57d0e259-openshift-service-ca\") pod \"09b684b8-991c-4478-9c29-c24a57d0e259\" (UID: \"09b684b8-991c-4478-9c29-c24a57d0e259\") " Jul 07 15:00:52.188134 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:00:52.188087 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5wrkm\" (UniqueName: \"kubernetes.io/projected/09b684b8-991c-4478-9c29-c24a57d0e259-kube-api-access-5wrkm\") pod \"09b684b8-991c-4478-9c29-c24a57d0e259\" (UID: \"09b684b8-991c-4478-9c29-c24a57d0e259\") " Jul 07 15:00:52.188475 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:00:52.188450 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/09b684b8-991c-4478-9c29-c24a57d0e259-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "09b684b8-991c-4478-9c29-c24a57d0e259" (UID: "09b684b8-991c-4478-9c29-c24a57d0e259"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 15:00:52.190114 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:00:52.190091 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/09b684b8-991c-4478-9c29-c24a57d0e259-kube-api-access-5wrkm" (OuterVolumeSpecName: "kube-api-access-5wrkm") pod "09b684b8-991c-4478-9c29-c24a57d0e259" (UID: "09b684b8-991c-4478-9c29-c24a57d0e259"). InnerVolumeSpecName "kube-api-access-5wrkm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 15:00:52.190187 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:00:52.190100 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/09b684b8-991c-4478-9c29-c24a57d0e259-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "09b684b8-991c-4478-9c29-c24a57d0e259" (UID: "09b684b8-991c-4478-9c29-c24a57d0e259"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 15:00:52.288859 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:00:52.288823 2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5wrkm\" (UniqueName: \"kubernetes.io/projected/09b684b8-991c-4478-9c29-c24a57d0e259-kube-api-access-5wrkm\") on node \"ip-10-0-134-42.ec2.internal\" DevicePath \"\"" Jul 07 15:00:52.288859 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:00:52.288852 2557 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/09b684b8-991c-4478-9c29-c24a57d0e259-maas-api-tls\") on node \"ip-10-0-134-42.ec2.internal\" DevicePath \"\"" Jul 07 15:00:52.288859 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:00:52.288862 2557 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/09b684b8-991c-4478-9c29-c24a57d0e259-openshift-service-ca\") on node \"ip-10-0-134-42.ec2.internal\" DevicePath \"\"" Jul 07 15:00:52.301764 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:00:52.301737 2557 generic.go:356] "Generic (PLEG): container finished" podID="09b684b8-991c-4478-9c29-c24a57d0e259" containerID="59fb53f206e502cb512dcf72b75a813f691046866c52f3201307ff53d582ee2f" exitCode=0 Jul 07 15:00:52.301904 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:00:52.301784 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-b-54f2c7-89cd6645-85vnz" Jul 07 15:00:52.301904 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:00:52.301806 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-b-54f2c7-89cd6645-85vnz" event={"ID":"09b684b8-991c-4478-9c29-c24a57d0e259","Type":"ContainerDied","Data":"59fb53f206e502cb512dcf72b75a813f691046866c52f3201307ff53d582ee2f"} Jul 07 15:00:52.301904 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:00:52.301851 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-b-54f2c7-89cd6645-85vnz" event={"ID":"09b684b8-991c-4478-9c29-c24a57d0e259","Type":"ContainerDied","Data":"771e1642eebf069354b75f3af28855791ac1e91b5b8bcb955743bc181ca4878c"} Jul 07 15:00:52.301904 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:00:52.301872 2557 scope.go:117] "RemoveContainer" containerID="59fb53f206e502cb512dcf72b75a813f691046866c52f3201307ff53d582ee2f" Jul 07 15:00:52.311858 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:00:52.311836 2557 scope.go:117] "RemoveContainer" containerID="59fb53f206e502cb512dcf72b75a813f691046866c52f3201307ff53d582ee2f" Jul 07 15:00:52.312096 ip-10-0-134-42 kubenswrapper[2557]: E0707 15:00:52.312074 2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"59fb53f206e502cb512dcf72b75a813f691046866c52f3201307ff53d582ee2f\": container with ID starting with 59fb53f206e502cb512dcf72b75a813f691046866c52f3201307ff53d582ee2f not found: ID does not exist" containerID="59fb53f206e502cb512dcf72b75a813f691046866c52f3201307ff53d582ee2f" Jul 07 15:00:52.312160 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:00:52.312106 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"59fb53f206e502cb512dcf72b75a813f691046866c52f3201307ff53d582ee2f"} err="failed to get container status \"59fb53f206e502cb512dcf72b75a813f691046866c52f3201307ff53d582ee2f\": rpc error: code = NotFound desc = could not find container \"59fb53f206e502cb512dcf72b75a813f691046866c52f3201307ff53d582ee2f\": container with ID starting with 59fb53f206e502cb512dcf72b75a813f691046866c52f3201307ff53d582ee2f not found: ID does not exist" Jul 07 15:00:52.323033 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:00:52.323006 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-inf-b-54f2c7-89cd6645-85vnz"] Jul 07 15:00:52.327406 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:00:52.327375 2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-inf-b-54f2c7-89cd6645-85vnz"] Jul 07 15:00:53.836782 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:00:53.836753 2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="09b684b8-991c-4478-9c29-c24a57d0e259" path="/var/lib/kubelet/pods/09b684b8-991c-4478-9c29-c24a57d0e259/volumes" Jul 07 15:00:58.703262 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:00:58.703222 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 15:00:58.817912 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:00:58.817879 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 15:00:59.050726 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:00:59.050628 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 15:00:59.274773 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:00:59.274741 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 15:00:59.806370 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:00:59.806342 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 15:01:00.036511 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:01:00.036479 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 15:01:00.199200 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:01:00.199167 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 15:01:02.709896 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:01:02.709856 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 15:01:02.771293 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:01:02.771263 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-api-a-18bb73-d8d657dff-rcp6m"] Jul 07 15:01:02.771721 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:01:02.771707 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="46fa2d39-183b-44f2-ac8b-216630a0fb60" containerName="cleanup" Jul 07 15:01:02.771721 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:01:02.771722 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="46fa2d39-183b-44f2-ac8b-216630a0fb60" containerName="cleanup" Jul 07 15:01:02.771825 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:01:02.771737 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="09b684b8-991c-4478-9c29-c24a57d0e259" containerName="maas-api" Jul 07 15:01:02.771825 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:01:02.771743 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="09b684b8-991c-4478-9c29-c24a57d0e259" containerName="maas-api" Jul 07 15:01:02.771825 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:01:02.771751 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="338a9d19-9258-4d24-b1ed-bcf984664ed3" containerName="cleanup" Jul 07 15:01:02.771825 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:01:02.771757 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="338a9d19-9258-4d24-b1ed-bcf984664ed3" containerName="cleanup" Jul 07 15:01:02.771825 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:01:02.771774 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="87bc723f-df5f-4e4b-892d-8fbbbf304e82" containerName="maas-api" Jul 07 15:01:02.771825 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:01:02.771779 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="87bc723f-df5f-4e4b-892d-8fbbbf304e82" containerName="maas-api" Jul 07 15:01:02.771825 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:01:02.771785 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="eff497ca-5cde-48bf-9101-43be457e22b4" containerName="cleanup" Jul 07 15:01:02.771825 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:01:02.771790 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="eff497ca-5cde-48bf-9101-43be457e22b4" containerName="cleanup" Jul 07 15:01:02.772074 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:01:02.771849 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="87bc723f-df5f-4e4b-892d-8fbbbf304e82" containerName="maas-api" Jul 07 15:01:02.772074 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:01:02.771855 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="eff497ca-5cde-48bf-9101-43be457e22b4" containerName="cleanup" Jul 07 15:01:02.772074 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:01:02.771864 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="46fa2d39-183b-44f2-ac8b-216630a0fb60" containerName="cleanup" Jul 07 15:01:02.772074 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:01:02.771870 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="338a9d19-9258-4d24-b1ed-bcf984664ed3" containerName="cleanup" Jul 07 15:01:02.772074 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:01:02.771877 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="09b684b8-991c-4478-9c29-c24a57d0e259" containerName="maas-api" Jul 07 15:01:02.776163 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:01:02.776140 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-a-18bb73-d8d657dff-rcp6m" Jul 07 15:01:02.778192 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:01:02.778171 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-api-a-18bb73-serving-cert\"" Jul 07 15:01:02.787683 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:01:02.787659 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-api-a-18bb73-d8d657dff-rcp6m"] Jul 07 15:01:02.790083 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:01:02.790054 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 15:01:02.795166 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:01:02.795143 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f01e3e3a-2b69-41d0-9d0a-614ee6cbaafd-maas-api-tls\") pod \"maas-api-e2e-api-a-18bb73-d8d657dff-rcp6m\" (UID: \"f01e3e3a-2b69-41d0-9d0a-614ee6cbaafd\") " pod="opendatahub/maas-api-e2e-api-a-18bb73-d8d657dff-rcp6m" Jul 07 15:01:02.795260 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:01:02.795201 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fl7wk\" (UniqueName: \"kubernetes.io/projected/f01e3e3a-2b69-41d0-9d0a-614ee6cbaafd-kube-api-access-fl7wk\") pod \"maas-api-e2e-api-a-18bb73-d8d657dff-rcp6m\" (UID: \"f01e3e3a-2b69-41d0-9d0a-614ee6cbaafd\") " pod="opendatahub/maas-api-e2e-api-a-18bb73-d8d657dff-rcp6m" Jul 07 15:01:02.795319 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:01:02.795278 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f01e3e3a-2b69-41d0-9d0a-614ee6cbaafd-openshift-service-ca\") pod \"maas-api-e2e-api-a-18bb73-d8d657dff-rcp6m\" (UID: \"f01e3e3a-2b69-41d0-9d0a-614ee6cbaafd\") " pod="opendatahub/maas-api-e2e-api-a-18bb73-d8d657dff-rcp6m" Jul 07 15:01:02.870384 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:01:02.870347 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 15:01:02.896714 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:01:02.896684 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f01e3e3a-2b69-41d0-9d0a-614ee6cbaafd-openshift-service-ca\") pod \"maas-api-e2e-api-a-18bb73-d8d657dff-rcp6m\" (UID: \"f01e3e3a-2b69-41d0-9d0a-614ee6cbaafd\") " pod="opendatahub/maas-api-e2e-api-a-18bb73-d8d657dff-rcp6m" Jul 07 15:01:02.896887 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:01:02.896780 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f01e3e3a-2b69-41d0-9d0a-614ee6cbaafd-maas-api-tls\") pod \"maas-api-e2e-api-a-18bb73-d8d657dff-rcp6m\" (UID: \"f01e3e3a-2b69-41d0-9d0a-614ee6cbaafd\") " pod="opendatahub/maas-api-e2e-api-a-18bb73-d8d657dff-rcp6m" Jul 07 15:01:02.896887 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:01:02.896821 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fl7wk\" (UniqueName: \"kubernetes.io/projected/f01e3e3a-2b69-41d0-9d0a-614ee6cbaafd-kube-api-access-fl7wk\") pod \"maas-api-e2e-api-a-18bb73-d8d657dff-rcp6m\" (UID: \"f01e3e3a-2b69-41d0-9d0a-614ee6cbaafd\") " pod="opendatahub/maas-api-e2e-api-a-18bb73-d8d657dff-rcp6m" Jul 07 15:01:02.896956 ip-10-0-134-42 kubenswrapper[2557]: E0707 15:01:02.896924 2557 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-api-a-18bb73-serving-cert: secret "maas-api-e2e-api-a-18bb73-serving-cert" not found Jul 07 15:01:02.896997 ip-10-0-134-42 kubenswrapper[2557]: E0707 15:01:02.896986 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f01e3e3a-2b69-41d0-9d0a-614ee6cbaafd-maas-api-tls podName:f01e3e3a-2b69-41d0-9d0a-614ee6cbaafd nodeName:}" failed. No retries permitted until 2026-07-07 15:01:03.396968044 +0000 UTC m=+2100.106106416 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/f01e3e3a-2b69-41d0-9d0a-614ee6cbaafd-maas-api-tls") pod "maas-api-e2e-api-a-18bb73-d8d657dff-rcp6m" (UID: "f01e3e3a-2b69-41d0-9d0a-614ee6cbaafd") : secret "maas-api-e2e-api-a-18bb73-serving-cert" not found Jul 07 15:01:02.897331 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:01:02.897314 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f01e3e3a-2b69-41d0-9d0a-614ee6cbaafd-openshift-service-ca\") pod \"maas-api-e2e-api-a-18bb73-d8d657dff-rcp6m\" (UID: \"f01e3e3a-2b69-41d0-9d0a-614ee6cbaafd\") " pod="opendatahub/maas-api-e2e-api-a-18bb73-d8d657dff-rcp6m" Jul 07 15:01:02.908661 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:01:02.908636 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fl7wk\" (UniqueName: \"kubernetes.io/projected/f01e3e3a-2b69-41d0-9d0a-614ee6cbaafd-kube-api-access-fl7wk\") pod \"maas-api-e2e-api-a-18bb73-d8d657dff-rcp6m\" (UID: \"f01e3e3a-2b69-41d0-9d0a-614ee6cbaafd\") " pod="opendatahub/maas-api-e2e-api-a-18bb73-d8d657dff-rcp6m" Jul 07 15:01:03.084189 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:01:03.084156 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 15:01:03.309808 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:01:03.309778 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 15:01:03.402575 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:01:03.402469 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 15:01:03.402921 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:01:03.402898 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f01e3e3a-2b69-41d0-9d0a-614ee6cbaafd-maas-api-tls\") pod \"maas-api-e2e-api-a-18bb73-d8d657dff-rcp6m\" (UID: \"f01e3e3a-2b69-41d0-9d0a-614ee6cbaafd\") " pod="opendatahub/maas-api-e2e-api-a-18bb73-d8d657dff-rcp6m" Jul 07 15:01:03.405279 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:01:03.405254 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f01e3e3a-2b69-41d0-9d0a-614ee6cbaafd-maas-api-tls\") pod \"maas-api-e2e-api-a-18bb73-d8d657dff-rcp6m\" (UID: \"f01e3e3a-2b69-41d0-9d0a-614ee6cbaafd\") " pod="opendatahub/maas-api-e2e-api-a-18bb73-d8d657dff-rcp6m" Jul 07 15:01:03.688435 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:01:03.688349 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-a-18bb73-d8d657dff-rcp6m" Jul 07 15:01:03.847148 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:01:03.847119 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-api-a-18bb73-d8d657dff-rcp6m"] Jul 07 15:01:03.848309 ip-10-0-134-42 kubenswrapper[2557]: W0707 15:01:03.848288 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf01e3e3a_2b69_41d0_9d0a_614ee6cbaafd.slice/crio-f9debef66f530d0126340a2ab82c87ca260ac65fa1520cba788b3dfae698db77 WatchSource:0}: Error finding container f9debef66f530d0126340a2ab82c87ca260ac65fa1520cba788b3dfae698db77: Status 404 returned error can't find the container with id f9debef66f530d0126340a2ab82c87ca260ac65fa1520cba788b3dfae698db77 Jul 07 15:01:04.379627 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:01:04.379590 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-a-18bb73-d8d657dff-rcp6m" event={"ID":"f01e3e3a-2b69-41d0-9d0a-614ee6cbaafd","Type":"ContainerStarted","Data":"f9debef66f530d0126340a2ab82c87ca260ac65fa1520cba788b3dfae698db77"} Jul 07 15:01:05.387987 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:01:05.387954 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-a-18bb73-d8d657dff-rcp6m" event={"ID":"f01e3e3a-2b69-41d0-9d0a-614ee6cbaafd","Type":"ContainerStarted","Data":"379bc887003a3eae8403d8e4bd37c451b7c746108344fa2ff8a17018aa46e612"} Jul 07 15:01:05.388371 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:01:05.388017 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-api-a-18bb73-d8d657dff-rcp6m" Jul 07 15:01:05.412140 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:01:05.412093 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-api-a-18bb73-d8d657dff-rcp6m" podStartSLOduration=2.531909567 podStartE2EDuration="3.412079174s" totalImagesPullingTime="880.169607ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:01:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:01:05.411246363 +0000 UTC m=+2102.120384743" watchObservedRunningTime="2026-07-07 15:01:05.412079174 +0000 UTC m=+2102.121217554" Jul 07 15:01:05.673865 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:01:05.673772 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-vd9g9_0fa80df4-c933-4758-b75f-3f70b7217c0d/console-operator/2.log" Jul 07 15:01:05.684901 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:01:05.684876 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-scfg4_90d2dce1-be42-4595-b9c4-74a024eef0e9/ovn-acl-logging/0.log" Jul 07 15:01:05.685566 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:01:05.685542 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-vd9g9_0fa80df4-c933-4758-b75f-3f70b7217c0d/console-operator/2.log" Jul 07 15:01:05.690321 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:01:05.690303 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-scfg4_90d2dce1-be42-4595-b9c4-74a024eef0e9/ovn-acl-logging/0.log" Jul 07 15:01:07.240266 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:01:07.240230 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 15:01:07.753034 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:01:07.752996 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 15:01:08.094437 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:01:08.094394 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 15:01:08.547103 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:01:08.547025 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 15:01:10.724712 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:01:10.724674 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 15:01:10.799902 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:01:10.799868 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-api-b-90fba0-7c88f9fb4c-bm8r2"] Jul 07 15:01:10.803055 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:01:10.803030 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-b-90fba0-7c88f9fb4c-bm8r2" Jul 07 15:01:10.805037 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:01:10.805015 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-api-b-90fba0-serving-cert\"" Jul 07 15:01:10.813910 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:01:10.813890 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-api-b-90fba0-7c88f9fb4c-bm8r2"] Jul 07 15:01:10.824342 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:01:10.824322 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 15:01:10.869296 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:01:10.869265 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-84zkp\" (UniqueName: \"kubernetes.io/projected/87d8b102-ab6d-409a-ad48-31f33d302f3a-kube-api-access-84zkp\") pod \"maas-api-e2e-api-b-90fba0-7c88f9fb4c-bm8r2\" (UID: \"87d8b102-ab6d-409a-ad48-31f33d302f3a\") " pod="opendatahub/maas-api-e2e-api-b-90fba0-7c88f9fb4c-bm8r2" Jul 07 15:01:10.869407 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:01:10.869365 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/87d8b102-ab6d-409a-ad48-31f33d302f3a-openshift-service-ca\") pod \"maas-api-e2e-api-b-90fba0-7c88f9fb4c-bm8r2\" (UID: \"87d8b102-ab6d-409a-ad48-31f33d302f3a\") " pod="opendatahub/maas-api-e2e-api-b-90fba0-7c88f9fb4c-bm8r2" Jul 07 15:01:10.869443 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:01:10.869417 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/87d8b102-ab6d-409a-ad48-31f33d302f3a-maas-api-tls\") pod \"maas-api-e2e-api-b-90fba0-7c88f9fb4c-bm8r2\" (UID: \"87d8b102-ab6d-409a-ad48-31f33d302f3a\") " pod="opendatahub/maas-api-e2e-api-b-90fba0-7c88f9fb4c-bm8r2" Jul 07 15:01:10.969904 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:01:10.969872 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-84zkp\" (UniqueName: \"kubernetes.io/projected/87d8b102-ab6d-409a-ad48-31f33d302f3a-kube-api-access-84zkp\") pod \"maas-api-e2e-api-b-90fba0-7c88f9fb4c-bm8r2\" (UID: \"87d8b102-ab6d-409a-ad48-31f33d302f3a\") " pod="opendatahub/maas-api-e2e-api-b-90fba0-7c88f9fb4c-bm8r2" Jul 07 15:01:10.970057 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:01:10.969922 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/87d8b102-ab6d-409a-ad48-31f33d302f3a-openshift-service-ca\") pod \"maas-api-e2e-api-b-90fba0-7c88f9fb4c-bm8r2\" (UID: \"87d8b102-ab6d-409a-ad48-31f33d302f3a\") " pod="opendatahub/maas-api-e2e-api-b-90fba0-7c88f9fb4c-bm8r2" Jul 07 15:01:10.970057 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:01:10.969962 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/87d8b102-ab6d-409a-ad48-31f33d302f3a-maas-api-tls\") pod \"maas-api-e2e-api-b-90fba0-7c88f9fb4c-bm8r2\" (UID: \"87d8b102-ab6d-409a-ad48-31f33d302f3a\") " pod="opendatahub/maas-api-e2e-api-b-90fba0-7c88f9fb4c-bm8r2" Jul 07 15:01:10.970134 ip-10-0-134-42 kubenswrapper[2557]: E0707 15:01:10.970066 2557 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-api-b-90fba0-serving-cert: secret "maas-api-e2e-api-b-90fba0-serving-cert" not found Jul 07 15:01:10.970134 ip-10-0-134-42 kubenswrapper[2557]: E0707 15:01:10.970120 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/87d8b102-ab6d-409a-ad48-31f33d302f3a-maas-api-tls podName:87d8b102-ab6d-409a-ad48-31f33d302f3a nodeName:}" failed. No retries permitted until 2026-07-07 15:01:11.470102391 +0000 UTC m=+2108.179240759 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/87d8b102-ab6d-409a-ad48-31f33d302f3a-maas-api-tls") pod "maas-api-e2e-api-b-90fba0-7c88f9fb4c-bm8r2" (UID: "87d8b102-ab6d-409a-ad48-31f33d302f3a") : secret "maas-api-e2e-api-b-90fba0-serving-cert" not found Jul 07 15:01:10.970572 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:01:10.970549 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/87d8b102-ab6d-409a-ad48-31f33d302f3a-openshift-service-ca\") pod \"maas-api-e2e-api-b-90fba0-7c88f9fb4c-bm8r2\" (UID: \"87d8b102-ab6d-409a-ad48-31f33d302f3a\") " pod="opendatahub/maas-api-e2e-api-b-90fba0-7c88f9fb4c-bm8r2" Jul 07 15:01:10.980252 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:01:10.980199 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-84zkp\" (UniqueName: \"kubernetes.io/projected/87d8b102-ab6d-409a-ad48-31f33d302f3a-kube-api-access-84zkp\") pod \"maas-api-e2e-api-b-90fba0-7c88f9fb4c-bm8r2\" (UID: \"87d8b102-ab6d-409a-ad48-31f33d302f3a\") " pod="opendatahub/maas-api-e2e-api-b-90fba0-7c88f9fb4c-bm8r2" Jul 07 15:01:11.125004 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:01:11.124966 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 15:01:11.348040 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:01:11.348006 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 15:01:11.399634 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:01:11.399610 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-api-a-18bb73-d8d657dff-rcp6m" Jul 07 15:01:11.423622 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:01:11.423596 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 15:01:11.475850 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:01:11.475817 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/87d8b102-ab6d-409a-ad48-31f33d302f3a-maas-api-tls\") pod \"maas-api-e2e-api-b-90fba0-7c88f9fb4c-bm8r2\" (UID: \"87d8b102-ab6d-409a-ad48-31f33d302f3a\") " pod="opendatahub/maas-api-e2e-api-b-90fba0-7c88f9fb4c-bm8r2" Jul 07 15:01:11.478051 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:01:11.478028 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/87d8b102-ab6d-409a-ad48-31f33d302f3a-maas-api-tls\") pod \"maas-api-e2e-api-b-90fba0-7c88f9fb4c-bm8r2\" (UID: \"87d8b102-ab6d-409a-ad48-31f33d302f3a\") " pod="opendatahub/maas-api-e2e-api-b-90fba0-7c88f9fb4c-bm8r2" Jul 07 15:01:11.717107 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:01:11.717078 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-b-90fba0-7c88f9fb4c-bm8r2" Jul 07 15:01:11.857502 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:01:11.857473 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-api-b-90fba0-7c88f9fb4c-bm8r2"] Jul 07 15:01:11.858982 ip-10-0-134-42 kubenswrapper[2557]: W0707 15:01:11.858956 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod87d8b102_ab6d_409a_ad48_31f33d302f3a.slice/crio-2a1db1319d53c95978e2073fe146e300b677010ac22095c4e26fe3a69d8e956f WatchSource:0}: Error finding container 2a1db1319d53c95978e2073fe146e300b677010ac22095c4e26fe3a69d8e956f: Status 404 returned error can't find the container with id 2a1db1319d53c95978e2073fe146e300b677010ac22095c4e26fe3a69d8e956f Jul 07 15:01:12.430066 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:01:12.430031 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-b-90fba0-7c88f9fb4c-bm8r2" event={"ID":"87d8b102-ab6d-409a-ad48-31f33d302f3a","Type":"ContainerStarted","Data":"2a1db1319d53c95978e2073fe146e300b677010ac22095c4e26fe3a69d8e956f"} Jul 07 15:01:13.437241 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:01:13.437212 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-b-90fba0-7c88f9fb4c-bm8r2" event={"ID":"87d8b102-ab6d-409a-ad48-31f33d302f3a","Type":"ContainerStarted","Data":"99d89737d573efff7d5224c7468929065a3d81459673cbdacc43fd4afb2bc6c5"} Jul 07 15:01:13.437673 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:01:13.437322 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-api-b-90fba0-7c88f9fb4c-bm8r2" Jul 07 15:01:13.456589 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:01:13.456543 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-api-b-90fba0-7c88f9fb4c-bm8r2" podStartSLOduration=2.460472815 podStartE2EDuration="3.456509343s" totalImagesPullingTime="996.036528ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:01:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:01:13.454898866 +0000 UTC m=+2110.164037246" watchObservedRunningTime="2026-07-07 15:01:13.456509343 +0000 UTC m=+2110.165647722" Jul 07 15:01:15.312568 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:01:15.312539 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 15:01:19.448442 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:01:19.448415 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-api-b-90fba0-7c88f9fb4c-bm8r2" Jul 07 15:01:23.628462 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:01:23.628426 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 15:01:23.984459 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:01:23.984379 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 15:01:24.083717 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:01:24.083689 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 15:01:24.212951 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:01:24.212919 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-api-a-18bb73-d8d657dff-rcp6m"] Jul 07 15:01:24.213154 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:01:24.213131 2557 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-api-a-18bb73-d8d657dff-rcp6m" podUID="f01e3e3a-2b69-41d0-9d0a-614ee6cbaafd" containerName="maas-api" containerID="cri-o://379bc887003a3eae8403d8e4bd37c451b7c746108344fa2ff8a17018aa46e612" gracePeriod=30 Jul 07 15:01:24.241630 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:01:24.241567 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 15:01:24.463240 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:01:24.463212 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-a-18bb73-d8d657dff-rcp6m" Jul 07 15:01:24.504369 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:01:24.504298 2557 generic.go:356] "Generic (PLEG): container finished" podID="f01e3e3a-2b69-41d0-9d0a-614ee6cbaafd" containerID="379bc887003a3eae8403d8e4bd37c451b7c746108344fa2ff8a17018aa46e612" exitCode=0 Jul 07 15:01:24.504369 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:01:24.504345 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-a-18bb73-d8d657dff-rcp6m" Jul 07 15:01:24.504572 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:01:24.504382 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-a-18bb73-d8d657dff-rcp6m" event={"ID":"f01e3e3a-2b69-41d0-9d0a-614ee6cbaafd","Type":"ContainerDied","Data":"379bc887003a3eae8403d8e4bd37c451b7c746108344fa2ff8a17018aa46e612"} Jul 07 15:01:24.504572 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:01:24.504434 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-a-18bb73-d8d657dff-rcp6m" event={"ID":"f01e3e3a-2b69-41d0-9d0a-614ee6cbaafd","Type":"ContainerDied","Data":"f9debef66f530d0126340a2ab82c87ca260ac65fa1520cba788b3dfae698db77"} Jul 07 15:01:24.504572 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:01:24.504457 2557 scope.go:117] "RemoveContainer" containerID="379bc887003a3eae8403d8e4bd37c451b7c746108344fa2ff8a17018aa46e612" Jul 07 15:01:24.513345 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:01:24.513327 2557 scope.go:117] "RemoveContainer" containerID="379bc887003a3eae8403d8e4bd37c451b7c746108344fa2ff8a17018aa46e612" Jul 07 15:01:24.513658 ip-10-0-134-42 kubenswrapper[2557]: E0707 15:01:24.513639 2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"379bc887003a3eae8403d8e4bd37c451b7c746108344fa2ff8a17018aa46e612\": container with ID starting with 379bc887003a3eae8403d8e4bd37c451b7c746108344fa2ff8a17018aa46e612 not found: ID does not exist" containerID="379bc887003a3eae8403d8e4bd37c451b7c746108344fa2ff8a17018aa46e612" Jul 07 15:01:24.513718 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:01:24.513669 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"379bc887003a3eae8403d8e4bd37c451b7c746108344fa2ff8a17018aa46e612"} err="failed to get container status \"379bc887003a3eae8403d8e4bd37c451b7c746108344fa2ff8a17018aa46e612\": rpc error: code = NotFound desc = could not find container \"379bc887003a3eae8403d8e4bd37c451b7c746108344fa2ff8a17018aa46e612\": container with ID starting with 379bc887003a3eae8403d8e4bd37c451b7c746108344fa2ff8a17018aa46e612 not found: ID does not exist" Jul 07 15:01:24.598851 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:01:24.598832 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f01e3e3a-2b69-41d0-9d0a-614ee6cbaafd-maas-api-tls\") pod \"f01e3e3a-2b69-41d0-9d0a-614ee6cbaafd\" (UID: \"f01e3e3a-2b69-41d0-9d0a-614ee6cbaafd\") " Jul 07 15:01:24.598982 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:01:24.598880 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f01e3e3a-2b69-41d0-9d0a-614ee6cbaafd-openshift-service-ca\") pod \"f01e3e3a-2b69-41d0-9d0a-614ee6cbaafd\" (UID: \"f01e3e3a-2b69-41d0-9d0a-614ee6cbaafd\") " Jul 07 15:01:24.598982 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:01:24.598926 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fl7wk\" (UniqueName: \"kubernetes.io/projected/f01e3e3a-2b69-41d0-9d0a-614ee6cbaafd-kube-api-access-fl7wk\") pod \"f01e3e3a-2b69-41d0-9d0a-614ee6cbaafd\" (UID: \"f01e3e3a-2b69-41d0-9d0a-614ee6cbaafd\") " Jul 07 15:01:24.599219 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:01:24.599200 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f01e3e3a-2b69-41d0-9d0a-614ee6cbaafd-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "f01e3e3a-2b69-41d0-9d0a-614ee6cbaafd" (UID: "f01e3e3a-2b69-41d0-9d0a-614ee6cbaafd"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 15:01:24.600935 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:01:24.600914 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f01e3e3a-2b69-41d0-9d0a-614ee6cbaafd-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "f01e3e3a-2b69-41d0-9d0a-614ee6cbaafd" (UID: "f01e3e3a-2b69-41d0-9d0a-614ee6cbaafd"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 15:01:24.601001 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:01:24.600913 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f01e3e3a-2b69-41d0-9d0a-614ee6cbaafd-kube-api-access-fl7wk" (OuterVolumeSpecName: "kube-api-access-fl7wk") pod "f01e3e3a-2b69-41d0-9d0a-614ee6cbaafd" (UID: "f01e3e3a-2b69-41d0-9d0a-614ee6cbaafd"). InnerVolumeSpecName "kube-api-access-fl7wk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 15:01:24.699703 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:01:24.699669 2557 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f01e3e3a-2b69-41d0-9d0a-614ee6cbaafd-maas-api-tls\") on node \"ip-10-0-134-42.ec2.internal\" DevicePath \"\"" Jul 07 15:01:24.699703 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:01:24.699697 2557 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f01e3e3a-2b69-41d0-9d0a-614ee6cbaafd-openshift-service-ca\") on node \"ip-10-0-134-42.ec2.internal\" DevicePath \"\"" Jul 07 15:01:24.699703 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:01:24.699709 2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fl7wk\" (UniqueName: \"kubernetes.io/projected/f01e3e3a-2b69-41d0-9d0a-614ee6cbaafd-kube-api-access-fl7wk\") on node \"ip-10-0-134-42.ec2.internal\" DevicePath \"\"" Jul 07 15:01:24.829395 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:01:24.829368 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-api-a-18bb73-d8d657dff-rcp6m"] Jul 07 15:01:24.834002 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:01:24.833980 2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-api-a-18bb73-d8d657dff-rcp6m"] Jul 07 15:01:25.837033 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:01:25.837004 2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f01e3e3a-2b69-41d0-9d0a-614ee6cbaafd" path="/var/lib/kubelet/pods/f01e3e3a-2b69-41d0-9d0a-614ee6cbaafd/volumes" Jul 07 15:01:31.103935 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:01:31.103906 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 15:01:31.216055 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:01:31.216024 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 15:01:31.482594 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:01:31.482491 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 15:01:31.665187 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:01:31.665148 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 15:01:31.975868 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:01:31.975840 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 15:01:32.315408 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:01:32.315332 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 15:01:32.479147 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:01:32.479107 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 15:01:32.579249 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:01:32.579213 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 15:01:32.718626 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:01:32.718595 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-api-b-90fba0-7c88f9fb4c-bm8r2"] Jul 07 15:01:32.718837 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:01:32.718817 2557 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-api-b-90fba0-7c88f9fb4c-bm8r2" podUID="87d8b102-ab6d-409a-ad48-31f33d302f3a" containerName="maas-api" containerID="cri-o://99d89737d573efff7d5224c7468929065a3d81459673cbdacc43fd4afb2bc6c5" gracePeriod=30 Jul 07 15:01:32.763996 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:01:32.763972 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 15:01:32.944997 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:01:32.944963 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 15:01:32.966966 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:01:32.966944 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-b-90fba0-7c88f9fb4c-bm8r2" Jul 07 15:01:33.078107 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:01:33.078083 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/87d8b102-ab6d-409a-ad48-31f33d302f3a-maas-api-tls\") pod \"87d8b102-ab6d-409a-ad48-31f33d302f3a\" (UID: \"87d8b102-ab6d-409a-ad48-31f33d302f3a\") " Jul 07 15:01:33.078260 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:01:33.078151 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/87d8b102-ab6d-409a-ad48-31f33d302f3a-openshift-service-ca\") pod \"87d8b102-ab6d-409a-ad48-31f33d302f3a\" (UID: \"87d8b102-ab6d-409a-ad48-31f33d302f3a\") " Jul 07 15:01:33.078260 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:01:33.078240 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-84zkp\" (UniqueName: \"kubernetes.io/projected/87d8b102-ab6d-409a-ad48-31f33d302f3a-kube-api-access-84zkp\") pod \"87d8b102-ab6d-409a-ad48-31f33d302f3a\" (UID: \"87d8b102-ab6d-409a-ad48-31f33d302f3a\") " Jul 07 15:01:33.078495 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:01:33.078473 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/87d8b102-ab6d-409a-ad48-31f33d302f3a-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "87d8b102-ab6d-409a-ad48-31f33d302f3a" (UID: "87d8b102-ab6d-409a-ad48-31f33d302f3a"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 15:01:33.078608 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:01:33.078592 2557 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/87d8b102-ab6d-409a-ad48-31f33d302f3a-openshift-service-ca\") on node \"ip-10-0-134-42.ec2.internal\" DevicePath \"\"" Jul 07 15:01:33.080068 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:01:33.080044 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/87d8b102-ab6d-409a-ad48-31f33d302f3a-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "87d8b102-ab6d-409a-ad48-31f33d302f3a" (UID: "87d8b102-ab6d-409a-ad48-31f33d302f3a"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 15:01:33.080180 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:01:33.080127 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/87d8b102-ab6d-409a-ad48-31f33d302f3a-kube-api-access-84zkp" (OuterVolumeSpecName: "kube-api-access-84zkp") pod "87d8b102-ab6d-409a-ad48-31f33d302f3a" (UID: "87d8b102-ab6d-409a-ad48-31f33d302f3a"). InnerVolumeSpecName "kube-api-access-84zkp". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 15:01:33.179963 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:01:33.179934 2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-84zkp\" (UniqueName: \"kubernetes.io/projected/87d8b102-ab6d-409a-ad48-31f33d302f3a-kube-api-access-84zkp\") on node \"ip-10-0-134-42.ec2.internal\" DevicePath \"\"" Jul 07 15:01:33.179963 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:01:33.179959 2557 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/87d8b102-ab6d-409a-ad48-31f33d302f3a-maas-api-tls\") on node \"ip-10-0-134-42.ec2.internal\" DevicePath \"\"" Jul 07 15:01:33.557775 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:01:33.557692 2557 generic.go:356] "Generic (PLEG): container finished" podID="87d8b102-ab6d-409a-ad48-31f33d302f3a" containerID="99d89737d573efff7d5224c7468929065a3d81459673cbdacc43fd4afb2bc6c5" exitCode=0 Jul 07 15:01:33.557775 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:01:33.557740 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-b-90fba0-7c88f9fb4c-bm8r2" Jul 07 15:01:33.558167 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:01:33.557775 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-b-90fba0-7c88f9fb4c-bm8r2" event={"ID":"87d8b102-ab6d-409a-ad48-31f33d302f3a","Type":"ContainerDied","Data":"99d89737d573efff7d5224c7468929065a3d81459673cbdacc43fd4afb2bc6c5"} Jul 07 15:01:33.558167 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:01:33.557824 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-b-90fba0-7c88f9fb4c-bm8r2" event={"ID":"87d8b102-ab6d-409a-ad48-31f33d302f3a","Type":"ContainerDied","Data":"2a1db1319d53c95978e2073fe146e300b677010ac22095c4e26fe3a69d8e956f"} Jul 07 15:01:33.558167 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:01:33.557845 2557 scope.go:117] "RemoveContainer" containerID="99d89737d573efff7d5224c7468929065a3d81459673cbdacc43fd4afb2bc6c5" Jul 07 15:01:33.567298 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:01:33.567282 2557 scope.go:117] "RemoveContainer" containerID="99d89737d573efff7d5224c7468929065a3d81459673cbdacc43fd4afb2bc6c5" Jul 07 15:01:33.567556 ip-10-0-134-42 kubenswrapper[2557]: E0707 15:01:33.567537 2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"99d89737d573efff7d5224c7468929065a3d81459673cbdacc43fd4afb2bc6c5\": container with ID starting with 99d89737d573efff7d5224c7468929065a3d81459673cbdacc43fd4afb2bc6c5 not found: ID does not exist" containerID="99d89737d573efff7d5224c7468929065a3d81459673cbdacc43fd4afb2bc6c5" Jul 07 15:01:33.567640 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:01:33.567563 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"99d89737d573efff7d5224c7468929065a3d81459673cbdacc43fd4afb2bc6c5"} err="failed to get container status \"99d89737d573efff7d5224c7468929065a3d81459673cbdacc43fd4afb2bc6c5\": rpc error: code = NotFound desc = could not find container \"99d89737d573efff7d5224c7468929065a3d81459673cbdacc43fd4afb2bc6c5\": container with ID starting with 99d89737d573efff7d5224c7468929065a3d81459673cbdacc43fd4afb2bc6c5 not found: ID does not exist" Jul 07 15:01:33.593099 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:01:33.593075 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-api-b-90fba0-7c88f9fb4c-bm8r2"] Jul 07 15:01:33.601936 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:01:33.601917 2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-api-b-90fba0-7c88f9fb4c-bm8r2"] Jul 07 15:01:33.836130 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:01:33.836105 2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="87d8b102-ab6d-409a-ad48-31f33d302f3a" path="/var/lib/kubelet/pods/87d8b102-ab6d-409a-ad48-31f33d302f3a/volumes" Jul 07 15:01:37.427949 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:01:37.427912 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 15:01:39.056011 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:01:39.055975 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 15:01:39.139835 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:01:39.139805 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 15:01:39.600158 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:01:39.600117 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 15:01:39.800557 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:01:39.800512 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 15:01:39.926374 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:01:39.926298 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 15:01:40.252830 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:01:40.252750 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 15:01:40.458652 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:01:40.458618 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 15:01:42.944397 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:01:42.944361 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 15:01:43.005092 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:01:43.005063 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-shared-a-26fa43-55b7ccb857-5qrln"] Jul 07 15:01:43.005475 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:01:43.005460 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f01e3e3a-2b69-41d0-9d0a-614ee6cbaafd" containerName="maas-api" Jul 07 15:01:43.005475 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:01:43.005474 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="f01e3e3a-2b69-41d0-9d0a-614ee6cbaafd" containerName="maas-api" Jul 07 15:01:43.005625 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:01:43.005484 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="87d8b102-ab6d-409a-ad48-31f33d302f3a" containerName="maas-api" Jul 07 15:01:43.005625 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:01:43.005489 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="87d8b102-ab6d-409a-ad48-31f33d302f3a" containerName="maas-api" Jul 07 15:01:43.005625 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:01:43.005578 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="f01e3e3a-2b69-41d0-9d0a-614ee6cbaafd" containerName="maas-api" Jul 07 15:01:43.005625 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:01:43.005589 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="87d8b102-ab6d-409a-ad48-31f33d302f3a" containerName="maas-api" Jul 07 15:01:43.008141 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:01:43.008125 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-a-26fa43-55b7ccb857-5qrln" Jul 07 15:01:43.010211 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:01:43.010189 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-shared-a-26fa43-serving-cert\"" Jul 07 15:01:43.017803 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:01:43.017781 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-shared-a-26fa43-55b7ccb857-5qrln"] Jul 07 15:01:43.024590 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:01:43.024563 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 15:01:43.063356 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:01:43.063321 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lphxp\" (UniqueName: \"kubernetes.io/projected/67490077-aae3-4b39-a59b-cdcc639c4684-kube-api-access-lphxp\") pod \"maas-api-e2e-shared-a-26fa43-55b7ccb857-5qrln\" (UID: \"67490077-aae3-4b39-a59b-cdcc639c4684\") " pod="opendatahub/maas-api-e2e-shared-a-26fa43-55b7ccb857-5qrln" Jul 07 15:01:43.063498 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:01:43.063384 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/67490077-aae3-4b39-a59b-cdcc639c4684-maas-api-tls\") pod \"maas-api-e2e-shared-a-26fa43-55b7ccb857-5qrln\" (UID: \"67490077-aae3-4b39-a59b-cdcc639c4684\") " pod="opendatahub/maas-api-e2e-shared-a-26fa43-55b7ccb857-5qrln" Jul 07 15:01:43.063498 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:01:43.063479 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/67490077-aae3-4b39-a59b-cdcc639c4684-openshift-service-ca\") pod \"maas-api-e2e-shared-a-26fa43-55b7ccb857-5qrln\" (UID: \"67490077-aae3-4b39-a59b-cdcc639c4684\") " pod="opendatahub/maas-api-e2e-shared-a-26fa43-55b7ccb857-5qrln" Jul 07 15:01:43.088561 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:01:43.088303 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 15:01:43.164563 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:01:43.164531 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/67490077-aae3-4b39-a59b-cdcc639c4684-openshift-service-ca\") pod \"maas-api-e2e-shared-a-26fa43-55b7ccb857-5qrln\" (UID: \"67490077-aae3-4b39-a59b-cdcc639c4684\") " pod="opendatahub/maas-api-e2e-shared-a-26fa43-55b7ccb857-5qrln" Jul 07 15:01:43.164730 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:01:43.164592 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lphxp\" (UniqueName: \"kubernetes.io/projected/67490077-aae3-4b39-a59b-cdcc639c4684-kube-api-access-lphxp\") pod \"maas-api-e2e-shared-a-26fa43-55b7ccb857-5qrln\" (UID: \"67490077-aae3-4b39-a59b-cdcc639c4684\") " pod="opendatahub/maas-api-e2e-shared-a-26fa43-55b7ccb857-5qrln" Jul 07 15:01:43.164730 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:01:43.164631 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/67490077-aae3-4b39-a59b-cdcc639c4684-maas-api-tls\") pod \"maas-api-e2e-shared-a-26fa43-55b7ccb857-5qrln\" (UID: \"67490077-aae3-4b39-a59b-cdcc639c4684\") " pod="opendatahub/maas-api-e2e-shared-a-26fa43-55b7ccb857-5qrln" Jul 07 15:01:43.164818 ip-10-0-134-42 kubenswrapper[2557]: E0707 15:01:43.164769 2557 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-shared-a-26fa43-serving-cert: secret "maas-api-e2e-shared-a-26fa43-serving-cert" not found Jul 07 15:01:43.164859 ip-10-0-134-42 kubenswrapper[2557]: E0707 15:01:43.164847 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/67490077-aae3-4b39-a59b-cdcc639c4684-maas-api-tls podName:67490077-aae3-4b39-a59b-cdcc639c4684 nodeName:}" failed. No retries permitted until 2026-07-07 15:01:43.664828401 +0000 UTC m=+2140.373966760 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/67490077-aae3-4b39-a59b-cdcc639c4684-maas-api-tls") pod "maas-api-e2e-shared-a-26fa43-55b7ccb857-5qrln" (UID: "67490077-aae3-4b39-a59b-cdcc639c4684") : secret "maas-api-e2e-shared-a-26fa43-serving-cert" not found Jul 07 15:01:43.165176 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:01:43.165157 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/67490077-aae3-4b39-a59b-cdcc639c4684-openshift-service-ca\") pod \"maas-api-e2e-shared-a-26fa43-55b7ccb857-5qrln\" (UID: \"67490077-aae3-4b39-a59b-cdcc639c4684\") " pod="opendatahub/maas-api-e2e-shared-a-26fa43-55b7ccb857-5qrln" Jul 07 15:01:43.172986 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:01:43.172962 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lphxp\" (UniqueName: \"kubernetes.io/projected/67490077-aae3-4b39-a59b-cdcc639c4684-kube-api-access-lphxp\") pod \"maas-api-e2e-shared-a-26fa43-55b7ccb857-5qrln\" (UID: \"67490077-aae3-4b39-a59b-cdcc639c4684\") " pod="opendatahub/maas-api-e2e-shared-a-26fa43-55b7ccb857-5qrln" Jul 07 15:01:43.478334 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:01:43.478303 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 15:01:43.576930 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:01:43.576882 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 15:01:43.667732 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:01:43.667703 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/67490077-aae3-4b39-a59b-cdcc639c4684-maas-api-tls\") pod \"maas-api-e2e-shared-a-26fa43-55b7ccb857-5qrln\" (UID: \"67490077-aae3-4b39-a59b-cdcc639c4684\") " pod="opendatahub/maas-api-e2e-shared-a-26fa43-55b7ccb857-5qrln" Jul 07 15:01:43.670018 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:01:43.669989 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/67490077-aae3-4b39-a59b-cdcc639c4684-maas-api-tls\") pod \"maas-api-e2e-shared-a-26fa43-55b7ccb857-5qrln\" (UID: \"67490077-aae3-4b39-a59b-cdcc639c4684\") " pod="opendatahub/maas-api-e2e-shared-a-26fa43-55b7ccb857-5qrln" Jul 07 15:01:43.919290 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:01:43.919255 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-a-26fa43-55b7ccb857-5qrln" Jul 07 15:01:44.060632 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:01:44.060607 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-shared-a-26fa43-55b7ccb857-5qrln"] Jul 07 15:01:44.062623 ip-10-0-134-42 kubenswrapper[2557]: W0707 15:01:44.062596 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod67490077_aae3_4b39_a59b_cdcc639c4684.slice/crio-62e6fff02135a3d993b84cdf0b2276fc09986649bb6962af62f839a3fd292f60 WatchSource:0}: Error finding container 62e6fff02135a3d993b84cdf0b2276fc09986649bb6962af62f839a3fd292f60: Status 404 returned error can't find the container with id 62e6fff02135a3d993b84cdf0b2276fc09986649bb6962af62f839a3fd292f60 Jul 07 15:01:44.624663 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:01:44.624629 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-a-26fa43-55b7ccb857-5qrln" event={"ID":"67490077-aae3-4b39-a59b-cdcc639c4684","Type":"ContainerStarted","Data":"62e6fff02135a3d993b84cdf0b2276fc09986649bb6962af62f839a3fd292f60"} Jul 07 15:01:45.248240 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:01:45.248210 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 15:01:45.347580 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:01:45.347550 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 15:01:45.631151 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:01:45.631120 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-a-26fa43-55b7ccb857-5qrln" event={"ID":"67490077-aae3-4b39-a59b-cdcc639c4684","Type":"ContainerStarted","Data":"39cef54083f583bf81c83dd529a06ae9a19fc064c74f0f820d919d9e43f7c095"} Jul 07 15:01:45.631329 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:01:45.631193 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-shared-a-26fa43-55b7ccb857-5qrln" Jul 07 15:01:45.649938 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:01:45.649896 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-shared-a-26fa43-55b7ccb857-5qrln" podStartSLOduration=2.5451192689999997 podStartE2EDuration="3.64988345s" totalImagesPullingTime="1.104764181s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:01:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:01:45.646911473 +0000 UTC m=+2142.356049891" watchObservedRunningTime="2026-07-07 15:01:45.64988345 +0000 UTC m=+2142.359021858" Jul 07 15:01:47.504490 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:01:47.504455 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 15:01:47.854807 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:01:47.854768 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 15:01:48.173431 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:01:48.173346 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 15:01:48.290987 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:01:48.290948 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 15:01:50.947587 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:01:50.947556 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 15:01:51.044300 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:01:51.044261 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 15:01:51.089802 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:01:51.089773 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-shared-b-2f8e54-65c96df586-v8mzc"] Jul 07 15:01:51.093122 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:01:51.093102 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-b-2f8e54-65c96df586-v8mzc" Jul 07 15:01:51.095578 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:01:51.095557 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-shared-b-2f8e54-serving-cert\"" Jul 07 15:01:51.104935 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:01:51.104911 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-shared-b-2f8e54-65c96df586-v8mzc"] Jul 07 15:01:51.127450 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:01:51.127412 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kchs2\" (UniqueName: \"kubernetes.io/projected/7067e7d0-5471-4dbf-967a-4b6ec4636b00-kube-api-access-kchs2\") pod \"maas-api-e2e-shared-b-2f8e54-65c96df586-v8mzc\" (UID: \"7067e7d0-5471-4dbf-967a-4b6ec4636b00\") " pod="opendatahub/maas-api-e2e-shared-b-2f8e54-65c96df586-v8mzc" Jul 07 15:01:51.127600 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:01:51.127458 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7067e7d0-5471-4dbf-967a-4b6ec4636b00-maas-api-tls\") pod \"maas-api-e2e-shared-b-2f8e54-65c96df586-v8mzc\" (UID: \"7067e7d0-5471-4dbf-967a-4b6ec4636b00\") " pod="opendatahub/maas-api-e2e-shared-b-2f8e54-65c96df586-v8mzc" Jul 07 15:01:51.127600 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:01:51.127564 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7067e7d0-5471-4dbf-967a-4b6ec4636b00-openshift-service-ca\") pod \"maas-api-e2e-shared-b-2f8e54-65c96df586-v8mzc\" (UID: \"7067e7d0-5471-4dbf-967a-4b6ec4636b00\") " pod="opendatahub/maas-api-e2e-shared-b-2f8e54-65c96df586-v8mzc" Jul 07 15:01:51.146354 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:01:51.146322 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 15:01:51.228888 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:01:51.228803 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kchs2\" (UniqueName: \"kubernetes.io/projected/7067e7d0-5471-4dbf-967a-4b6ec4636b00-kube-api-access-kchs2\") pod \"maas-api-e2e-shared-b-2f8e54-65c96df586-v8mzc\" (UID: \"7067e7d0-5471-4dbf-967a-4b6ec4636b00\") " pod="opendatahub/maas-api-e2e-shared-b-2f8e54-65c96df586-v8mzc" Jul 07 15:01:51.228888 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:01:51.228838 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7067e7d0-5471-4dbf-967a-4b6ec4636b00-maas-api-tls\") pod \"maas-api-e2e-shared-b-2f8e54-65c96df586-v8mzc\" (UID: \"7067e7d0-5471-4dbf-967a-4b6ec4636b00\") " pod="opendatahub/maas-api-e2e-shared-b-2f8e54-65c96df586-v8mzc" Jul 07 15:01:51.228888 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:01:51.228878 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7067e7d0-5471-4dbf-967a-4b6ec4636b00-openshift-service-ca\") pod \"maas-api-e2e-shared-b-2f8e54-65c96df586-v8mzc\" (UID: \"7067e7d0-5471-4dbf-967a-4b6ec4636b00\") " pod="opendatahub/maas-api-e2e-shared-b-2f8e54-65c96df586-v8mzc" Jul 07 15:01:51.229508 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:01:51.229482 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7067e7d0-5471-4dbf-967a-4b6ec4636b00-openshift-service-ca\") pod \"maas-api-e2e-shared-b-2f8e54-65c96df586-v8mzc\" (UID: \"7067e7d0-5471-4dbf-967a-4b6ec4636b00\") " pod="opendatahub/maas-api-e2e-shared-b-2f8e54-65c96df586-v8mzc" Jul 07 15:01:51.231322 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:01:51.231301 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7067e7d0-5471-4dbf-967a-4b6ec4636b00-maas-api-tls\") pod \"maas-api-e2e-shared-b-2f8e54-65c96df586-v8mzc\" (UID: \"7067e7d0-5471-4dbf-967a-4b6ec4636b00\") " pod="opendatahub/maas-api-e2e-shared-b-2f8e54-65c96df586-v8mzc" Jul 07 15:01:51.236597 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:01:51.236576 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kchs2\" (UniqueName: \"kubernetes.io/projected/7067e7d0-5471-4dbf-967a-4b6ec4636b00-kube-api-access-kchs2\") pod \"maas-api-e2e-shared-b-2f8e54-65c96df586-v8mzc\" (UID: \"7067e7d0-5471-4dbf-967a-4b6ec4636b00\") " pod="opendatahub/maas-api-e2e-shared-b-2f8e54-65c96df586-v8mzc" Jul 07 15:01:51.247249 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:01:51.247225 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 15:01:51.406391 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:01:51.406359 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-b-2f8e54-65c96df586-v8mzc" Jul 07 15:01:51.545046 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:01:51.545010 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-shared-b-2f8e54-65c96df586-v8mzc"] Jul 07 15:01:51.546151 ip-10-0-134-42 kubenswrapper[2557]: W0707 15:01:51.546126 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7067e7d0_5471_4dbf_967a_4b6ec4636b00.slice/crio-a49f542eb76dc8247cb5729f29de45f154ad9c3891c85cba357a380898074570 WatchSource:0}: Error finding container a49f542eb76dc8247cb5729f29de45f154ad9c3891c85cba357a380898074570: Status 404 returned error can't find the container with id a49f542eb76dc8247cb5729f29de45f154ad9c3891c85cba357a380898074570 Jul 07 15:01:51.643744 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:01:51.643723 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-shared-a-26fa43-55b7ccb857-5qrln" Jul 07 15:01:51.669927 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:01:51.669904 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-b-2f8e54-65c96df586-v8mzc" event={"ID":"7067e7d0-5471-4dbf-967a-4b6ec4636b00","Type":"ContainerStarted","Data":"a49f542eb76dc8247cb5729f29de45f154ad9c3891c85cba357a380898074570"} Jul 07 15:01:51.748103 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:01:51.748039 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 15:01:51.849316 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:01:51.849289 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 15:01:52.676760 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:01:52.676723 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-b-2f8e54-65c96df586-v8mzc" event={"ID":"7067e7d0-5471-4dbf-967a-4b6ec4636b00","Type":"ContainerStarted","Data":"8eb7a583266ca4c3697f94352c311270daa8984480416efa1c7fba5e180cb09f"} Jul 07 15:01:52.677258 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:01:52.676805 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-shared-b-2f8e54-65c96df586-v8mzc" Jul 07 15:01:52.697710 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:01:52.697667 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-shared-b-2f8e54-65c96df586-v8mzc" podStartSLOduration=1.30307529 podStartE2EDuration="1.697652601s" totalImagesPullingTime="394.577311ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:01:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:01:52.695939608 +0000 UTC m=+2149.405077992" watchObservedRunningTime="2026-07-07 15:01:52.697652601 +0000 UTC m=+2149.406790980" Jul 07 15:01:55.516875 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:01:55.516843 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 15:01:58.688559 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:01:58.688449 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-shared-b-2f8e54-65c96df586-v8mzc" Jul 07 15:02:04.185690 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:02:04.185654 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 15:02:04.286966 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:02:04.286937 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 15:02:07.197542 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:02:07.197484 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 15:02:13.216502 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:02:13.216470 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 15:02:13.316683 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:02:13.316646 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 15:02:29.365383 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:02:29.365345 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 15:03:57.068909 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:03:57.068802 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 15:04:25.587592 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:04:25.587553 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 15:04:26.555677 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:04:26.555642 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 15:04:37.474599 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:04:37.474567 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 15:05:00.764631 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:05:00.764547 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 15:05:36.308171 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:05:36.308131 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 15:05:37.599936 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:05:37.599900 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 15:05:38.588038 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:05:38.588003 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 15:05:43.319150 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:05:43.319113 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 15:06:05.719930 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:06:05.719895 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-vd9g9_0fa80df4-c933-4758-b75f-3f70b7217c0d/console-operator/2.log" Jul 07 15:06:05.724534 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:06:05.724498 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-scfg4_90d2dce1-be42-4595-b9c4-74a024eef0e9/ovn-acl-logging/0.log" Jul 07 15:06:05.726320 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:06:05.726299 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-vd9g9_0fa80df4-c933-4758-b75f-3f70b7217c0d/console-operator/2.log" Jul 07 15:06:05.730806 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:06:05.730789 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-scfg4_90d2dce1-be42-4595-b9c4-74a024eef0e9/ovn-acl-logging/0.log" Jul 07 15:06:05.782306 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:06:05.782281 2557 scope.go:117] "RemoveContainer" containerID="17e3a097fad9516d4ba9a965fcded096b892ee8c54de730259db7b3083bdf6db" Jul 07 15:06:05.790978 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:06:05.790963 2557 scope.go:117] "RemoveContainer" containerID="82b357bdddac4de299d92355f2e7d3e5d924f861e8cd13c75415e4347cebf6e1" Jul 07 15:06:11.472012 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:06:11.471970 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 15:06:11.561651 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:06:11.561625 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 15:06:24.974356 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:06:24.974313 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 15:06:46.088282 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:06:46.088245 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 15:06:46.190677 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:06:46.190640 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 15:06:54.059504 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:06:54.059472 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 15:06:54.159680 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:06:54.159650 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 15:07:04.949648 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:07:04.949611 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 15:07:05.049767 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:07:05.049733 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 15:07:10.154401 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:07:10.154366 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 15:07:10.973849 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:07:10.973816 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 15:07:12.096778 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:07:12.096748 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 15:07:12.392037 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:07:12.391960 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-shared-a-26fa43-55b7ccb857-5qrln"] Jul 07 15:07:12.392178 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:07:12.392154 2557 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-shared-a-26fa43-55b7ccb857-5qrln" podUID="67490077-aae3-4b39-a59b-cdcc639c4684" containerName="maas-api" containerID="cri-o://39cef54083f583bf81c83dd529a06ae9a19fc064c74f0f820d919d9e43f7c095" gracePeriod=30 Jul 07 15:07:12.582990 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:07:12.582955 2557 generic.go:356] "Generic (PLEG): container finished" podID="67490077-aae3-4b39-a59b-cdcc639c4684" containerID="39cef54083f583bf81c83dd529a06ae9a19fc064c74f0f820d919d9e43f7c095" exitCode=0 Jul 07 15:07:12.583127 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:07:12.583024 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-a-26fa43-55b7ccb857-5qrln" event={"ID":"67490077-aae3-4b39-a59b-cdcc639c4684","Type":"ContainerDied","Data":"39cef54083f583bf81c83dd529a06ae9a19fc064c74f0f820d919d9e43f7c095"} Jul 07 15:07:12.637176 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:07:12.637138 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 15:07:12.644500 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:07:12.644446 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-a-26fa43-55b7ccb857-5qrln" Jul 07 15:07:12.696994 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:07:12.696960 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 15:07:12.792165 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:07:12.792128 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/67490077-aae3-4b39-a59b-cdcc639c4684-maas-api-tls\") pod \"67490077-aae3-4b39-a59b-cdcc639c4684\" (UID: \"67490077-aae3-4b39-a59b-cdcc639c4684\") " Jul 07 15:07:12.792165 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:07:12.792167 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/67490077-aae3-4b39-a59b-cdcc639c4684-openshift-service-ca\") pod \"67490077-aae3-4b39-a59b-cdcc639c4684\" (UID: \"67490077-aae3-4b39-a59b-cdcc639c4684\") " Jul 07 15:07:12.792402 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:07:12.792386 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lphxp\" (UniqueName: \"kubernetes.io/projected/67490077-aae3-4b39-a59b-cdcc639c4684-kube-api-access-lphxp\") pod \"67490077-aae3-4b39-a59b-cdcc639c4684\" (UID: \"67490077-aae3-4b39-a59b-cdcc639c4684\") " Jul 07 15:07:12.792661 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:07:12.792628 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/67490077-aae3-4b39-a59b-cdcc639c4684-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "67490077-aae3-4b39-a59b-cdcc639c4684" (UID: "67490077-aae3-4b39-a59b-cdcc639c4684"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 15:07:12.792786 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:07:12.792733 2557 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/67490077-aae3-4b39-a59b-cdcc639c4684-openshift-service-ca\") on node \"ip-10-0-134-42.ec2.internal\" DevicePath \"\"" Jul 07 15:07:12.794356 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:07:12.794327 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/67490077-aae3-4b39-a59b-cdcc639c4684-kube-api-access-lphxp" (OuterVolumeSpecName: "kube-api-access-lphxp") pod "67490077-aae3-4b39-a59b-cdcc639c4684" (UID: "67490077-aae3-4b39-a59b-cdcc639c4684"). InnerVolumeSpecName "kube-api-access-lphxp". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 15:07:12.794443 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:07:12.794374 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/67490077-aae3-4b39-a59b-cdcc639c4684-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "67490077-aae3-4b39-a59b-cdcc639c4684" (UID: "67490077-aae3-4b39-a59b-cdcc639c4684"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 15:07:12.893267 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:07:12.893230 2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lphxp\" (UniqueName: \"kubernetes.io/projected/67490077-aae3-4b39-a59b-cdcc639c4684-kube-api-access-lphxp\") on node \"ip-10-0-134-42.ec2.internal\" DevicePath \"\"" Jul 07 15:07:12.893267 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:07:12.893262 2557 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/67490077-aae3-4b39-a59b-cdcc639c4684-maas-api-tls\") on node \"ip-10-0-134-42.ec2.internal\" DevicePath \"\"" Jul 07 15:07:13.591170 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:07:13.591142 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-a-26fa43-55b7ccb857-5qrln" Jul 07 15:07:13.591170 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:07:13.591152 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-a-26fa43-55b7ccb857-5qrln" event={"ID":"67490077-aae3-4b39-a59b-cdcc639c4684","Type":"ContainerDied","Data":"62e6fff02135a3d993b84cdf0b2276fc09986649bb6962af62f839a3fd292f60"} Jul 07 15:07:13.591686 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:07:13.591212 2557 scope.go:117] "RemoveContainer" containerID="39cef54083f583bf81c83dd529a06ae9a19fc064c74f0f820d919d9e43f7c095" Jul 07 15:07:13.612227 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:07:13.612201 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-shared-a-26fa43-55b7ccb857-5qrln"] Jul 07 15:07:13.618388 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:07:13.618365 2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-shared-a-26fa43-55b7ccb857-5qrln"] Jul 07 15:07:13.837091 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:07:13.837061 2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="67490077-aae3-4b39-a59b-cdcc639c4684" path="/var/lib/kubelet/pods/67490077-aae3-4b39-a59b-cdcc639c4684/volumes" Jul 07 15:07:13.965316 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:07:13.965187 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 15:07:14.064532 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:07:14.064491 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 15:07:17.540907 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:07:17.540843 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 15:07:47.457811 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:07:47.457777 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 15:07:47.707029 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:07:47.706992 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 15:07:47.988856 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:07:47.988821 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 15:07:48.105281 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:07:48.105253 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 15:07:48.402250 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:07:48.402209 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-shared-b-2f8e54-65c96df586-v8mzc"] Jul 07 15:07:48.402715 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:07:48.402443 2557 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-shared-b-2f8e54-65c96df586-v8mzc" podUID="7067e7d0-5471-4dbf-967a-4b6ec4636b00" containerName="maas-api" containerID="cri-o://8eb7a583266ca4c3697f94352c311270daa8984480416efa1c7fba5e180cb09f" gracePeriod=30 Jul 07 15:07:48.490384 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:07:48.490349 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 15:07:48.590830 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:07:48.590800 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 15:07:48.651688 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:07:48.651664 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-b-2f8e54-65c96df586-v8mzc" Jul 07 15:07:48.697323 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:07:48.697259 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7067e7d0-5471-4dbf-967a-4b6ec4636b00-openshift-service-ca\") pod \"7067e7d0-5471-4dbf-967a-4b6ec4636b00\" (UID: \"7067e7d0-5471-4dbf-967a-4b6ec4636b00\") " Jul 07 15:07:48.697445 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:07:48.697340 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7067e7d0-5471-4dbf-967a-4b6ec4636b00-maas-api-tls\") pod \"7067e7d0-5471-4dbf-967a-4b6ec4636b00\" (UID: \"7067e7d0-5471-4dbf-967a-4b6ec4636b00\") " Jul 07 15:07:48.697445 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:07:48.697379 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kchs2\" (UniqueName: \"kubernetes.io/projected/7067e7d0-5471-4dbf-967a-4b6ec4636b00-kube-api-access-kchs2\") pod \"7067e7d0-5471-4dbf-967a-4b6ec4636b00\" (UID: \"7067e7d0-5471-4dbf-967a-4b6ec4636b00\") " Jul 07 15:07:48.697739 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:07:48.697710 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7067e7d0-5471-4dbf-967a-4b6ec4636b00-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "7067e7d0-5471-4dbf-967a-4b6ec4636b00" (UID: "7067e7d0-5471-4dbf-967a-4b6ec4636b00"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 15:07:48.699429 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:07:48.699403 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7067e7d0-5471-4dbf-967a-4b6ec4636b00-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "7067e7d0-5471-4dbf-967a-4b6ec4636b00" (UID: "7067e7d0-5471-4dbf-967a-4b6ec4636b00"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 15:07:48.699553 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:07:48.699486 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7067e7d0-5471-4dbf-967a-4b6ec4636b00-kube-api-access-kchs2" (OuterVolumeSpecName: "kube-api-access-kchs2") pod "7067e7d0-5471-4dbf-967a-4b6ec4636b00" (UID: "7067e7d0-5471-4dbf-967a-4b6ec4636b00"). InnerVolumeSpecName "kube-api-access-kchs2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 15:07:48.798411 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:07:48.798390 2557 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7067e7d0-5471-4dbf-967a-4b6ec4636b00-openshift-service-ca\") on node \"ip-10-0-134-42.ec2.internal\" DevicePath \"\"" Jul 07 15:07:48.798411 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:07:48.798413 2557 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7067e7d0-5471-4dbf-967a-4b6ec4636b00-maas-api-tls\") on node \"ip-10-0-134-42.ec2.internal\" DevicePath \"\"" Jul 07 15:07:48.798605 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:07:48.798426 2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kchs2\" (UniqueName: \"kubernetes.io/projected/7067e7d0-5471-4dbf-967a-4b6ec4636b00-kube-api-access-kchs2\") on node \"ip-10-0-134-42.ec2.internal\" DevicePath \"\"" Jul 07 15:07:48.799381 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:07:48.799356 2557 generic.go:356] "Generic (PLEG): container finished" podID="7067e7d0-5471-4dbf-967a-4b6ec4636b00" containerID="8eb7a583266ca4c3697f94352c311270daa8984480416efa1c7fba5e180cb09f" exitCode=0 Jul 07 15:07:48.799451 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:07:48.799399 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-b-2f8e54-65c96df586-v8mzc" Jul 07 15:07:48.799497 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:07:48.799440 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-b-2f8e54-65c96df586-v8mzc" event={"ID":"7067e7d0-5471-4dbf-967a-4b6ec4636b00","Type":"ContainerDied","Data":"8eb7a583266ca4c3697f94352c311270daa8984480416efa1c7fba5e180cb09f"} Jul 07 15:07:48.799553 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:07:48.799497 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-b-2f8e54-65c96df586-v8mzc" event={"ID":"7067e7d0-5471-4dbf-967a-4b6ec4636b00","Type":"ContainerDied","Data":"a49f542eb76dc8247cb5729f29de45f154ad9c3891c85cba357a380898074570"} Jul 07 15:07:48.799595 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:07:48.799551 2557 scope.go:117] "RemoveContainer" containerID="8eb7a583266ca4c3697f94352c311270daa8984480416efa1c7fba5e180cb09f" Jul 07 15:07:48.808644 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:07:48.808627 2557 scope.go:117] "RemoveContainer" containerID="8eb7a583266ca4c3697f94352c311270daa8984480416efa1c7fba5e180cb09f" Jul 07 15:07:48.808890 ip-10-0-134-42 kubenswrapper[2557]: E0707 15:07:48.808873 2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8eb7a583266ca4c3697f94352c311270daa8984480416efa1c7fba5e180cb09f\": container with ID starting with 8eb7a583266ca4c3697f94352c311270daa8984480416efa1c7fba5e180cb09f not found: ID does not exist" containerID="8eb7a583266ca4c3697f94352c311270daa8984480416efa1c7fba5e180cb09f" Jul 07 15:07:48.808934 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:07:48.808898 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8eb7a583266ca4c3697f94352c311270daa8984480416efa1c7fba5e180cb09f"} err="failed to get container status \"8eb7a583266ca4c3697f94352c311270daa8984480416efa1c7fba5e180cb09f\": rpc error: code = NotFound desc = could not find container \"8eb7a583266ca4c3697f94352c311270daa8984480416efa1c7fba5e180cb09f\": container with ID starting with 8eb7a583266ca4c3697f94352c311270daa8984480416efa1c7fba5e180cb09f not found: ID does not exist" Jul 07 15:07:48.821116 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:07:48.821097 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-shared-b-2f8e54-65c96df586-v8mzc"] Jul 07 15:07:48.826862 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:07:48.826842 2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-shared-b-2f8e54-65c96df586-v8mzc"] Jul 07 15:07:49.835148 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:07:49.835116 2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7067e7d0-5471-4dbf-967a-4b6ec4636b00" path="/var/lib/kubelet/pods/7067e7d0-5471-4dbf-967a-4b6ec4636b00/volumes" Jul 07 15:07:55.463362 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:07:55.463309 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 15:07:55.563671 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:07:55.563643 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 15:07:55.794088 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:07:55.794008 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 15:07:56.020949 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:07:56.020872 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 15:08:01.798178 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:08:01.798130 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-6d878c79-c99rg_2f7b375b-aaed-4a14-bec8-8b72bfbe89fa/maas-api/0.log" Jul 07 15:08:01.928025 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:08:01.927997 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-key-cleanup-29723925-clxr2_9b8a5584-1e8a-45df-a7d1-e9bf146ea96f/cleanup/0.log" Jul 07 15:08:02.067440 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:08:02.067377 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-key-cleanup-29723940-d5h4k_338a9d19-9258-4d24-b1ed-bcf984664ed3/cleanup/0.log" Jul 07 15:08:02.196820 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:08:02.196794 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-d4dcbc546-dp5b9_12fd21f2-da38-49be-a702-eb46c7d7ea97/manager/0.log" Jul 07 15:08:02.835583 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:08:02.835554 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7868f6cbb6-t7m9p_344eede0-1be4-4e5e-87b9-81aae9469040/copy-manifests/0.log" Jul 07 15:08:02.854819 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:08:02.854796 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7868f6cbb6-t7m9p_344eede0-1be4-4e5e-87b9-81aae9469040/manager/0.log" Jul 07 15:08:02.990889 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:08:02.990857 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_postgres-69fb4594cf-dzxfg_b1bfa496-aad2-43a3-9e1e-7068752146e5/postgres/0.log" Jul 07 15:08:03.840094 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:08:03.840066 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759ps8sv_dd0c1d7e-44ea-4e2f-a1a4-d8b7877d2069/util/0.log" Jul 07 15:08:03.846816 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:08:03.846789 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759ps8sv_dd0c1d7e-44ea-4e2f-a1a4-d8b7877d2069/pull/0.log" Jul 07 15:08:03.852915 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:08:03.852895 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759ps8sv_dd0c1d7e-44ea-4e2f-a1a4-d8b7877d2069/extract/0.log" Jul 07 15:08:03.970500 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:08:03.970479 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0kmp97_ac44a2b1-1db5-4e2b-a3a4-7f08168b3902/util/0.log" Jul 07 15:08:03.977912 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:08:03.977867 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0kmp97_ac44a2b1-1db5-4e2b-a3a4-7f08168b3902/pull/0.log" Jul 07 15:08:03.984844 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:08:03.984819 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0kmp97_ac44a2b1-1db5-4e2b-a3a4-7f08168b3902/extract/0.log" Jul 07 15:08:04.098861 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:08:04.098782 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73pr78p_2870ed2a-a2da-44d8-9a7b-51b5b550469b/extract/0.log" Jul 07 15:08:04.105866 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:08:04.105844 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73pr78p_2870ed2a-a2da-44d8-9a7b-51b5b550469b/util/0.log" Jul 07 15:08:04.113316 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:08:04.113291 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73pr78p_2870ed2a-a2da-44d8-9a7b-51b5b550469b/pull/0.log" Jul 07 15:08:04.234048 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:08:04.234029 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12m9zw_740e7355-ef13-48ab-a537-87418f967bb4/util/0.log" Jul 07 15:08:04.241124 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:08:04.241096 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12m9zw_740e7355-ef13-48ab-a537-87418f967bb4/pull/0.log" Jul 07 15:08:04.247949 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:08:04.247920 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12m9zw_740e7355-ef13-48ab-a537-87418f967bb4/extract/0.log" Jul 07 15:08:04.494499 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:08:04.494435 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-vp6jj_53de9697-efbf-4494-8566-f32c55621a29/manager/0.log" Jul 07 15:08:04.843331 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:08:04.843295 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-ngld5_35e57567-53ab-4fe8-a9d9-55c156e703c7/registry-server/0.log" Jul 07 15:08:05.709263 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:08:05.709219 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-vffwj_d4731d32-8190-4426-b1f0-d66f3f00488c/discovery/0.log" Jul 07 15:08:05.938077 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:08:05.938053 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7646845f6c-hcxf5_1da3f1a9-9803-42bc-b609-8fdd84a03972/kube-auth-proxy/0.log" Jul 07 15:08:06.068013 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:08:06.067919 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-8cnl6_1861c4fd-2974-42f3-96e8-79601e052d18/istio-proxy/1.log" Jul 07 15:08:06.196946 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:08:06.196922 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-5wrr4_511ba927-b39a-4c92-a3c8-2ec64e3472ba/payload-pre-processing/0.log" Jul 07 15:08:06.319434 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:08:06.319359 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-nwgts_37f81559-4444-49b6-9fea-263efdf23c0a/payload-processing/0.log" Jul 07 15:08:06.449406 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:08:06.449386 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-7c58ff4df-55gs2_450aeb2e-26e4-4ebb-8006-eb9c43c6e2d1/router/0.log" Jul 07 15:08:06.791397 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:08:06.791370 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-67d8d5d6d-pqlxw_6d9a58b7-bb87-448c-b196-6f3756c687da/main/0.log" Jul 07 15:08:07.017885 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:08:07.017830 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-crpwq_9f32c8e5-c239-40f9-bcc2-1b13930acf48/main/0.log" Jul 07 15:08:07.245876 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:08:07.245845 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-74b68447df-nwzgs_352914b1-246c-4f00-a424-29b9323b9223/main/0.log" Jul 07 15:08:11.084245 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:08:11.084208 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 15:08:11.172251 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:08:11.172222 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" limit=1024 Jul 07 15:08:14.380251 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:08:14.380225 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-hvlzx_ce774983-4cbb-4526-ad84-72478efce781/global-pull-secret-syncer/0.log" Jul 07 15:08:14.517477 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:08:14.517451 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-2m5td_6c88a149-4504-482b-90e7-87d3065096c2/konnectivity-agent/0.log" Jul 07 15:08:14.683578 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:08:14.683495 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-134-42.ec2.internal_61dca777ea2260edb003f493cc5637ae/haproxy/0.log" Jul 07 15:08:18.947051 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:08:18.946967 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759ps8sv_dd0c1d7e-44ea-4e2f-a1a4-d8b7877d2069/extract/0.log" Jul 07 15:08:19.014475 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:08:19.014449 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759ps8sv_dd0c1d7e-44ea-4e2f-a1a4-d8b7877d2069/util/0.log" Jul 07 15:08:19.073356 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:08:19.073333 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759ps8sv_dd0c1d7e-44ea-4e2f-a1a4-d8b7877d2069/pull/0.log" Jul 07 15:08:19.118606 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:08:19.118585 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0kmp97_ac44a2b1-1db5-4e2b-a3a4-7f08168b3902/extract/0.log" Jul 07 15:08:19.172427 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:08:19.172401 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0kmp97_ac44a2b1-1db5-4e2b-a3a4-7f08168b3902/util/0.log" Jul 07 15:08:19.203394 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:08:19.203341 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0kmp97_ac44a2b1-1db5-4e2b-a3a4-7f08168b3902/pull/0.log" Jul 07 15:08:19.242308 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:08:19.242291 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73pr78p_2870ed2a-a2da-44d8-9a7b-51b5b550469b/extract/0.log" Jul 07 15:08:19.283226 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:08:19.283205 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73pr78p_2870ed2a-a2da-44d8-9a7b-51b5b550469b/util/0.log" Jul 07 15:08:19.320816 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:08:19.320798 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73pr78p_2870ed2a-a2da-44d8-9a7b-51b5b550469b/pull/0.log" Jul 07 15:08:19.364169 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:08:19.364149 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12m9zw_740e7355-ef13-48ab-a537-87418f967bb4/extract/0.log" Jul 07 15:08:19.408662 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:08:19.408643 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12m9zw_740e7355-ef13-48ab-a537-87418f967bb4/util/0.log" Jul 07 15:08:19.447640 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:08:19.447623 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12m9zw_740e7355-ef13-48ab-a537-87418f967bb4/pull/0.log" Jul 07 15:08:19.896797 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:08:19.896770 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-vp6jj_53de9697-efbf-4494-8566-f32c55621a29/manager/0.log" Jul 07 15:08:20.044145 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:08:20.044115 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-ngld5_35e57567-53ab-4fe8-a9d9-55c156e703c7/registry-server/0.log" Jul 07 15:08:22.216758 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:08:22.216729 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-64786d9cbb-bkdv4_d75c5911-092c-408a-83a7-871dfb379d28/cluster-monitoring-operator/0.log" Jul 07 15:08:22.411264 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:08:22.411231 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-ftkv4_b7247b62-46dd-4b19-8afd-52f281ee1f66/node-exporter/0.log" Jul 07 15:08:22.437038 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:08:22.437012 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-ftkv4_b7247b62-46dd-4b19-8afd-52f281ee1f66/kube-rbac-proxy/0.log" Jul 07 15:08:22.461195 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:08:22.461170 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-ftkv4_b7247b62-46dd-4b19-8afd-52f281ee1f66/init-textfile/0.log" Jul 07 15:08:22.679180 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:08:22.679150 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-54cf96dfcf-hdnqk_860b5cea-cdae-499f-b01c-1fdf449ab505/kube-rbac-proxy-main/0.log" Jul 07 15:08:22.704312 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:08:22.704287 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-54cf96dfcf-hdnqk_860b5cea-cdae-499f-b01c-1fdf449ab505/kube-rbac-proxy-self/0.log" Jul 07 15:08:22.729139 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:08:22.729118 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-54cf96dfcf-hdnqk_860b5cea-cdae-499f-b01c-1fdf449ab505/openshift-state-metrics/0.log" Jul 07 15:08:23.052632 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:08:23.052541 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-d559d66d5-fpkmh_e3ead245-ab3d-41f4-9e62-c256b1968e9c/prometheus-operator-admission-webhook/0.log" Jul 07 15:08:23.089488 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:08:23.089463 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-564c4f7c89-g6czx_9d863603-e1b9-49c8-93c0-dbff146ca2d7/telemeter-client/0.log" Jul 07 15:08:23.113096 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:08:23.113075 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-564c4f7c89-g6czx_9d863603-e1b9-49c8-93c0-dbff146ca2d7/reload/0.log" Jul 07 15:08:23.141172 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:08:23.141153 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-564c4f7c89-g6czx_9d863603-e1b9-49c8-93c0-dbff146ca2d7/kube-rbac-proxy/0.log" Jul 07 15:08:24.025796 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:08:24.025770 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-btsff/perf-node-gather-daemonset-mckcv"] Jul 07 15:08:24.026236 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:08:24.026219 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7067e7d0-5471-4dbf-967a-4b6ec4636b00" containerName="maas-api" Jul 07 15:08:24.026285 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:08:24.026241 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="7067e7d0-5471-4dbf-967a-4b6ec4636b00" containerName="maas-api" Jul 07 15:08:24.026285 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:08:24.026269 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="67490077-aae3-4b39-a59b-cdcc639c4684" containerName="maas-api" Jul 07 15:08:24.026285 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:08:24.026278 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="67490077-aae3-4b39-a59b-cdcc639c4684" containerName="maas-api" Jul 07 15:08:24.026393 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:08:24.026358 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="67490077-aae3-4b39-a59b-cdcc639c4684" containerName="maas-api" Jul 07 15:08:24.026393 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:08:24.026371 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="7067e7d0-5471-4dbf-967a-4b6ec4636b00" containerName="maas-api" Jul 07 15:08:24.029573 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:08:24.029557 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-btsff/perf-node-gather-daemonset-mckcv" Jul 07 15:08:24.031482 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:08:24.031441 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-btsff\"/\"default-dockercfg-kcwx6\"" Jul 07 15:08:24.031482 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:08:24.031474 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-btsff\"/\"openshift-service-ca.crt\"" Jul 07 15:08:24.031881 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:08:24.031864 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-btsff\"/\"kube-root-ca.crt\"" Jul 07 15:08:24.039949 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:08:24.039925 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-btsff/perf-node-gather-daemonset-mckcv"] Jul 07 15:08:24.115153 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:08:24.115120 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b839dd48-0194-45f1-9d2b-a659dccf8bfd-lib-modules\") pod \"perf-node-gather-daemonset-mckcv\" (UID: \"b839dd48-0194-45f1-9d2b-a659dccf8bfd\") " pod="openshift-must-gather-btsff/perf-node-gather-daemonset-mckcv" Jul 07 15:08:24.115357 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:08:24.115161 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r6std\" (UniqueName: \"kubernetes.io/projected/b839dd48-0194-45f1-9d2b-a659dccf8bfd-kube-api-access-r6std\") pod \"perf-node-gather-daemonset-mckcv\" (UID: \"b839dd48-0194-45f1-9d2b-a659dccf8bfd\") " pod="openshift-must-gather-btsff/perf-node-gather-daemonset-mckcv" Jul 07 15:08:24.115357 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:08:24.115255 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b839dd48-0194-45f1-9d2b-a659dccf8bfd-sys\") pod \"perf-node-gather-daemonset-mckcv\" (UID: \"b839dd48-0194-45f1-9d2b-a659dccf8bfd\") " pod="openshift-must-gather-btsff/perf-node-gather-daemonset-mckcv" Jul 07 15:08:24.115357 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:08:24.115301 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/b839dd48-0194-45f1-9d2b-a659dccf8bfd-proc\") pod \"perf-node-gather-daemonset-mckcv\" (UID: \"b839dd48-0194-45f1-9d2b-a659dccf8bfd\") " pod="openshift-must-gather-btsff/perf-node-gather-daemonset-mckcv" Jul 07 15:08:24.115357 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:08:24.115319 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/b839dd48-0194-45f1-9d2b-a659dccf8bfd-podres\") pod \"perf-node-gather-daemonset-mckcv\" (UID: \"b839dd48-0194-45f1-9d2b-a659dccf8bfd\") " pod="openshift-must-gather-btsff/perf-node-gather-daemonset-mckcv" Jul 07 15:08:24.216290 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:08:24.216249 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r6std\" (UniqueName: \"kubernetes.io/projected/b839dd48-0194-45f1-9d2b-a659dccf8bfd-kube-api-access-r6std\") pod \"perf-node-gather-daemonset-mckcv\" (UID: \"b839dd48-0194-45f1-9d2b-a659dccf8bfd\") " pod="openshift-must-gather-btsff/perf-node-gather-daemonset-mckcv" Jul 07 15:08:24.216635 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:08:24.216615 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b839dd48-0194-45f1-9d2b-a659dccf8bfd-sys\") pod \"perf-node-gather-daemonset-mckcv\" (UID: \"b839dd48-0194-45f1-9d2b-a659dccf8bfd\") " pod="openshift-must-gather-btsff/perf-node-gather-daemonset-mckcv" Jul 07 15:08:24.216803 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:08:24.216787 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/b839dd48-0194-45f1-9d2b-a659dccf8bfd-proc\") pod \"perf-node-gather-daemonset-mckcv\" (UID: \"b839dd48-0194-45f1-9d2b-a659dccf8bfd\") " pod="openshift-must-gather-btsff/perf-node-gather-daemonset-mckcv" Jul 07 15:08:24.216907 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:08:24.216894 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/b839dd48-0194-45f1-9d2b-a659dccf8bfd-podres\") pod \"perf-node-gather-daemonset-mckcv\" (UID: \"b839dd48-0194-45f1-9d2b-a659dccf8bfd\") " pod="openshift-must-gather-btsff/perf-node-gather-daemonset-mckcv" Jul 07 15:08:24.217072 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:08:24.217059 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b839dd48-0194-45f1-9d2b-a659dccf8bfd-lib-modules\") pod \"perf-node-gather-daemonset-mckcv\" (UID: \"b839dd48-0194-45f1-9d2b-a659dccf8bfd\") " pod="openshift-must-gather-btsff/perf-node-gather-daemonset-mckcv" Jul 07 15:08:24.217353 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:08:24.217337 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b839dd48-0194-45f1-9d2b-a659dccf8bfd-lib-modules\") pod \"perf-node-gather-daemonset-mckcv\" (UID: \"b839dd48-0194-45f1-9d2b-a659dccf8bfd\") " pod="openshift-must-gather-btsff/perf-node-gather-daemonset-mckcv" Jul 07 15:08:24.217572 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:08:24.217556 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b839dd48-0194-45f1-9d2b-a659dccf8bfd-sys\") pod \"perf-node-gather-daemonset-mckcv\" (UID: \"b839dd48-0194-45f1-9d2b-a659dccf8bfd\") " pod="openshift-must-gather-btsff/perf-node-gather-daemonset-mckcv" Jul 07 15:08:24.217737 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:08:24.217722 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/b839dd48-0194-45f1-9d2b-a659dccf8bfd-proc\") pod \"perf-node-gather-daemonset-mckcv\" (UID: \"b839dd48-0194-45f1-9d2b-a659dccf8bfd\") " pod="openshift-must-gather-btsff/perf-node-gather-daemonset-mckcv" Jul 07 15:08:24.217932 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:08:24.217915 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/b839dd48-0194-45f1-9d2b-a659dccf8bfd-podres\") pod \"perf-node-gather-daemonset-mckcv\" (UID: \"b839dd48-0194-45f1-9d2b-a659dccf8bfd\") " pod="openshift-must-gather-btsff/perf-node-gather-daemonset-mckcv" Jul 07 15:08:24.226273 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:08:24.226242 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r6std\" (UniqueName: \"kubernetes.io/projected/b839dd48-0194-45f1-9d2b-a659dccf8bfd-kube-api-access-r6std\") pod \"perf-node-gather-daemonset-mckcv\" (UID: \"b839dd48-0194-45f1-9d2b-a659dccf8bfd\") " pod="openshift-must-gather-btsff/perf-node-gather-daemonset-mckcv" Jul 07 15:08:24.341182 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:08:24.341158 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-btsff/perf-node-gather-daemonset-mckcv" Jul 07 15:08:24.473955 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:08:24.473930 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-btsff/perf-node-gather-daemonset-mckcv"] Jul 07 15:08:24.476562 ip-10-0-134-42 kubenswrapper[2557]: W0707 15:08:24.476506 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podb839dd48_0194_45f1_9d2b_a659dccf8bfd.slice/crio-d354bb2e976bc77ca039a295d22e6e8508fdfefb7c739223663d5e22a90b8e3d WatchSource:0}: Error finding container d354bb2e976bc77ca039a295d22e6e8508fdfefb7c739223663d5e22a90b8e3d: Status 404 returned error can't find the container with id d354bb2e976bc77ca039a295d22e6e8508fdfefb7c739223663d5e22a90b8e3d Jul 07 15:08:24.577310 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:08:24.577285 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-6bffdf9778-2hh7m_7a17e389-0667-415a-b38d-216a19842db3/networking-console-plugin/0.log" Jul 07 15:08:25.009111 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:08:25.009081 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-btsff/perf-node-gather-daemonset-mckcv" event={"ID":"b839dd48-0194-45f1-9d2b-a659dccf8bfd","Type":"ContainerStarted","Data":"8c5fc066026401f548366f79940be8927237f8e4c158af01668fd7116aa033a0"} Jul 07 15:08:25.009282 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:08:25.009128 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-btsff/perf-node-gather-daemonset-mckcv" event={"ID":"b839dd48-0194-45f1-9d2b-a659dccf8bfd","Type":"ContainerStarted","Data":"d354bb2e976bc77ca039a295d22e6e8508fdfefb7c739223663d5e22a90b8e3d"} Jul 07 15:08:25.009282 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:08:25.009173 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-btsff/perf-node-gather-daemonset-mckcv" Jul 07 15:08:25.033154 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:08:25.033109 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-btsff/perf-node-gather-daemonset-mckcv" podStartSLOduration=1.033096915 podStartE2EDuration="1.033096915s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 15:08:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:08:25.03085716 +0000 UTC m=+2541.739995542" watchObservedRunningTime="2026-07-07 15:08:25.033096915 +0000 UTC m=+2541.742235294" Jul 07 15:08:25.148702 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:08:25.148676 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-vd9g9_0fa80df4-c933-4758-b75f-3f70b7217c0d/console-operator/2.log" Jul 07 15:08:25.156646 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:08:25.156624 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-vd9g9_0fa80df4-c933-4758-b75f-3f70b7217c0d/console-operator/3.log" Jul 07 15:08:25.670565 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:08:25.670540 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-7d4b64bf94-ptm9z_44e65409-bca6-4333-8aa6-c6a0ca6ea8ba/console/0.log" Jul 07 15:08:26.294578 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:08:26.294549 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-5d456fdbcd-kzmk9_b77d08a8-a421-4412-9879-b094ef9bb36e/volume-data-source-validator/0.log" Jul 07 15:08:27.371918 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:08:27.371895 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-5ngmq_3d597bc9-74bf-4809-a33d-0a28dec34cb1/dns/0.log" Jul 07 15:08:27.398394 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:08:27.398367 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-5ngmq_3d597bc9-74bf-4809-a33d-0a28dec34cb1/kube-rbac-proxy/0.log" Jul 07 15:08:27.628301 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:08:27.628222 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-wmt48_aa141f90-5856-4779-90d7-c6377b1e32d6/dns-node-resolver/0.log" Jul 07 15:08:28.236767 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:08:28.236739 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-5g7bh_0b70b350-feef-419b-91c0-45a22ae7f30b/node-ca/0.log" Jul 07 15:08:29.522985 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:08:29.522932 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-vffwj_d4731d32-8190-4426-b1f0-d66f3f00488c/discovery/0.log" Jul 07 15:08:29.579181 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:08:29.579155 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7646845f6c-hcxf5_1da3f1a9-9803-42bc-b609-8fdd84a03972/kube-auth-proxy/0.log" Jul 07 15:08:29.618014 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:08:29.617983 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-8cnl6_1861c4fd-2974-42f3-96e8-79601e052d18/istio-proxy/0.log" Jul 07 15:08:29.673774 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:08:29.673753 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-8cnl6_1861c4fd-2974-42f3-96e8-79601e052d18/istio-proxy/1.log" Jul 07 15:08:29.718146 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:08:29.718124 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-5wrr4_511ba927-b39a-4c92-a3c8-2ec64e3472ba/payload-pre-processing/0.log" Jul 07 15:08:29.794024 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:08:29.793959 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-nwgts_37f81559-4444-49b6-9fea-263efdf23c0a/payload-processing/0.log" Jul 07 15:08:29.831623 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:08:29.831589 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-7c58ff4df-55gs2_450aeb2e-26e4-4ebb-8006-eb9c43c6e2d1/router/0.log" Jul 07 15:08:30.596372 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:08:30.596333 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-8dtp8_8c3755dc-b805-4f6a-8c8e-6ec9077a8efb/serve-healthcheck-canary/0.log" Jul 07 15:08:31.024702 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:08:31.024628 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-btsff/perf-node-gather-daemonset-mckcv" Jul 07 15:08:31.158164 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:08:31.158133 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5c847dcfc4-xpdmb_c7028698-fc37-401b-966d-359bad5c835e/insights-operator/0.log" Jul 07 15:08:31.161950 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:08:31.161918 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5c847dcfc4-xpdmb_c7028698-fc37-401b-966d-359bad5c835e/insights-operator/1.log" Jul 07 15:08:31.369876 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:08:31.369843 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-q75jf_d65b6030-55a6-4986-9642-a3bbe6be320c/kube-rbac-proxy/0.log" Jul 07 15:08:31.394362 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:08:31.394336 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-q75jf_d65b6030-55a6-4986-9642-a3bbe6be320c/exporter/0.log" Jul 07 15:08:31.420503 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:08:31.420475 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-q75jf_d65b6030-55a6-4986-9642-a3bbe6be320c/extractor/0.log" Jul 07 15:08:34.089646 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:08:34.089624 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-6d878c79-c99rg_2f7b375b-aaed-4a14-bec8-8b72bfbe89fa/maas-api/0.log" Jul 07 15:08:34.112236 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:08:34.112222 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-key-cleanup-29723925-clxr2_9b8a5584-1e8a-45df-a7d1-e9bf146ea96f/cleanup/0.log" Jul 07 15:08:34.142911 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:08:34.142827 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-key-cleanup-29723940-d5h4k_338a9d19-9258-4d24-b1ed-bcf984664ed3/cleanup/0.log" Jul 07 15:08:34.311759 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:08:34.311730 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-d4dcbc546-dp5b9_12fd21f2-da38-49be-a702-eb46c7d7ea97/manager/0.log" Jul 07 15:08:34.652273 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:08:34.652240 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7868f6cbb6-t7m9p_344eede0-1be4-4e5e-87b9-81aae9469040/manager/0.log" Jul 07 15:08:34.673422 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:08:34.673400 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7868f6cbb6-t7m9p_344eede0-1be4-4e5e-87b9-81aae9469040/copy-manifests/0.log" Jul 07 15:08:34.706059 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:08:34.706018 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_postgres-69fb4594cf-dzxfg_b1bfa496-aad2-43a3-9e1e-7068752146e5/postgres/0.log" Jul 07 15:08:34.938806 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:08:34.938715 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-vffwj_d4731d32-8190-4426-b1f0-d66f3f00488c/discovery/0.log" Jul 07 15:08:34.967043 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:08:34.967014 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7646845f6c-hcxf5_1da3f1a9-9803-42bc-b609-8fdd84a03972/kube-auth-proxy/0.log" Jul 07 15:08:34.990091 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:08:34.990062 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-8cnl6_1861c4fd-2974-42f3-96e8-79601e052d18/istio-proxy/0.log" Jul 07 15:08:35.044508 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:08:35.044481 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-8cnl6_1861c4fd-2974-42f3-96e8-79601e052d18/istio-proxy/1.log" Jul 07 15:08:35.079443 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:08:35.079415 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-5wrr4_511ba927-b39a-4c92-a3c8-2ec64e3472ba/payload-pre-processing/0.log" Jul 07 15:08:35.144102 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:08:35.144075 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-nwgts_37f81559-4444-49b6-9fea-263efdf23c0a/payload-processing/0.log" Jul 07 15:08:35.162793 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:08:35.162769 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-7c58ff4df-55gs2_450aeb2e-26e4-4ebb-8006-eb9c43c6e2d1/router/0.log" Jul 07 15:08:36.175450 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:08:36.175378 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-79j49_14c8a9df-2859-42c0-9e42-2036662ba764/openshift-lws-operator/0.log" Jul 07 15:08:36.657908 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:08:36.657879 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-vffwj_d4731d32-8190-4426-b1f0-d66f3f00488c/discovery/0.log" Jul 07 15:08:36.725699 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:08:36.725676 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7646845f6c-hcxf5_1da3f1a9-9803-42bc-b609-8fdd84a03972/kube-auth-proxy/0.log" Jul 07 15:08:36.750843 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:08:36.750822 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-8cnl6_1861c4fd-2974-42f3-96e8-79601e052d18/istio-proxy/0.log" Jul 07 15:08:36.805590 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:08:36.805571 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-8cnl6_1861c4fd-2974-42f3-96e8-79601e052d18/istio-proxy/1.log" Jul 07 15:08:36.836253 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:08:36.836232 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-5wrr4_511ba927-b39a-4c92-a3c8-2ec64e3472ba/payload-pre-processing/0.log" Jul 07 15:08:36.899329 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:08:36.899306 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-nwgts_37f81559-4444-49b6-9fea-263efdf23c0a/payload-processing/0.log" Jul 07 15:08:36.918815 ip-10-0-134-42 kubenswrapper[2557]: I0707 15:08:36.918763 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-7c58ff4df-55gs2_450aeb2e-26e4-4ebb-8006-eb9c43c6e2d1/router/0.log"