Jul 06 16:21:56.017404 ip-10-0-128-22 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Jul 06 16:21:56.017418 ip-10-0-128-22 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Jul 06 16:21:56.017428 ip-10-0-128-22 systemd[1]: kubelet.service: Failed with result 'resources'. Jul 06 16:21:56.017746 ip-10-0-128-22 systemd[1]: Failed to start Kubernetes Kubelet. Jul 06 16:22:07.174525 ip-10-0-128-22 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found. Jul 06 16:22:07.174547 ip-10-0-128-22 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot 3f8436a205484e5c9389995e0ad5c654 -- Jul 06 16:24:09.261047 ip-10-0-128-22 systemd[1]: Starting Kubernetes Kubelet... Jul 06 16:24:09.768440 ip-10-0-128-22 kubenswrapper[2584]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 06 16:24:09.768440 ip-10-0-128-22 kubenswrapper[2584]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 06 16:24:09.768440 ip-10-0-128-22 kubenswrapper[2584]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 06 16:24:09.768440 ip-10-0-128-22 kubenswrapper[2584]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 06 16:24:09.768440 ip-10-0-128-22 kubenswrapper[2584]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 06 16:24:09.768440 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:09.698477 2584 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 06 16:24:09.768440 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.704727 2584 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 06 16:24:09.768440 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.704761 2584 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 06 16:24:09.768440 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.704774 2584 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 06 16:24:09.768440 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.704781 2584 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 06 16:24:09.768440 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.704785 2584 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 06 16:24:09.768440 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.704789 2584 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 06 16:24:09.768440 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.704793 2584 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 06 16:24:09.768440 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.704796 2584 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 06 16:24:09.770072 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.704800 2584 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 06 16:24:09.770072 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.704803 2584 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 06 16:24:09.770072 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.704806 2584 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 06 16:24:09.770072 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.704809 2584 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 06 16:24:09.770072 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.704813 2584 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 06 16:24:09.770072 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.704816 2584 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 06 16:24:09.770072 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.704819 2584 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 06 16:24:09.770072 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.704822 2584 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 06 16:24:09.770072 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.704825 2584 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 06 16:24:09.770072 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.704828 2584 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 06 16:24:09.770072 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.704831 2584 feature_gate.go:328] unrecognized feature gate: Example2 Jul 06 16:24:09.770072 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.704834 2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 06 16:24:09.770072 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.704837 2584 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 06 16:24:09.770072 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.704840 2584 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 06 16:24:09.770072 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.704843 2584 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 06 16:24:09.770072 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.704846 2584 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 06 16:24:09.770072 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.704848 2584 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 06 16:24:09.770072 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.704851 2584 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 06 16:24:09.770072 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.704854 2584 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 06 16:24:09.771033 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.704857 2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 06 16:24:09.771033 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.704861 2584 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 06 16:24:09.771033 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.704864 2584 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 06 16:24:09.771033 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.704866 2584 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 06 16:24:09.771033 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.704869 2584 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 06 16:24:09.771033 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.704873 2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 06 16:24:09.771033 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.704875 2584 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 06 16:24:09.771033 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.704878 2584 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 06 16:24:09.771033 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.704881 2584 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 06 16:24:09.771033 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.704884 2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 06 16:24:09.771033 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.704886 2584 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 06 16:24:09.771033 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.704889 2584 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 06 16:24:09.771033 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.704892 2584 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 06 16:24:09.771033 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.704895 2584 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 06 16:24:09.771033 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.704898 2584 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 06 16:24:09.771033 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.704901 2584 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 06 16:24:09.771033 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.704904 2584 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 06 16:24:09.771033 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.704906 2584 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 06 16:24:09.771033 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.704909 2584 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 06 16:24:09.771033 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.704912 2584 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 06 16:24:09.771965 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.704915 2584 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 06 16:24:09.771965 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.704918 2584 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 06 16:24:09.771965 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.704921 2584 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 06 16:24:09.771965 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.704925 2584 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 06 16:24:09.771965 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.704929 2584 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 06 16:24:09.771965 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.704933 2584 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 06 16:24:09.771965 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.704937 2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 06 16:24:09.771965 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.704941 2584 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 06 16:24:09.771965 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.704944 2584 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 06 16:24:09.771965 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.704947 2584 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 06 16:24:09.771965 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.704950 2584 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 06 16:24:09.771965 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.704952 2584 feature_gate.go:328] unrecognized feature gate: Example Jul 06 16:24:09.771965 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.704955 2584 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 06 16:24:09.771965 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.704958 2584 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 06 16:24:09.771965 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.704961 2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 06 16:24:09.771965 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.704964 2584 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 06 16:24:09.771965 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.704966 2584 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 06 16:24:09.771965 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.704969 2584 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 06 16:24:09.771965 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.704972 2584 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 06 16:24:09.771965 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.704975 2584 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 06 16:24:09.773144 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.704979 2584 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 06 16:24:09.773144 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.704982 2584 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 06 16:24:09.773144 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.704985 2584 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 06 16:24:09.773144 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.704987 2584 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 06 16:24:09.773144 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.704991 2584 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 06 16:24:09.773144 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.704994 2584 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 06 16:24:09.773144 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.704997 2584 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 06 16:24:09.773144 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.704999 2584 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 06 16:24:09.773144 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.705002 2584 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 06 16:24:09.773144 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.705005 2584 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 06 16:24:09.773144 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.705009 2584 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 06 16:24:09.773144 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.705014 2584 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 06 16:24:09.773144 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.705017 2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 06 16:24:09.773144 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.705020 2584 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 06 16:24:09.773144 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.705023 2584 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 06 16:24:09.773144 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.705027 2584 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 06 16:24:09.773144 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.705040 2584 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 06 16:24:09.773144 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.705043 2584 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 06 16:24:09.773144 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.705046 2584 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 06 16:24:09.774095 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.705049 2584 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 06 16:24:09.774095 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.705052 2584 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 06 16:24:09.774095 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.705055 2584 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 06 16:24:09.774095 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.705058 2584 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 06 16:24:09.774095 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.705061 2584 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 06 16:24:09.774095 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.705064 2584 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 06 16:24:09.774095 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.705067 2584 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 06 16:24:09.774095 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.705070 2584 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 06 16:24:09.774095 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.705073 2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 06 16:24:09.774095 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.705076 2584 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 06 16:24:09.774095 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.705081 2584 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 06 16:24:09.774095 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.705084 2584 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 06 16:24:09.774095 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.705086 2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 06 16:24:09.774095 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.705920 2584 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 06 16:24:09.774095 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.705929 2584 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 06 16:24:09.774095 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.705933 2584 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 06 16:24:09.774095 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.705936 2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 06 16:24:09.774095 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.705939 2584 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 06 16:24:09.774095 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.705943 2584 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 06 16:24:09.774095 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.705946 2584 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 06 16:24:09.775052 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.705949 2584 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 06 16:24:09.775052 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.705952 2584 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 06 16:24:09.775052 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.705955 2584 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 06 16:24:09.775052 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.705958 2584 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 06 16:24:09.775052 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.705961 2584 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 06 16:24:09.775052 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.705963 2584 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 06 16:24:09.775052 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.705966 2584 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 06 16:24:09.775052 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.705970 2584 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 06 16:24:09.775052 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.705973 2584 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 06 16:24:09.775052 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.705976 2584 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 06 16:24:09.775052 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.705979 2584 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 06 16:24:09.775052 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.705983 2584 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 06 16:24:09.775052 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.705986 2584 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 06 16:24:09.775052 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.705989 2584 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 06 16:24:09.775052 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.705992 2584 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 06 16:24:09.775052 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.705995 2584 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 06 16:24:09.775052 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.706005 2584 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 06 16:24:09.775052 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.706009 2584 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 06 16:24:09.775052 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.706013 2584 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 06 16:24:09.775980 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.706016 2584 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 06 16:24:09.775980 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.706019 2584 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 06 16:24:09.775980 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.706023 2584 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 06 16:24:09.775980 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.706026 2584 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 06 16:24:09.775980 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.706029 2584 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 06 16:24:09.775980 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.706032 2584 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 06 16:24:09.775980 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.706034 2584 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 06 16:24:09.775980 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.706037 2584 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 06 16:24:09.775980 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.706040 2584 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 06 16:24:09.775980 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.706044 2584 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 06 16:24:09.775980 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.706048 2584 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 06 16:24:09.775980 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.706050 2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 06 16:24:09.775980 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.706054 2584 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 06 16:24:09.775980 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.706057 2584 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 06 16:24:09.775980 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.706060 2584 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 06 16:24:09.775980 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.706063 2584 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 06 16:24:09.775980 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.706066 2584 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 06 16:24:09.775980 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.706069 2584 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 06 16:24:09.775980 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.706072 2584 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 06 16:24:09.775980 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.706075 2584 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 06 16:24:09.777157 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.706078 2584 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 06 16:24:09.777157 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.706081 2584 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 06 16:24:09.777157 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.706084 2584 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 06 16:24:09.777157 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.706087 2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 06 16:24:09.777157 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.706091 2584 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 06 16:24:09.777157 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.706094 2584 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 06 16:24:09.777157 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.706097 2584 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 06 16:24:09.777157 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.706100 2584 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 06 16:24:09.777157 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.706103 2584 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 06 16:24:09.777157 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.706106 2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 06 16:24:09.777157 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.706109 2584 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 06 16:24:09.777157 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.706112 2584 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 06 16:24:09.777157 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.706115 2584 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 06 16:24:09.777157 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.706118 2584 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 06 16:24:09.777157 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.706121 2584 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 06 16:24:09.777157 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.706124 2584 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 06 16:24:09.777157 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.706127 2584 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 06 16:24:09.777157 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.706129 2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 06 16:24:09.777157 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.706132 2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 06 16:24:09.777157 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.706135 2584 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 06 16:24:09.778345 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.706138 2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 06 16:24:09.778345 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.706143 2584 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 06 16:24:09.778345 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.706146 2584 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 06 16:24:09.778345 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.706149 2584 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 06 16:24:09.778345 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.706152 2584 feature_gate.go:328] unrecognized feature gate: Example2 Jul 06 16:24:09.778345 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.706156 2584 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 06 16:24:09.778345 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.706158 2584 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 06 16:24:09.778345 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.706161 2584 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 06 16:24:09.778345 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.706164 2584 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 06 16:24:09.778345 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.706167 2584 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 06 16:24:09.778345 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.706170 2584 feature_gate.go:328] unrecognized feature gate: Example Jul 06 16:24:09.778345 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.706173 2584 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 06 16:24:09.778345 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.706175 2584 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 06 16:24:09.778345 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.706178 2584 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 06 16:24:09.778345 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.706181 2584 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 06 16:24:09.778345 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.706183 2584 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 06 16:24:09.778345 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.706188 2584 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 06 16:24:09.778345 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.706191 2584 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 06 16:24:09.778345 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.706194 2584 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 06 16:24:09.778345 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.706196 2584 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 06 16:24:09.779308 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.706199 2584 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 06 16:24:09.779308 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.706204 2584 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 06 16:24:09.779308 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.706208 2584 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 06 16:24:09.779308 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.706212 2584 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 06 16:24:09.779308 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.706215 2584 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 06 16:24:09.779308 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.706218 2584 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 06 16:24:09.779308 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.706221 2584 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 06 16:24:09.779308 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.706224 2584 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 06 16:24:09.779308 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.706227 2584 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 06 16:24:09.779308 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.706230 2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 06 16:24:09.779308 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.706233 2584 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 06 16:24:09.779308 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.706235 2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 06 16:24:09.779308 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.706238 2584 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 06 16:24:09.779308 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.706679 2584 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 06 16:24:09.779308 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.706685 2584 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 06 16:24:09.779308 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.706688 2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 06 16:24:09.779308 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.706691 2584 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 06 16:24:09.779308 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.706695 2584 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 06 16:24:09.779308 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.706698 2584 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 06 16:24:09.780253 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.706701 2584 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 06 16:24:09.780253 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.706704 2584 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 06 16:24:09.780253 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.706707 2584 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 06 16:24:09.780253 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.706709 2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 06 16:24:09.780253 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.706712 2584 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 06 16:24:09.780253 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.706715 2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 06 16:24:09.780253 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.706719 2584 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 06 16:24:09.780253 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.706721 2584 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 06 16:24:09.780253 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.706724 2584 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 06 16:24:09.780253 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.706729 2584 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 06 16:24:09.780253 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.706732 2584 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 06 16:24:09.780253 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.706735 2584 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 06 16:24:09.780253 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.706737 2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 06 16:24:09.780253 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.706740 2584 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 06 16:24:09.780253 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.706745 2584 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 06 16:24:09.780253 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.706748 2584 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 06 16:24:09.780253 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.706750 2584 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 06 16:24:09.780253 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.706753 2584 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 06 16:24:09.780253 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.706756 2584 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 06 16:24:09.780253 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.706759 2584 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 06 16:24:09.781211 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.706762 2584 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 06 16:24:09.781211 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.706765 2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 06 16:24:09.781211 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.706767 2584 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 06 16:24:09.781211 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.706770 2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 06 16:24:09.781211 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.706773 2584 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 06 16:24:09.781211 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.706776 2584 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 06 16:24:09.781211 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.706779 2584 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 06 16:24:09.781211 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.706781 2584 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 06 16:24:09.781211 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.706784 2584 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 06 16:24:09.781211 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.706787 2584 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 06 16:24:09.781211 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.706790 2584 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 06 16:24:09.781211 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.706792 2584 feature_gate.go:328] unrecognized feature gate: Example2 Jul 06 16:24:09.781211 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.706812 2584 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 06 16:24:09.781211 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.706815 2584 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 06 16:24:09.781211 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.706818 2584 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 06 16:24:09.781211 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.706822 2584 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 06 16:24:09.781211 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.706826 2584 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 06 16:24:09.781211 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.706830 2584 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 06 16:24:09.781211 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.706835 2584 feature_gate.go:328] unrecognized feature gate: Example Jul 06 16:24:09.782162 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.706838 2584 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 06 16:24:09.782162 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.706842 2584 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 06 16:24:09.782162 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.706846 2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 06 16:24:09.782162 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.706850 2584 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 06 16:24:09.782162 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.706854 2584 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 06 16:24:09.782162 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.706857 2584 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 06 16:24:09.782162 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.706860 2584 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 06 16:24:09.782162 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.706864 2584 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 06 16:24:09.782162 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.706867 2584 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 06 16:24:09.782162 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.706870 2584 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 06 16:24:09.782162 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.706872 2584 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 06 16:24:09.782162 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.706875 2584 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 06 16:24:09.782162 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.706878 2584 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 06 16:24:09.782162 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.706881 2584 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 06 16:24:09.782162 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.706884 2584 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 06 16:24:09.782162 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.706887 2584 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 06 16:24:09.782162 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.706890 2584 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 06 16:24:09.782162 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.706892 2584 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 06 16:24:09.782162 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.706895 2584 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 06 16:24:09.782162 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.706898 2584 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 06 16:24:09.783134 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.706900 2584 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 06 16:24:09.783134 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.706903 2584 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 06 16:24:09.783134 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.706906 2584 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 06 16:24:09.783134 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.706909 2584 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 06 16:24:09.783134 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.706911 2584 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 06 16:24:09.783134 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.706914 2584 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 06 16:24:09.783134 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.706917 2584 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 06 16:24:09.783134 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.706920 2584 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 06 16:24:09.783134 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.706922 2584 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 06 16:24:09.783134 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.706925 2584 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 06 16:24:09.783134 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.706928 2584 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 06 16:24:09.783134 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.706931 2584 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 06 16:24:09.783134 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.706933 2584 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 06 16:24:09.783134 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.706936 2584 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 06 16:24:09.783134 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.706940 2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 06 16:24:09.783134 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.706943 2584 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 06 16:24:09.783134 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.706946 2584 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 06 16:24:09.783134 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.706949 2584 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 06 16:24:09.783134 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.706954 2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 06 16:24:09.783134 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.706958 2584 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 06 16:24:09.784090 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.706961 2584 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 06 16:24:09.784090 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.706964 2584 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 06 16:24:09.784090 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.706967 2584 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 06 16:24:09.784090 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.706971 2584 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 06 16:24:09.784090 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.706974 2584 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 06 16:24:09.784090 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.706977 2584 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 06 16:24:09.784090 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.706981 2584 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 06 16:24:09.784090 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.706984 2584 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 06 16:24:09.784090 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.706987 2584 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 06 16:24:09.784090 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.706989 2584 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 06 16:24:09.784090 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.706992 2584 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 06 16:24:09.784090 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.706995 2584 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 06 16:24:09.784090 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.706998 2584 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 06 16:24:09.784090 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.707001 2584 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 06 16:24:09.784090 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:09.707084 2584 flags.go:64] FLAG: --address="0.0.0.0" Jul 06 16:24:09.784090 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:09.707093 2584 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 06 16:24:09.784090 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:09.707100 2584 flags.go:64] FLAG: --anonymous-auth="true" Jul 06 16:24:09.784090 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:09.707106 2584 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 06 16:24:09.784090 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:09.707110 2584 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 06 16:24:09.784090 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:09.707114 2584 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 06 16:24:09.784090 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:09.707118 2584 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 06 16:24:09.787579 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:09.707123 2584 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 06 16:24:09.787579 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:09.707127 2584 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 06 16:24:09.787579 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:09.707130 2584 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 06 16:24:09.787579 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:09.707134 2584 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 06 16:24:09.787579 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:09.707137 2584 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 06 16:24:09.787579 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:09.707142 2584 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 06 16:24:09.787579 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:09.707146 2584 flags.go:64] FLAG: --cgroup-root="" Jul 06 16:24:09.787579 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:09.707150 2584 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 06 16:24:09.787579 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:09.707153 2584 flags.go:64] FLAG: --client-ca-file="" Jul 06 16:24:09.787579 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:09.707157 2584 flags.go:64] FLAG: --cloud-provider="external" Jul 06 16:24:09.787579 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:09.707161 2584 flags.go:64] FLAG: --cluster-dns="[]" Jul 06 16:24:09.787579 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:09.707166 2584 flags.go:64] FLAG: --cluster-domain="" Jul 06 16:24:09.787579 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:09.707169 2584 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 06 16:24:09.787579 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:09.707172 2584 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 06 16:24:09.787579 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:09.707176 2584 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 06 16:24:09.787579 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:09.707179 2584 flags.go:64] FLAG: --container-log-max-files="5" Jul 06 16:24:09.787579 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:09.707183 2584 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 06 16:24:09.787579 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:09.707187 2584 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 06 16:24:09.787579 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:09.707190 2584 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 06 16:24:09.787579 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:09.707193 2584 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 06 16:24:09.787579 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:09.707197 2584 flags.go:64] FLAG: --contention-profiling="false" Jul 06 16:24:09.787579 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:09.707200 2584 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 06 16:24:09.787579 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:09.707203 2584 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 06 16:24:09.787579 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:09.707207 2584 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 06 16:24:09.788763 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:09.707210 2584 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 06 16:24:09.788763 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:09.707214 2584 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 06 16:24:09.788763 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:09.707218 2584 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 06 16:24:09.788763 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:09.707221 2584 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 06 16:24:09.788763 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:09.707224 2584 flags.go:64] FLAG: --enable-load-reader="false" Jul 06 16:24:09.788763 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:09.707227 2584 flags.go:64] FLAG: --enable-server="true" Jul 06 16:24:09.788763 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:09.707231 2584 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 06 16:24:09.788763 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:09.707236 2584 flags.go:64] FLAG: --event-burst="100" Jul 06 16:24:09.788763 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:09.707239 2584 flags.go:64] FLAG: --event-qps="50" Jul 06 16:24:09.788763 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:09.707242 2584 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 06 16:24:09.788763 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:09.707246 2584 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 06 16:24:09.788763 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:09.707249 2584 flags.go:64] FLAG: --eviction-hard="" Jul 06 16:24:09.788763 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:09.707253 2584 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 06 16:24:09.788763 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:09.707258 2584 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 06 16:24:09.788763 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:09.707261 2584 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 06 16:24:09.788763 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:09.707264 2584 flags.go:64] FLAG: --eviction-soft="" Jul 06 16:24:09.788763 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:09.707267 2584 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 06 16:24:09.788763 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:09.707271 2584 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 06 16:24:09.788763 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:09.707276 2584 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 06 16:24:09.788763 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:09.707279 2584 flags.go:64] FLAG: --experimental-mounter-path="" Jul 06 16:24:09.788763 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:09.707282 2584 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 06 16:24:09.788763 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:09.707285 2584 flags.go:64] FLAG: --fail-swap-on="true" Jul 06 16:24:09.788763 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:09.707289 2584 flags.go:64] FLAG: --feature-gates="" Jul 06 16:24:09.788763 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:09.707293 2584 flags.go:64] FLAG: --file-check-frequency="20s" Jul 06 16:24:09.788763 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:09.707296 2584 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 06 16:24:09.796540 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:09.707300 2584 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 06 16:24:09.796540 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:09.707303 2584 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 06 16:24:09.796540 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:09.707306 2584 flags.go:64] FLAG: --healthz-port="10248" Jul 06 16:24:09.796540 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:09.707310 2584 flags.go:64] FLAG: --help="false" Jul 06 16:24:09.796540 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:09.707313 2584 flags.go:64] FLAG: --hostname-override="ip-10-0-128-22.ec2.internal" Jul 06 16:24:09.796540 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:09.707317 2584 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 06 16:24:09.796540 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:09.707320 2584 flags.go:64] FLAG: --http-check-frequency="20s" Jul 06 16:24:09.796540 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:09.707323 2584 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 06 16:24:09.796540 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:09.707326 2584 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 06 16:24:09.796540 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:09.707330 2584 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 06 16:24:09.796540 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:09.707333 2584 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 06 16:24:09.796540 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:09.707336 2584 flags.go:64] FLAG: --image-service-endpoint="" Jul 06 16:24:09.796540 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:09.707339 2584 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 06 16:24:09.796540 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:09.707342 2584 flags.go:64] FLAG: --kube-api-burst="100" Jul 06 16:24:09.796540 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:09.707345 2584 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 06 16:24:09.796540 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:09.707349 2584 flags.go:64] FLAG: --kube-api-qps="50" Jul 06 16:24:09.796540 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:09.707352 2584 flags.go:64] FLAG: --kube-reserved="" Jul 06 16:24:09.796540 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:09.707355 2584 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 06 16:24:09.796540 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:09.707358 2584 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 06 16:24:09.796540 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:09.707361 2584 flags.go:64] FLAG: --kubelet-cgroups="" Jul 06 16:24:09.796540 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:09.707366 2584 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 06 16:24:09.796540 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:09.707369 2584 flags.go:64] FLAG: --lock-file="" Jul 06 16:24:09.796540 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:09.707372 2584 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 06 16:24:09.796540 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:09.707375 2584 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 06 16:24:09.797424 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:09.707378 2584 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 06 16:24:09.797424 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:09.707386 2584 flags.go:64] FLAG: --log-json-split-stream="false" Jul 06 16:24:09.797424 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:09.707390 2584 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 06 16:24:09.797424 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:09.707393 2584 flags.go:64] FLAG: --log-text-split-stream="false" Jul 06 16:24:09.797424 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:09.707396 2584 flags.go:64] FLAG: --logging-format="text" Jul 06 16:24:09.797424 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:09.707399 2584 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 06 16:24:09.797424 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:09.707403 2584 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 06 16:24:09.797424 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:09.707407 2584 flags.go:64] FLAG: --manifest-url="" Jul 06 16:24:09.797424 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:09.707410 2584 flags.go:64] FLAG: --manifest-url-header="" Jul 06 16:24:09.797424 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:09.707414 2584 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 06 16:24:09.797424 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:09.707417 2584 flags.go:64] FLAG: --max-open-files="1000000" Jul 06 16:24:09.797424 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:09.707422 2584 flags.go:64] FLAG: --max-pods="110" Jul 06 16:24:09.797424 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:09.707425 2584 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 06 16:24:09.797424 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:09.707428 2584 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 06 16:24:09.797424 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:09.707431 2584 flags.go:64] FLAG: --memory-manager-policy="None" Jul 06 16:24:09.797424 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:09.707434 2584 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 06 16:24:09.797424 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:09.707438 2584 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 06 16:24:09.797424 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:09.707441 2584 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 06 16:24:09.797424 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:09.707445 2584 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 06 16:24:09.797424 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:09.707458 2584 flags.go:64] FLAG: --node-status-max-images="50" Jul 06 16:24:09.797424 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:09.707462 2584 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 06 16:24:09.797424 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:09.707465 2584 flags.go:64] FLAG: --oom-score-adj="-999" Jul 06 16:24:09.797424 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:09.707468 2584 flags.go:64] FLAG: --pod-cidr="" Jul 06 16:24:09.798333 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:09.707471 2584 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ce93f6b4197b9c62a26a3131f8057ffe9c9379c5079a4f62e5919b75151394e5" Jul 06 16:24:09.798333 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:09.707477 2584 flags.go:64] FLAG: --pod-manifest-path="" Jul 06 16:24:09.798333 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:09.707480 2584 flags.go:64] FLAG: --pod-max-pids="-1" Jul 06 16:24:09.798333 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:09.707483 2584 flags.go:64] FLAG: --pods-per-core="0" Jul 06 16:24:09.798333 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:09.707487 2584 flags.go:64] FLAG: --port="10250" Jul 06 16:24:09.798333 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:09.707491 2584 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 06 16:24:09.798333 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:09.707494 2584 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-001dad603af96ef88" Jul 06 16:24:09.798333 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:09.707498 2584 flags.go:64] FLAG: --qos-reserved="" Jul 06 16:24:09.798333 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:09.707501 2584 flags.go:64] FLAG: --read-only-port="10255" Jul 06 16:24:09.798333 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:09.707504 2584 flags.go:64] FLAG: --register-node="true" Jul 06 16:24:09.798333 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:09.707508 2584 flags.go:64] FLAG: --register-with-taints="" Jul 06 16:24:09.798333 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:09.707518 2584 flags.go:64] FLAG: --registry-burst="10" Jul 06 16:24:09.798333 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:09.707522 2584 flags.go:64] FLAG: --registry-qps="5" Jul 06 16:24:09.798333 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:09.707525 2584 flags.go:64] FLAG: --reserved-cpus="" Jul 06 16:24:09.798333 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:09.707529 2584 flags.go:64] FLAG: --reserved-memory="" Jul 06 16:24:09.798333 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:09.707533 2584 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 06 16:24:09.798333 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:09.707537 2584 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 06 16:24:09.798333 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:09.707540 2584 flags.go:64] FLAG: --rotate-certificates="false" Jul 06 16:24:09.798333 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:09.707543 2584 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 06 16:24:09.798333 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:09.707546 2584 flags.go:64] FLAG: --runonce="false" Jul 06 16:24:09.798333 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:09.707549 2584 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 06 16:24:09.798333 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:09.707552 2584 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 06 16:24:09.798333 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:09.707556 2584 flags.go:64] FLAG: --seccomp-default="false" Jul 06 16:24:09.798333 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:09.707559 2584 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 06 16:24:09.798333 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:09.707562 2584 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 06 16:24:09.923043 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:09.707565 2584 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 06 16:24:09.923043 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:09.707568 2584 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 06 16:24:09.923043 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:09.707571 2584 flags.go:64] FLAG: --storage-driver-password="root" Jul 06 16:24:09.923043 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:09.707574 2584 flags.go:64] FLAG: --storage-driver-secure="false" Jul 06 16:24:09.923043 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:09.707578 2584 flags.go:64] FLAG: --storage-driver-table="stats" Jul 06 16:24:09.923043 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:09.707581 2584 flags.go:64] FLAG: --storage-driver-user="root" Jul 06 16:24:09.923043 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:09.707584 2584 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 06 16:24:09.923043 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:09.707587 2584 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 06 16:24:09.923043 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:09.707591 2584 flags.go:64] FLAG: --system-cgroups="" Jul 06 16:24:09.923043 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:09.707593 2584 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 06 16:24:09.923043 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:09.707599 2584 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 06 16:24:09.923043 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:09.707602 2584 flags.go:64] FLAG: --tls-cert-file="" Jul 06 16:24:09.923043 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:09.707607 2584 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 06 16:24:09.923043 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:09.707611 2584 flags.go:64] FLAG: --tls-min-version="" Jul 06 16:24:09.923043 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:09.707614 2584 flags.go:64] FLAG: --tls-private-key-file="" Jul 06 16:24:09.923043 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:09.707617 2584 flags.go:64] FLAG: --topology-manager-policy="none" Jul 06 16:24:09.923043 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:09.707620 2584 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 06 16:24:09.923043 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:09.707623 2584 flags.go:64] FLAG: --topology-manager-scope="container" Jul 06 16:24:09.923043 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:09.707628 2584 flags.go:64] FLAG: --v="2" Jul 06 16:24:09.923043 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:09.707632 2584 flags.go:64] FLAG: --version="false" Jul 06 16:24:09.923043 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:09.707642 2584 flags.go:64] FLAG: --vmodule="" Jul 06 16:24:09.923043 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:09.707651 2584 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 06 16:24:09.923043 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:09.707655 2584 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 06 16:24:09.923043 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.707777 2584 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 06 16:24:09.923043 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.707781 2584 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 06 16:24:09.814165 ip-10-0-128-22 systemd[1]: Started Kubernetes Kubelet. Jul 06 16:24:09.924144 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.707784 2584 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 06 16:24:09.924144 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.707787 2584 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 06 16:24:09.924144 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.707790 2584 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 06 16:24:09.924144 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.707793 2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 06 16:24:09.924144 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.707812 2584 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 06 16:24:09.924144 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.707815 2584 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 06 16:24:09.924144 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.707819 2584 feature_gate.go:328] unrecognized feature gate: Example Jul 06 16:24:09.924144 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.707822 2584 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 06 16:24:09.924144 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.707825 2584 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 06 16:24:09.924144 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.707828 2584 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 06 16:24:09.924144 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.707831 2584 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 06 16:24:09.924144 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.707834 2584 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 06 16:24:09.924144 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.707837 2584 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 06 16:24:09.924144 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.707840 2584 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 06 16:24:09.924144 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.707843 2584 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 06 16:24:09.924144 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.707846 2584 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 06 16:24:09.924144 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.707849 2584 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 06 16:24:09.924144 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.707851 2584 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 06 16:24:09.924144 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.707854 2584 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 06 16:24:09.924144 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.707859 2584 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 06 16:24:09.925016 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.707862 2584 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 06 16:24:09.925016 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.707865 2584 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 06 16:24:09.925016 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.707867 2584 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 06 16:24:09.925016 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.707870 2584 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 06 16:24:09.925016 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.707874 2584 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 06 16:24:09.925016 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.707878 2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 06 16:24:09.925016 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.707881 2584 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 06 16:24:09.925016 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.707884 2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 06 16:24:09.925016 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.707888 2584 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 06 16:24:09.925016 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.707890 2584 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 06 16:24:09.925016 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.707893 2584 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 06 16:24:09.925016 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.707896 2584 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 06 16:24:09.925016 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.707898 2584 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 06 16:24:09.925016 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.707901 2584 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 06 16:24:09.925016 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.707904 2584 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 06 16:24:09.925016 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.707907 2584 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 06 16:24:09.925016 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.707909 2584 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 06 16:24:09.925016 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.707912 2584 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 06 16:24:09.925016 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.707915 2584 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 06 16:24:09.925691 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.707918 2584 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 06 16:24:09.925691 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.707921 2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 06 16:24:09.925691 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.707924 2584 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 06 16:24:09.925691 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.707926 2584 feature_gate.go:328] unrecognized feature gate: Example2 Jul 06 16:24:09.925691 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.707929 2584 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 06 16:24:09.925691 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.707934 2584 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 06 16:24:09.925691 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.707938 2584 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 06 16:24:09.925691 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.707941 2584 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 06 16:24:09.925691 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.707944 2584 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 06 16:24:09.925691 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.707947 2584 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 06 16:24:09.925691 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.707951 2584 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 06 16:24:09.925691 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.707953 2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 06 16:24:09.925691 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.707958 2584 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 06 16:24:09.925691 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.707961 2584 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 06 16:24:09.925691 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.707964 2584 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 06 16:24:09.925691 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.707967 2584 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 06 16:24:09.925691 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.707970 2584 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 06 16:24:09.925691 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.707973 2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 06 16:24:09.925691 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.707977 2584 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 06 16:24:09.925691 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.707980 2584 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 06 16:24:09.926425 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.707983 2584 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 06 16:24:09.926425 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.707985 2584 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 06 16:24:09.926425 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.707989 2584 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 06 16:24:09.926425 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.707992 2584 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 06 16:24:09.926425 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.707995 2584 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 06 16:24:09.926425 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.707998 2584 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 06 16:24:09.926425 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.708001 2584 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 06 16:24:09.926425 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.708003 2584 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 06 16:24:09.926425 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.708006 2584 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 06 16:24:09.926425 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.708009 2584 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 06 16:24:09.926425 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.708012 2584 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 06 16:24:09.926425 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.708015 2584 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 06 16:24:09.926425 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.708018 2584 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 06 16:24:09.926425 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.708020 2584 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 06 16:24:09.926425 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.708024 2584 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 06 16:24:09.926425 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.708027 2584 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 06 16:24:09.926425 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.708029 2584 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 06 16:24:09.926425 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.708032 2584 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 06 16:24:09.926425 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.708035 2584 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 06 16:24:09.926425 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.708038 2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 06 16:24:09.927107 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.708041 2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 06 16:24:09.927107 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.708044 2584 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 06 16:24:09.927107 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.708046 2584 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 06 16:24:09.927107 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.708049 2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 06 16:24:09.927107 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.708055 2584 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 06 16:24:09.927107 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.708059 2584 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 06 16:24:09.927107 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.708062 2584 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 06 16:24:09.927107 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.708065 2584 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 06 16:24:09.927107 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.708068 2584 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 06 16:24:09.927107 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.708072 2584 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 06 16:24:09.927107 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.708076 2584 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 06 16:24:09.927107 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.708079 2584 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 06 16:24:09.927107 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.708082 2584 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 06 16:24:09.927107 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.708086 2584 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 06 16:24:09.927107 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.708090 2584 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 06 16:24:09.927107 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.708101 2584 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 06 16:24:09.927107 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.708104 2584 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 06 16:24:09.927107 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.708107 2584 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 06 16:24:09.927777 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:09.709904 2584 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 06 16:24:09.927777 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:09.717813 2584 server.go:529] "Kubelet version" kubeletVersion="v1.34.8" Jul 06 16:24:09.927777 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:09.717836 2584 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 06 16:24:09.927777 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:09.717861 2584 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 06 16:24:09.927777 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:09.717871 2584 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 06 16:24:09.927777 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.717936 2584 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 06 16:24:09.927777 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.717942 2584 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 06 16:24:09.927777 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.717946 2584 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 06 16:24:09.927777 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.717949 2584 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 06 16:24:09.927777 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.717952 2584 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 06 16:24:09.927777 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.717955 2584 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 06 16:24:09.927777 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.717958 2584 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 06 16:24:09.927777 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.717961 2584 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 06 16:24:09.927777 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.717964 2584 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 06 16:24:09.927777 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.717967 2584 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 06 16:24:09.928437 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.717970 2584 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 06 16:24:09.928437 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.717976 2584 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 06 16:24:09.928437 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.717979 2584 feature_gate.go:328] unrecognized feature gate: Example Jul 06 16:24:09.928437 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.717982 2584 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 06 16:24:09.928437 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.717985 2584 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 06 16:24:09.928437 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.717990 2584 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 06 16:24:09.928437 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.717993 2584 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 06 16:24:09.928437 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.717996 2584 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 06 16:24:09.928437 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.717998 2584 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 06 16:24:09.928437 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.718001 2584 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 06 16:24:09.928437 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.718004 2584 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 06 16:24:09.928437 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.718007 2584 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 06 16:24:09.928437 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.718010 2584 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 06 16:24:09.928437 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.718013 2584 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 06 16:24:09.928437 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.718016 2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 06 16:24:09.928437 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.718020 2584 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 06 16:24:09.928437 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.718022 2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 06 16:24:09.928437 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.718025 2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 06 16:24:09.928437 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.718028 2584 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 06 16:24:09.929115 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.718031 2584 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 06 16:24:09.929115 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.718035 2584 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 06 16:24:09.929115 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.718040 2584 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 06 16:24:09.929115 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.718043 2584 feature_gate.go:328] unrecognized feature gate: Example2 Jul 06 16:24:09.929115 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.718046 2584 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 06 16:24:09.929115 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.718049 2584 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 06 16:24:09.929115 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.718051 2584 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 06 16:24:09.929115 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.718054 2584 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 06 16:24:09.929115 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.718057 2584 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 06 16:24:09.929115 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.718060 2584 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 06 16:24:09.929115 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.718062 2584 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 06 16:24:09.929115 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.718066 2584 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 06 16:24:09.929115 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.718069 2584 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 06 16:24:09.929115 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.718072 2584 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 06 16:24:09.929115 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.718076 2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 06 16:24:09.929115 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.718079 2584 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 06 16:24:09.929115 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.718082 2584 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 06 16:24:09.929115 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.718085 2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 06 16:24:09.929115 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.718089 2584 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 06 16:24:09.929115 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.718092 2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 06 16:24:09.929826 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.718095 2584 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 06 16:24:09.929826 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.718098 2584 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 06 16:24:09.929826 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.718101 2584 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 06 16:24:09.929826 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.718105 2584 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 06 16:24:09.929826 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.718108 2584 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 06 16:24:09.929826 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.718111 2584 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 06 16:24:09.929826 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.718114 2584 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 06 16:24:09.929826 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.718116 2584 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 06 16:24:09.929826 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.718124 2584 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 06 16:24:09.929826 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.718127 2584 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 06 16:24:09.929826 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.718130 2584 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 06 16:24:09.929826 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.718133 2584 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 06 16:24:09.929826 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.718136 2584 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 06 16:24:09.929826 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.718138 2584 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 06 16:24:09.929826 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.718142 2584 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 06 16:24:09.929826 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.718146 2584 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 06 16:24:09.929826 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.718150 2584 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 06 16:24:09.929826 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.718153 2584 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 06 16:24:09.929826 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.718156 2584 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 06 16:24:09.929826 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.718158 2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 06 16:24:09.930497 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.718161 2584 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 06 16:24:09.930497 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.718164 2584 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 06 16:24:09.930497 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.718167 2584 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 06 16:24:09.930497 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.718170 2584 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 06 16:24:09.930497 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.718173 2584 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 06 16:24:09.930497 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.718176 2584 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 06 16:24:09.930497 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.718180 2584 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 06 16:24:09.930497 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.718183 2584 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 06 16:24:09.930497 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.718186 2584 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 06 16:24:09.930497 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.718189 2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 06 16:24:09.930497 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.718194 2584 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 06 16:24:09.930497 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.718197 2584 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 06 16:24:09.930497 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.718200 2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 06 16:24:09.930497 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.718203 2584 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 06 16:24:09.930497 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.718205 2584 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 06 16:24:09.930497 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.718208 2584 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 06 16:24:09.930497 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.718211 2584 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 06 16:24:09.930497 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.718214 2584 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 06 16:24:09.930497 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.718217 2584 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 06 16:24:09.930497 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.718220 2584 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 06 16:24:09.931143 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.718224 2584 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 06 16:24:09.931143 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.718228 2584 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 06 16:24:09.931143 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.718232 2584 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 06 16:24:09.931143 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.718235 2584 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 06 16:24:09.931143 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.718238 2584 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 06 16:24:09.931143 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.718241 2584 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 06 16:24:09.931143 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.718244 2584 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 06 16:24:09.931143 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.718247 2584 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 06 16:24:09.931143 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.718249 2584 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 06 16:24:09.931143 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.718252 2584 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 06 16:24:09.931143 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:09.718258 2584 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 06 16:24:09.931143 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.718399 2584 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 06 16:24:09.931143 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.718404 2584 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 06 16:24:09.931143 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.718407 2584 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 06 16:24:09.931650 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.718410 2584 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 06 16:24:09.931650 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.718413 2584 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 06 16:24:09.931650 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.718416 2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 06 16:24:09.931650 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.718419 2584 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 06 16:24:09.931650 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.718422 2584 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 06 16:24:09.931650 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.718427 2584 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 06 16:24:09.931650 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.718431 2584 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 06 16:24:09.931650 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.718434 2584 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 06 16:24:09.931650 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.718439 2584 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 06 16:24:09.931650 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.718442 2584 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 06 16:24:09.931650 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.718445 2584 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 06 16:24:09.931650 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.718448 2584 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 06 16:24:09.931650 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.718450 2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 06 16:24:09.931650 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.718453 2584 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 06 16:24:09.931650 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.718456 2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 06 16:24:09.931650 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.718460 2584 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 06 16:24:09.931650 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.718464 2584 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 06 16:24:09.931650 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.718467 2584 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 06 16:24:09.931650 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.718470 2584 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 06 16:24:09.931650 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.718473 2584 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 06 16:24:09.932257 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.718476 2584 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 06 16:24:09.932257 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.718479 2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 06 16:24:09.932257 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.718482 2584 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 06 16:24:09.932257 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.718485 2584 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 06 16:24:09.932257 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.718487 2584 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 06 16:24:09.932257 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.718490 2584 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 06 16:24:09.932257 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.718493 2584 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 06 16:24:09.932257 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.718496 2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 06 16:24:09.932257 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.718498 2584 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 06 16:24:09.932257 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.718501 2584 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 06 16:24:09.932257 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.718504 2584 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 06 16:24:09.932257 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.718507 2584 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 06 16:24:09.932257 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.718509 2584 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 06 16:24:09.932257 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.718512 2584 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 06 16:24:09.932257 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.718515 2584 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 06 16:24:09.932257 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.718518 2584 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 06 16:24:09.932257 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.718521 2584 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 06 16:24:09.932257 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.718525 2584 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 06 16:24:09.932257 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.718528 2584 feature_gate.go:328] unrecognized feature gate: Example Jul 06 16:24:09.932257 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.718531 2584 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 06 16:24:09.932877 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.718536 2584 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 06 16:24:09.932877 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.718540 2584 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 06 16:24:09.932877 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.718544 2584 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 06 16:24:09.932877 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.718547 2584 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 06 16:24:09.932877 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.718551 2584 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 06 16:24:09.932877 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.718555 2584 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 06 16:24:09.932877 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.718558 2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 06 16:24:09.932877 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.718561 2584 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 06 16:24:09.932877 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.718563 2584 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 06 16:24:09.932877 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.718566 2584 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 06 16:24:09.932877 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.718569 2584 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 06 16:24:09.932877 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.718571 2584 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 06 16:24:09.932877 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.718574 2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 06 16:24:09.932877 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.718578 2584 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 06 16:24:09.932877 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.718580 2584 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 06 16:24:09.932877 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.718584 2584 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 06 16:24:09.932877 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.718588 2584 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 06 16:24:09.932877 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.718591 2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 06 16:24:09.932877 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.718594 2584 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 06 16:24:09.933418 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.718596 2584 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 06 16:24:09.933418 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.718600 2584 feature_gate.go:328] unrecognized feature gate: Example2 Jul 06 16:24:09.933418 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.718602 2584 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 06 16:24:09.933418 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.718605 2584 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 06 16:24:09.933418 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.718608 2584 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 06 16:24:09.933418 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.718611 2584 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 06 16:24:09.933418 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.718613 2584 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 06 16:24:09.933418 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.718616 2584 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 06 16:24:09.933418 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.718619 2584 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 06 16:24:09.933418 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.718622 2584 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 06 16:24:09.933418 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.718626 2584 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 06 16:24:09.933418 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.718629 2584 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 06 16:24:09.933418 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.718632 2584 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 06 16:24:09.933418 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.718637 2584 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 06 16:24:09.933418 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.718640 2584 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 06 16:24:09.933418 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.718643 2584 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 06 16:24:09.933418 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.718646 2584 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 06 16:24:09.933418 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.718649 2584 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 06 16:24:09.933418 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.718652 2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 06 16:24:09.933418 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.718655 2584 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 06 16:24:09.934098 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.718657 2584 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 06 16:24:09.934098 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.718660 2584 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 06 16:24:09.934098 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.718663 2584 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 06 16:24:09.934098 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.718666 2584 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 06 16:24:09.934098 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.718668 2584 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 06 16:24:09.934098 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.718671 2584 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 06 16:24:09.934098 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.718674 2584 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 06 16:24:09.934098 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.718677 2584 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 06 16:24:09.934098 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.718679 2584 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 06 16:24:09.934098 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.718682 2584 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 06 16:24:09.934098 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.718685 2584 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 06 16:24:09.934098 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.718687 2584 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 06 16:24:09.934098 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.718690 2584 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 06 16:24:09.934098 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.718693 2584 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 06 16:24:09.934098 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.718696 2584 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 06 16:24:09.934098 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.718699 2584 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 06 16:24:09.934098 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:09.718701 2584 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 06 16:24:09.934912 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:09.718706 2584 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 06 16:24:09.934912 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:09.718859 2584 server.go:962] "Client rotation is on, will bootstrap in background" Jul 06 16:24:09.934912 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:09.721464 2584 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 06 16:24:09.934912 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:09.723309 2584 server.go:1019] "Starting client certificate rotation" Jul 06 16:24:09.934912 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:09.723418 2584 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 06 16:24:09.934912 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:09.723495 2584 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 06 16:24:09.934912 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:09.726269 2584 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 06 16:24:09.934912 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:09.729264 2584 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 06 16:24:09.934912 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:09.744868 2584 log.go:25] "Validated CRI v1 runtime API" Jul 06 16:24:09.934912 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:09.747697 2584 log.go:25] "Validated CRI v1 image API" Jul 06 16:24:09.934912 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:09.749495 2584 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 06 16:24:09.934912 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:09.753199 2584 fs.go:135] Filesystem UUIDs: map[335228fe-e394-47dc-b8b3-e8060d0e5935:/dev/nvme0n1p4 3f8a76f6-3015-479c-b6af-916568362acb:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2] Jul 06 16:24:09.935324 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:09.753227 2584 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 06 16:24:09.935488 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:09.760536 2584 manager.go:216] Machine: {Timestamp:2026-07-06 16:24:09.758622081 +0000 UTC m=+0.372762198 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3101206 MemoryCapacity:33164496896 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec23c7b0712b4c509264b50130fa9029 SystemUUID:ec23c7b0-712b-4c50-9264-b50130fa9029 BootID:3f8436a2-0548-4e5c-9389-995e0ad5c654 Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:76:10:55:4f:1b Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:76:10:55:4f:1b Speed:0 Mtu:9001} {Name:ovs-system MacAddress:6a:9d:87:f7:3d:26 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164496896 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 06 16:24:09.935488 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:09.760755 2584 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 06 16:24:09.935488 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:09.760965 2584 manager.go:232] Version: {KernelVersion:5.14.0-570.125.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260630-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 06 16:24:09.935488 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:09.762355 2584 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 06 16:24:09.935488 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:09.762397 2584 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-128-22.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 06 16:24:09.935488 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:09.764215 2584 topology_manager.go:138] "Creating topology manager with none policy" Jul 06 16:24:09.935488 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:09.764232 2584 container_manager_linux.go:309] "Creating device plugin manager" Jul 06 16:24:09.935488 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:09.764244 2584 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 06 16:24:09.935782 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:09.764975 2584 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 06 16:24:09.935782 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:09.766539 2584 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 06 16:24:09.935782 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:09.770775 2584 state_mem.go:36] "Initialized new in-memory state store" Jul 06 16:24:09.935782 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:09.798404 2584 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 06 16:24:09.935782 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:09.800025 2584 kubelet.go:486] "Attempting to sync node with API server" Jul 06 16:24:09.935782 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:09.800041 2584 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 06 16:24:09.935782 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:09.800056 2584 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 06 16:24:09.935782 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:09.800068 2584 kubelet.go:398] "Adding apiserver pod source" Jul 06 16:24:09.935782 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:09.800079 2584 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 06 16:24:09.935782 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:09.800584 2584 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 06 16:24:09.935782 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:09.800607 2584 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 06 16:24:09.935782 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:09.803222 2584 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 06 16:24:09.935782 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:09.803317 2584 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-qwtxj" Jul 06 16:24:09.935782 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:09.804769 2584 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.8-3.rhaos4.21.git91b4b0b.el9" apiVersion="v1" Jul 06 16:24:09.935782 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:09.807142 2584 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 06 16:24:09.935782 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:09.807171 2584 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 06 16:24:09.935782 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:09.807697 2584 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-qwtxj" Jul 06 16:24:09.935782 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:09.808726 2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 06 16:24:09.935782 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:09.808742 2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 06 16:24:09.935782 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:09.808748 2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 06 16:24:09.935782 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:09.808755 2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 06 16:24:09.935782 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:09.808774 2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 06 16:24:09.935782 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:09.808782 2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 06 16:24:09.935782 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:09.808790 2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 06 16:24:09.935782 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:09.808815 2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 06 16:24:09.935782 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:09.808824 2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 06 16:24:09.935782 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:09.808829 2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 06 16:24:09.935782 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:09.808845 2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 06 16:24:09.935782 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:09.808852 2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 06 16:24:09.935782 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:09.809537 2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 06 16:24:09.935782 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:09.809558 2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 06 16:24:09.935782 ip-10-0-128-22 kubenswrapper[2584]: E0706 16:24:09.813018 2584 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-128-22.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 06 16:24:09.935782 ip-10-0-128-22 kubenswrapper[2584]: E0706 16:24:09.813016 2584 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 06 16:24:09.935782 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:09.813199 2584 server.go:1268] "Started kubelet" Jul 06 16:24:09.935782 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:09.813300 2584 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 06 16:24:09.935782 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:09.813287 2584 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 06 16:24:09.935782 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:09.813417 2584 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 06 16:24:09.935782 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:09.815328 2584 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 06 16:24:09.935782 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:09.816131 2584 server.go:310] "Adding debug handlers to kubelet server" Jul 06 16:24:09.936988 ip-10-0-128-22 kubenswrapper[2584]: E0706 16:24:09.824638 2584 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 06 16:24:09.936988 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:09.825527 2584 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 06 16:24:09.936988 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:09.825544 2584 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 06 16:24:09.936988 ip-10-0-128-22 kubenswrapper[2584]: E0706 16:24:09.825640 2584 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-22.ec2.internal\" not found" Jul 06 16:24:09.936988 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:09.826277 2584 factory.go:55] Registering systemd factory Jul 06 16:24:09.936988 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:09.826294 2584 factory.go:223] Registration of the systemd container factory successfully Jul 06 16:24:09.936988 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:09.826414 2584 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 06 16:24:09.936988 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:09.826416 2584 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 06 16:24:09.936988 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:09.826467 2584 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 06 16:24:09.936988 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:09.826515 2584 factory.go:153] Registering CRI-O factory Jul 06 16:24:09.936988 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:09.826527 2584 factory.go:223] Registration of the crio container factory successfully Jul 06 16:24:09.936988 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:09.826597 2584 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 06 16:24:09.936988 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:09.826623 2584 factory.go:103] Registering Raw factory Jul 06 16:24:09.936988 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:09.826640 2584 manager.go:1196] Started watching for new ooms in manager Jul 06 16:24:09.936988 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:09.827167 2584 manager.go:349] Starting recovery of all containers Jul 06 16:24:09.936988 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:09.827562 2584 reconstruct.go:97] "Volume reconstruction finished" Jul 06 16:24:09.936988 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:09.827573 2584 reconciler.go:29] "Reconciler: start to sync state" Jul 06 16:24:09.936988 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:09.832683 2584 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 06 16:24:09.936988 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:09.832751 2584 nodeinfomanager.go:417] Failed to publish CSINode: nodes "ip-10-0-128-22.ec2.internal" not found Jul 06 16:24:09.936988 ip-10-0-128-22 kubenswrapper[2584]: E0706 16:24:09.835927 2584 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-128-22.ec2.internal\" not found" node="ip-10-0-128-22.ec2.internal" Jul 06 16:24:09.936988 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:09.854750 2584 manager.go:354] Recovery completed Jul 06 16:24:09.936988 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:09.864425 2584 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 06 16:24:09.936988 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:09.865253 2584 nodeinfomanager.go:417] Failed to publish CSINode: nodes "ip-10-0-128-22.ec2.internal" not found Jul 06 16:24:09.936988 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:09.866435 2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-22.ec2.internal" event="NodeHasSufficientMemory" Jul 06 16:24:09.936988 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:09.866468 2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-22.ec2.internal" event="NodeHasNoDiskPressure" Jul 06 16:24:09.936988 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:09.866480 2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-22.ec2.internal" event="NodeHasSufficientPID" Jul 06 16:24:09.936988 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:09.867060 2584 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 06 16:24:09.936988 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:09.867069 2584 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 06 16:24:09.936988 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:09.867087 2584 state_mem.go:36] "Initialized new in-memory state store" Jul 06 16:24:09.936988 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:09.867157 2584 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 06 16:24:09.936988 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:09.867936 2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-22.ec2.internal" event="NodeHasSufficientMemory" Jul 06 16:24:09.936988 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:09.867964 2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-22.ec2.internal" event="NodeHasNoDiskPressure" Jul 06 16:24:09.936988 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:09.867977 2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-22.ec2.internal" event="NodeHasSufficientPID" Jul 06 16:24:09.936988 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:09.868483 2584 policy_none.go:49] "None policy: Start" Jul 06 16:24:09.936988 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:09.868498 2584 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 06 16:24:09.936988 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:09.868508 2584 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 06 16:24:09.936988 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:09.869617 2584 policy_none.go:47] "Start" Jul 06 16:24:09.936988 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:09.917545 2584 manager.go:337] "Starting Device Plugin manager" Jul 06 16:24:09.936988 ip-10-0-128-22 kubenswrapper[2584]: E0706 16:24:09.917819 2584 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 06 16:24:09.936988 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:09.917834 2584 server.go:87] "Starting device plugin registration server" Jul 06 16:24:09.936988 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:09.918090 2584 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 06 16:24:09.936988 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:09.918100 2584 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 06 16:24:09.936988 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:09.918202 2584 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 06 16:24:09.936988 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:09.918298 2584 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 06 16:24:09.938191 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:09.918309 2584 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 06 16:24:09.938191 ip-10-0-128-22 kubenswrapper[2584]: E0706 16:24:09.918850 2584 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 06 16:24:09.938191 ip-10-0-128-22 kubenswrapper[2584]: E0706 16:24:09.918893 2584 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-128-22.ec2.internal\" not found" Jul 06 16:24:09.938191 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:09.923447 2584 nodeinfomanager.go:417] Failed to publish CSINode: nodes "ip-10-0-128-22.ec2.internal" not found Jul 06 16:24:10.133301 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:10.019622 2584 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 06 16:24:10.133301 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:10.020876 2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-22.ec2.internal" event="NodeHasSufficientMemory" Jul 06 16:24:10.133301 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:10.020911 2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-22.ec2.internal" event="NodeHasNoDiskPressure" Jul 06 16:24:10.133301 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:10.020923 2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-22.ec2.internal" event="NodeHasSufficientPID" Jul 06 16:24:10.133301 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:10.020956 2584 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-128-22.ec2.internal" Jul 06 16:24:10.133301 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:10.031956 2584 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-128-22.ec2.internal" Jul 06 16:24:10.133301 ip-10-0-128-22 kubenswrapper[2584]: E0706 16:24:10.031982 2584 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-128-22.ec2.internal\": node \"ip-10-0-128-22.ec2.internal\" not found" Jul 06 16:24:10.133301 ip-10-0-128-22 kubenswrapper[2584]: E0706 16:24:10.060459 2584 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-22.ec2.internal\" not found" Jul 06 16:24:10.133301 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:10.110388 2584 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 06 16:24:10.133301 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:10.111941 2584 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 06 16:24:10.133301 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:10.112007 2584 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 06 16:24:10.133301 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:10.112038 2584 kubelet.go:2448] "Starting kubelet main sync loop" Jul 06 16:24:10.133301 ip-10-0-128-22 kubenswrapper[2584]: E0706 16:24:10.112080 2584 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 06 16:24:10.133301 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:10.123780 2584 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 06 16:24:10.326103 ip-10-0-128-22 kubenswrapper[2584]: E0706 16:24:10.161624 2584 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-22.ec2.internal\" not found" Jul 06 16:24:10.326103 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:10.212136 2584 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-22.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-128-22.ec2.internal"] Jul 06 16:24:10.326103 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:10.212209 2584 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 06 16:24:10.326103 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:10.213492 2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-22.ec2.internal" event="NodeHasSufficientMemory" Jul 06 16:24:10.326103 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:10.213524 2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-22.ec2.internal" event="NodeHasNoDiskPressure" Jul 06 16:24:10.326103 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:10.213534 2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-22.ec2.internal" event="NodeHasSufficientPID" Jul 06 16:24:10.326103 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:10.214892 2584 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 06 16:24:10.326103 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:10.215001 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-22.ec2.internal" Jul 06 16:24:10.326103 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:10.215027 2584 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 06 16:24:10.326103 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:10.215652 2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-22.ec2.internal" event="NodeHasSufficientMemory" Jul 06 16:24:10.326103 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:10.215677 2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-22.ec2.internal" event="NodeHasNoDiskPressure" Jul 06 16:24:10.326103 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:10.215691 2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-22.ec2.internal" event="NodeHasSufficientPID" Jul 06 16:24:10.326103 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:10.215708 2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-22.ec2.internal" event="NodeHasSufficientMemory" Jul 06 16:24:10.326103 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:10.215750 2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-22.ec2.internal" event="NodeHasNoDiskPressure" Jul 06 16:24:10.326103 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:10.215762 2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-22.ec2.internal" event="NodeHasSufficientPID" Jul 06 16:24:10.326103 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:10.216894 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-22.ec2.internal" Jul 06 16:24:10.326103 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:10.216917 2584 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 06 16:24:10.326103 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:10.217553 2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-22.ec2.internal" event="NodeHasSufficientMemory" Jul 06 16:24:10.326103 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:10.217573 2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-22.ec2.internal" event="NodeHasNoDiskPressure" Jul 06 16:24:10.326103 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:10.217583 2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-22.ec2.internal" event="NodeHasSufficientPID" Jul 06 16:24:10.326103 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:10.229953 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/681ecc11fd0a8e367cd02518060abe2d-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-22.ec2.internal\" (UID: \"681ecc11fd0a8e367cd02518060abe2d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-22.ec2.internal" Jul 06 16:24:10.326103 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:10.229982 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/681ecc11fd0a8e367cd02518060abe2d-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-22.ec2.internal\" (UID: \"681ecc11fd0a8e367cd02518060abe2d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-22.ec2.internal" Jul 06 16:24:10.326103 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:10.230003 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/a4d219c131416ebd840bc1507523f196-config\") pod \"kube-apiserver-proxy-ip-10-0-128-22.ec2.internal\" (UID: \"a4d219c131416ebd840bc1507523f196\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-22.ec2.internal" Jul 06 16:24:10.326103 ip-10-0-128-22 kubenswrapper[2584]: E0706 16:24:10.257916 2584 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-128-22.ec2.internal\" not found" node="ip-10-0-128-22.ec2.internal" Jul 06 16:24:10.326103 ip-10-0-128-22 kubenswrapper[2584]: E0706 16:24:10.261853 2584 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-22.ec2.internal\" not found" Jul 06 16:24:10.326103 ip-10-0-128-22 kubenswrapper[2584]: E0706 16:24:10.262737 2584 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-128-22.ec2.internal\" not found" node="ip-10-0-128-22.ec2.internal" Jul 06 16:24:10.546662 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:10.330372 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/a4d219c131416ebd840bc1507523f196-config\") pod \"kube-apiserver-proxy-ip-10-0-128-22.ec2.internal\" (UID: \"a4d219c131416ebd840bc1507523f196\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-22.ec2.internal" Jul 06 16:24:10.546662 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:10.330405 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/681ecc11fd0a8e367cd02518060abe2d-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-22.ec2.internal\" (UID: \"681ecc11fd0a8e367cd02518060abe2d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-22.ec2.internal" Jul 06 16:24:10.546662 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:10.330426 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/681ecc11fd0a8e367cd02518060abe2d-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-22.ec2.internal\" (UID: \"681ecc11fd0a8e367cd02518060abe2d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-22.ec2.internal" Jul 06 16:24:10.546662 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:10.330480 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/681ecc11fd0a8e367cd02518060abe2d-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-22.ec2.internal\" (UID: \"681ecc11fd0a8e367cd02518060abe2d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-22.ec2.internal" Jul 06 16:24:10.546662 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:10.330483 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/a4d219c131416ebd840bc1507523f196-config\") pod \"kube-apiserver-proxy-ip-10-0-128-22.ec2.internal\" (UID: \"a4d219c131416ebd840bc1507523f196\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-22.ec2.internal" Jul 06 16:24:10.546662 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:10.330555 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/681ecc11fd0a8e367cd02518060abe2d-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-22.ec2.internal\" (UID: \"681ecc11fd0a8e367cd02518060abe2d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-22.ec2.internal" Jul 06 16:24:10.546662 ip-10-0-128-22 kubenswrapper[2584]: E0706 16:24:10.362629 2584 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-22.ec2.internal\" not found" Jul 06 16:24:10.546662 ip-10-0-128-22 kubenswrapper[2584]: E0706 16:24:10.464427 2584 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-22.ec2.internal\" not found" Jul 06 16:24:10.673685 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:10.561129 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-22.ec2.internal" Jul 06 16:24:10.673685 ip-10-0-128-22 kubenswrapper[2584]: E0706 16:24:10.565550 2584 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-22.ec2.internal\" not found" Jul 06 16:24:10.673685 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:10.565572 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-22.ec2.internal" Jul 06 16:24:10.673685 ip-10-0-128-22 kubenswrapper[2584]: E0706 16:24:10.666240 2584 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-22.ec2.internal\" not found" Jul 06 16:24:10.836590 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:10.722515 2584 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 06 16:24:10.836590 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:10.722669 2584 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 06 16:24:10.836590 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:10.722699 2584 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 06 16:24:10.836590 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:10.763861 2584 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 06 16:24:10.836590 ip-10-0-128-22 kubenswrapper[2584]: E0706 16:24:10.766740 2584 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-22.ec2.internal\" not found" Jul 06 16:24:10.836590 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:10.813600 2584 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-05 16:19:10 +0000 UTC" deadline="2028-01-21 21:20:18.367041836 +0000 UTC" Jul 06 16:24:10.836590 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:10.813650 2584 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13540h56m7.553395887s" Jul 06 16:24:11.028784 ip-10-0-128-22 kubenswrapper[2584]: E0706 16:24:10.867935 2584 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-22.ec2.internal\" not found" Jul 06 16:24:11.028784 ip-10-0-128-22 kubenswrapper[2584]: E0706 16:24:10.968221 2584 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-22.ec2.internal\" not found" Jul 06 16:24:11.176553 ip-10-0-128-22 kubenswrapper[2584]: E0706 16:24:11.069264 2584 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-22.ec2.internal\" not found" Jul 06 16:24:11.176553 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:11.149336 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda4d219c131416ebd840bc1507523f196.slice/crio-d84d6c434ae977dd9a488d3549e8844a1c0ec4b04bad2c5382106e076a210b6b WatchSource:0}: Error finding container d84d6c434ae977dd9a488d3549e8844a1c0ec4b04bad2c5382106e076a210b6b: Status 404 returned error can't find the container with id d84d6c434ae977dd9a488d3549e8844a1c0ec4b04bad2c5382106e076a210b6b Jul 06 16:24:11.176553 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:11.159352 2584 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 06 16:24:11.176553 ip-10-0-128-22 kubenswrapper[2584]: E0706 16:24:11.169721 2584 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-22.ec2.internal\" not found" Jul 06 16:24:11.391055 ip-10-0-128-22 kubenswrapper[2584]: E0706 16:24:11.272394 2584 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-22.ec2.internal\" not found" Jul 06 16:24:11.391055 ip-10-0-128-22 kubenswrapper[2584]: E0706 16:24:11.372477 2584 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-22.ec2.internal\" not found" Jul 06 16:24:11.391055 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:11.375617 2584 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 06 16:24:11.579805 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:11.428854 2584 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-22.ec2.internal" Jul 06 16:24:11.579805 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:11.446121 2584 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 06 16:24:11.579805 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:11.447387 2584 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-22.ec2.internal" Jul 06 16:24:11.579805 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:11.461271 2584 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 06 16:24:11.757766 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:11.620121 2584 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 06 16:24:11.851692 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:11.786173 2584 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 06 16:24:11.851692 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:11.801188 2584 apiserver.go:52] "Watching apiserver" Jul 06 16:24:11.851692 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:11.810677 2584 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 06 16:24:11.851692 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:11.812416 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/kube-apiserver-proxy-ip-10-0-128-22.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8mtdq","openshift-dns/node-resolver-wg98x","openshift-image-registry/node-ca-27rkm","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-22.ec2.internal","openshift-multus/multus-rrqzz","kube-system/konnectivity-agent-k4gbb","openshift-cluster-node-tuning-operator/tuned-gq74s","openshift-multus/multus-additional-cni-plugins-55rq5","openshift-multus/network-metrics-daemon-2zdgn","openshift-network-diagnostics/network-check-target-v8kjk","openshift-network-operator/iptables-alerter-smm2d","openshift-ovn-kubernetes/ovnkube-node-294bx","kube-system/global-pull-secret-syncer-wvcq5"] Jul 06 16:24:11.851692 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:11.814130 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-rrqzz" Jul 06 16:24:11.851692 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:11.815397 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-294bx" Jul 06 16:24:11.851692 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:11.816649 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-wg98x" Jul 06 16:24:11.851692 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:11.817442 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 06 16:24:11.851692 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:11.817467 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 06 16:24:11.851692 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:11.817442 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 06 16:24:11.851692 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:11.817782 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 06 16:24:11.851692 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:11.818031 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-smm2d" Jul 06 16:24:11.851692 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:11.819457 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-ljtkh\"" Jul 06 16:24:11.851692 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:11.819561 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 06 16:24:11.851692 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:11.819684 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-hv92n\"" Jul 06 16:24:11.851692 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:11.819798 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 06 16:24:11.851692 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:11.819841 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 06 16:24:11.851692 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:11.820068 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 06 16:24:11.851692 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:11.820077 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 06 16:24:11.851692 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:11.820072 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 06 16:24:11.851692 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:11.820538 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 06 16:24:11.851692 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:11.820541 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 06 16:24:11.851692 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:11.820586 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-bnnq4\"" Jul 06 16:24:11.851692 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:11.821081 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 06 16:24:11.851692 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:11.821088 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 06 16:24:11.851692 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:11.821160 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 06 16:24:11.851692 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:11.821320 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-pkzkx\"" Jul 06 16:24:11.851692 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:11.826907 2584 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 06 16:24:11.851692 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:11.829599 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-gq74s" Jul 06 16:24:11.851692 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:11.831078 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-k4gbb" Jul 06 16:24:11.855150 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:11.832614 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 06 16:24:11.855150 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:11.832849 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 06 16:24:11.855150 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:11.833136 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-hc6tv\"" Jul 06 16:24:11.855150 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:11.834128 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 06 16:24:11.855150 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:11.834145 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 06 16:24:11.855150 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:11.834224 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-ldrvh\"" Jul 06 16:24:11.855150 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:11.834444 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-55rq5" Jul 06 16:24:11.855150 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:11.835816 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2zdgn" Jul 06 16:24:11.855150 ip-10-0-128-22 kubenswrapper[2584]: E0706 16:24:11.835910 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2zdgn" podUID="3106072a-c698-44ac-9b92-8a7fd1e08d8a" Jul 06 16:24:11.855150 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:11.837456 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-v8kjk" Jul 06 16:24:11.855150 ip-10-0-128-22 kubenswrapper[2584]: E0706 16:24:11.837532 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-v8kjk" podUID="6c5de78f-76c0-4c6a-9c41-787f8452e5b4" Jul 06 16:24:11.855150 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:11.837698 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 06 16:24:11.855150 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:11.837733 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-7f9px\"" Jul 06 16:24:11.855150 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:11.837835 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 06 16:24:11.855150 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:11.838827 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-27rkm" Jul 06 16:24:11.855150 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:11.840603 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8mtdq" Jul 06 16:24:11.855150 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:11.840837 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3d80051d-4dd9-44d5-9be4-64a500f8c435-system-cni-dir\") pod \"multus-rrqzz\" (UID: \"3d80051d-4dd9-44d5-9be4-64a500f8c435\") " pod="openshift-multus/multus-rrqzz" Jul 06 16:24:11.855150 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:11.840868 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3d80051d-4dd9-44d5-9be4-64a500f8c435-multus-cni-dir\") pod \"multus-rrqzz\" (UID: \"3d80051d-4dd9-44d5-9be4-64a500f8c435\") " pod="openshift-multus/multus-rrqzz" Jul 06 16:24:11.855150 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:11.840895 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/6a8a4796-dbbc-43e2-a459-7acf11f9468d-ovnkube-config\") pod \"ovnkube-node-294bx\" (UID: \"6a8a4796-dbbc-43e2-a459-7acf11f9468d\") " pod="openshift-ovn-kubernetes/ovnkube-node-294bx" Jul 06 16:24:11.855150 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:11.840918 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/6a8a4796-dbbc-43e2-a459-7acf11f9468d-env-overrides\") pod \"ovnkube-node-294bx\" (UID: \"6a8a4796-dbbc-43e2-a459-7acf11f9468d\") " pod="openshift-ovn-kubernetes/ovnkube-node-294bx" Jul 06 16:24:11.855150 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:11.840945 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/6a8a4796-dbbc-43e2-a459-7acf11f9468d-ovnkube-script-lib\") pod \"ovnkube-node-294bx\" (UID: \"6a8a4796-dbbc-43e2-a459-7acf11f9468d\") " pod="openshift-ovn-kubernetes/ovnkube-node-294bx" Jul 06 16:24:11.857757 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:11.840975 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/3d80051d-4dd9-44d5-9be4-64a500f8c435-host-var-lib-cni-bin\") pod \"multus-rrqzz\" (UID: \"3d80051d-4dd9-44d5-9be4-64a500f8c435\") " pod="openshift-multus/multus-rrqzz" Jul 06 16:24:11.857757 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:11.841010 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/6a8a4796-dbbc-43e2-a459-7acf11f9468d-host-run-netns\") pod \"ovnkube-node-294bx\" (UID: \"6a8a4796-dbbc-43e2-a459-7acf11f9468d\") " pod="openshift-ovn-kubernetes/ovnkube-node-294bx" Jul 06 16:24:11.857757 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:11.841087 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6a8a4796-dbbc-43e2-a459-7acf11f9468d-var-lib-openvswitch\") pod \"ovnkube-node-294bx\" (UID: \"6a8a4796-dbbc-43e2-a459-7acf11f9468d\") " pod="openshift-ovn-kubernetes/ovnkube-node-294bx" Jul 06 16:24:11.857757 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:11.841135 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/6a8a4796-dbbc-43e2-a459-7acf11f9468d-host-cni-netd\") pod \"ovnkube-node-294bx\" (UID: \"6a8a4796-dbbc-43e2-a459-7acf11f9468d\") " pod="openshift-ovn-kubernetes/ovnkube-node-294bx" Jul 06 16:24:11.857757 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:11.841163 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/6a8a4796-dbbc-43e2-a459-7acf11f9468d-node-log\") pod \"ovnkube-node-294bx\" (UID: \"6a8a4796-dbbc-43e2-a459-7acf11f9468d\") " pod="openshift-ovn-kubernetes/ovnkube-node-294bx" Jul 06 16:24:11.857757 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:11.841187 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/28844609-46d4-4ebf-9c6a-e714ef3bba00-etc-sysctl-conf\") pod \"tuned-gq74s\" (UID: \"28844609-46d4-4ebf-9c6a-e714ef3bba00\") " pod="openshift-cluster-node-tuning-operator/tuned-gq74s" Jul 06 16:24:11.857757 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:11.841220 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/3d80051d-4dd9-44d5-9be4-64a500f8c435-host-run-multus-certs\") pod \"multus-rrqzz\" (UID: \"3d80051d-4dd9-44d5-9be4-64a500f8c435\") " pod="openshift-multus/multus-rrqzz" Jul 06 16:24:11.857757 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:11.841262 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/6a8a4796-dbbc-43e2-a459-7acf11f9468d-systemd-units\") pod \"ovnkube-node-294bx\" (UID: \"6a8a4796-dbbc-43e2-a459-7acf11f9468d\") " pod="openshift-ovn-kubernetes/ovnkube-node-294bx" Jul 06 16:24:11.857757 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:11.841301 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/6a8a4796-dbbc-43e2-a459-7acf11f9468d-host-slash\") pod \"ovnkube-node-294bx\" (UID: \"6a8a4796-dbbc-43e2-a459-7acf11f9468d\") " pod="openshift-ovn-kubernetes/ovnkube-node-294bx" Jul 06 16:24:11.857757 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:11.841330 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4c862983-24dd-471d-928b-aa352b202fd9-tmp-dir\") pod \"node-resolver-wg98x\" (UID: \"4c862983-24dd-471d-928b-aa352b202fd9\") " pod="openshift-dns/node-resolver-wg98x" Jul 06 16:24:11.857757 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:11.841361 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-psnvc\" (UniqueName: \"kubernetes.io/projected/28844609-46d4-4ebf-9c6a-e714ef3bba00-kube-api-access-psnvc\") pod \"tuned-gq74s\" (UID: \"28844609-46d4-4ebf-9c6a-e714ef3bba00\") " pod="openshift-cluster-node-tuning-operator/tuned-gq74s" Jul 06 16:24:11.857757 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:11.841410 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/3d80051d-4dd9-44d5-9be4-64a500f8c435-os-release\") pod \"multus-rrqzz\" (UID: \"3d80051d-4dd9-44d5-9be4-64a500f8c435\") " pod="openshift-multus/multus-rrqzz" Jul 06 16:24:11.857757 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:11.841446 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/3d80051d-4dd9-44d5-9be4-64a500f8c435-cni-binary-copy\") pod \"multus-rrqzz\" (UID: \"3d80051d-4dd9-44d5-9be4-64a500f8c435\") " pod="openshift-multus/multus-rrqzz" Jul 06 16:24:11.857757 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:11.841495 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3d80051d-4dd9-44d5-9be4-64a500f8c435-host-var-lib-kubelet\") pod \"multus-rrqzz\" (UID: \"3d80051d-4dd9-44d5-9be4-64a500f8c435\") " pod="openshift-multus/multus-rrqzz" Jul 06 16:24:11.858541 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:11.841521 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dtsdb\" (UniqueName: \"kubernetes.io/projected/3d80051d-4dd9-44d5-9be4-64a500f8c435-kube-api-access-dtsdb\") pod \"multus-rrqzz\" (UID: \"3d80051d-4dd9-44d5-9be4-64a500f8c435\") " pod="openshift-multus/multus-rrqzz" Jul 06 16:24:11.858541 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:11.841548 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cbhjk\" (UniqueName: \"kubernetes.io/projected/4c862983-24dd-471d-928b-aa352b202fd9-kube-api-access-cbhjk\") pod \"node-resolver-wg98x\" (UID: \"4c862983-24dd-471d-928b-aa352b202fd9\") " pod="openshift-dns/node-resolver-wg98x" Jul 06 16:24:11.858541 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:11.841583 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/28844609-46d4-4ebf-9c6a-e714ef3bba00-sys\") pod \"tuned-gq74s\" (UID: \"28844609-46d4-4ebf-9c6a-e714ef3bba00\") " pod="openshift-cluster-node-tuning-operator/tuned-gq74s" Jul 06 16:24:11.858541 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:11.841628 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/6d4323bb-289e-490c-938f-d4501d0fd81b-iptables-alerter-script\") pod \"iptables-alerter-smm2d\" (UID: \"6d4323bb-289e-490c-938f-d4501d0fd81b\") " pod="openshift-network-operator/iptables-alerter-smm2d" Jul 06 16:24:11.858541 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:11.841655 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/28844609-46d4-4ebf-9c6a-e714ef3bba00-etc-tuned\") pod \"tuned-gq74s\" (UID: \"28844609-46d4-4ebf-9c6a-e714ef3bba00\") " pod="openshift-cluster-node-tuning-operator/tuned-gq74s" Jul 06 16:24:11.858541 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:11.841691 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/28844609-46d4-4ebf-9c6a-e714ef3bba00-tmp\") pod \"tuned-gq74s\" (UID: \"28844609-46d4-4ebf-9c6a-e714ef3bba00\") " pod="openshift-cluster-node-tuning-operator/tuned-gq74s" Jul 06 16:24:11.858541 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:11.841716 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/3d80051d-4dd9-44d5-9be4-64a500f8c435-multus-socket-dir-parent\") pod \"multus-rrqzz\" (UID: \"3d80051d-4dd9-44d5-9be4-64a500f8c435\") " pod="openshift-multus/multus-rrqzz" Jul 06 16:24:11.858541 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:11.841781 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/3d80051d-4dd9-44d5-9be4-64a500f8c435-multus-daemon-config\") pod \"multus-rrqzz\" (UID: \"3d80051d-4dd9-44d5-9be4-64a500f8c435\") " pod="openshift-multus/multus-rrqzz" Jul 06 16:24:11.858541 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:11.841808 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/6a8a4796-dbbc-43e2-a459-7acf11f9468d-run-ovn\") pod \"ovnkube-node-294bx\" (UID: \"6a8a4796-dbbc-43e2-a459-7acf11f9468d\") " pod="openshift-ovn-kubernetes/ovnkube-node-294bx" Jul 06 16:24:11.858541 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:11.841827 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 06 16:24:11.858541 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:11.841838 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-glzns\" (UniqueName: \"kubernetes.io/projected/6d4323bb-289e-490c-938f-d4501d0fd81b-kube-api-access-glzns\") pod \"iptables-alerter-smm2d\" (UID: \"6d4323bb-289e-490c-938f-d4501d0fd81b\") " pod="openshift-network-operator/iptables-alerter-smm2d" Jul 06 16:24:11.858541 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:11.841871 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 06 16:24:11.858541 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:11.842029 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/28844609-46d4-4ebf-9c6a-e714ef3bba00-etc-modprobe-d\") pod \"tuned-gq74s\" (UID: \"28844609-46d4-4ebf-9c6a-e714ef3bba00\") " pod="openshift-cluster-node-tuning-operator/tuned-gq74s" Jul 06 16:24:11.858541 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:11.842049 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-k5kdj\"" Jul 06 16:24:11.858541 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:11.842060 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/28844609-46d4-4ebf-9c6a-e714ef3bba00-etc-kubernetes\") pod \"tuned-gq74s\" (UID: \"28844609-46d4-4ebf-9c6a-e714ef3bba00\") " pod="openshift-cluster-node-tuning-operator/tuned-gq74s" Jul 06 16:24:11.859291 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:11.842098 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/28844609-46d4-4ebf-9c6a-e714ef3bba00-run\") pod \"tuned-gq74s\" (UID: \"28844609-46d4-4ebf-9c6a-e714ef3bba00\") " pod="openshift-cluster-node-tuning-operator/tuned-gq74s" Jul 06 16:24:11.859291 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:11.842114 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 06 16:24:11.859291 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:11.842035 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wvcq5" Jul 06 16:24:11.859291 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:11.842121 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/3d80051d-4dd9-44d5-9be4-64a500f8c435-multus-conf-dir\") pod \"multus-rrqzz\" (UID: \"3d80051d-4dd9-44d5-9be4-64a500f8c435\") " pod="openshift-multus/multus-rrqzz" Jul 06 16:24:11.859291 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:11.842197 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/6a8a4796-dbbc-43e2-a459-7acf11f9468d-host-kubelet\") pod \"ovnkube-node-294bx\" (UID: \"6a8a4796-dbbc-43e2-a459-7acf11f9468d\") " pod="openshift-ovn-kubernetes/ovnkube-node-294bx" Jul 06 16:24:11.859291 ip-10-0-128-22 kubenswrapper[2584]: E0706 16:24:11.842196 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wvcq5" podUID="d695ec8b-3624-40aa-afe2-841d320d0a66" Jul 06 16:24:11.859291 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:11.842222 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6a8a4796-dbbc-43e2-a459-7acf11f9468d-etc-openvswitch\") pod \"ovnkube-node-294bx\" (UID: \"6a8a4796-dbbc-43e2-a459-7acf11f9468d\") " pod="openshift-ovn-kubernetes/ovnkube-node-294bx" Jul 06 16:24:11.859291 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:11.842256 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/28844609-46d4-4ebf-9c6a-e714ef3bba00-etc-sysctl-d\") pod \"tuned-gq74s\" (UID: \"28844609-46d4-4ebf-9c6a-e714ef3bba00\") " pod="openshift-cluster-node-tuning-operator/tuned-gq74s" Jul 06 16:24:11.859291 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:11.842280 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/28844609-46d4-4ebf-9c6a-e714ef3bba00-etc-systemd\") pod \"tuned-gq74s\" (UID: \"28844609-46d4-4ebf-9c6a-e714ef3bba00\") " pod="openshift-cluster-node-tuning-operator/tuned-gq74s" Jul 06 16:24:11.859291 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:11.842304 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/3d80051d-4dd9-44d5-9be4-64a500f8c435-host-run-netns\") pod \"multus-rrqzz\" (UID: \"3d80051d-4dd9-44d5-9be4-64a500f8c435\") " pod="openshift-multus/multus-rrqzz" Jul 06 16:24:11.859291 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:11.842334 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/3d80051d-4dd9-44d5-9be4-64a500f8c435-cnibin\") pod \"multus-rrqzz\" (UID: \"3d80051d-4dd9-44d5-9be4-64a500f8c435\") " pod="openshift-multus/multus-rrqzz" Jul 06 16:24:11.859291 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:11.842356 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/3d80051d-4dd9-44d5-9be4-64a500f8c435-host-run-k8s-cni-cncf-io\") pod \"multus-rrqzz\" (UID: \"3d80051d-4dd9-44d5-9be4-64a500f8c435\") " pod="openshift-multus/multus-rrqzz" Jul 06 16:24:11.859291 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:11.842380 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/3d80051d-4dd9-44d5-9be4-64a500f8c435-hostroot\") pod \"multus-rrqzz\" (UID: \"3d80051d-4dd9-44d5-9be4-64a500f8c435\") " pod="openshift-multus/multus-rrqzz" Jul 06 16:24:11.859291 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:11.842414 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3d80051d-4dd9-44d5-9be4-64a500f8c435-etc-kubernetes\") pod \"multus-rrqzz\" (UID: \"3d80051d-4dd9-44d5-9be4-64a500f8c435\") " pod="openshift-multus/multus-rrqzz" Jul 06 16:24:11.859291 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:11.842455 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/6a8a4796-dbbc-43e2-a459-7acf11f9468d-ovn-node-metrics-cert\") pod \"ovnkube-node-294bx\" (UID: \"6a8a4796-dbbc-43e2-a459-7acf11f9468d\") " pod="openshift-ovn-kubernetes/ovnkube-node-294bx" Jul 06 16:24:11.859909 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:11.842480 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/28844609-46d4-4ebf-9c6a-e714ef3bba00-var-lib-kubelet\") pod \"tuned-gq74s\" (UID: \"28844609-46d4-4ebf-9c6a-e714ef3bba00\") " pod="openshift-cluster-node-tuning-operator/tuned-gq74s" Jul 06 16:24:11.859909 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:11.842503 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6a8a4796-dbbc-43e2-a459-7acf11f9468d-host-run-ovn-kubernetes\") pod \"ovnkube-node-294bx\" (UID: \"6a8a4796-dbbc-43e2-a459-7acf11f9468d\") " pod="openshift-ovn-kubernetes/ovnkube-node-294bx" Jul 06 16:24:11.859909 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:11.842529 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/6a8a4796-dbbc-43e2-a459-7acf11f9468d-host-cni-bin\") pod \"ovnkube-node-294bx\" (UID: \"6a8a4796-dbbc-43e2-a459-7acf11f9468d\") " pod="openshift-ovn-kubernetes/ovnkube-node-294bx" Jul 06 16:24:11.859909 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:11.842562 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6a8a4796-dbbc-43e2-a459-7acf11f9468d-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-294bx\" (UID: \"6a8a4796-dbbc-43e2-a459-7acf11f9468d\") " pod="openshift-ovn-kubernetes/ovnkube-node-294bx" Jul 06 16:24:11.859909 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:11.842589 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/6a8a4796-dbbc-43e2-a459-7acf11f9468d-run-systemd\") pod \"ovnkube-node-294bx\" (UID: \"6a8a4796-dbbc-43e2-a459-7acf11f9468d\") " pod="openshift-ovn-kubernetes/ovnkube-node-294bx" Jul 06 16:24:11.859909 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:11.842612 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6a8a4796-dbbc-43e2-a459-7acf11f9468d-run-openvswitch\") pod \"ovnkube-node-294bx\" (UID: \"6a8a4796-dbbc-43e2-a459-7acf11f9468d\") " pod="openshift-ovn-kubernetes/ovnkube-node-294bx" Jul 06 16:24:11.859909 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:11.842636 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-59ftq\" (UniqueName: \"kubernetes.io/projected/6a8a4796-dbbc-43e2-a459-7acf11f9468d-kube-api-access-59ftq\") pod \"ovnkube-node-294bx\" (UID: \"6a8a4796-dbbc-43e2-a459-7acf11f9468d\") " pod="openshift-ovn-kubernetes/ovnkube-node-294bx" Jul 06 16:24:11.859909 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:11.842659 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/4c862983-24dd-471d-928b-aa352b202fd9-hosts-file\") pod \"node-resolver-wg98x\" (UID: \"4c862983-24dd-471d-928b-aa352b202fd9\") " pod="openshift-dns/node-resolver-wg98x" Jul 06 16:24:11.859909 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:11.842682 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/28844609-46d4-4ebf-9c6a-e714ef3bba00-etc-sysconfig\") pod \"tuned-gq74s\" (UID: \"28844609-46d4-4ebf-9c6a-e714ef3bba00\") " pod="openshift-cluster-node-tuning-operator/tuned-gq74s" Jul 06 16:24:11.859909 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:11.842721 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/3d80051d-4dd9-44d5-9be4-64a500f8c435-host-var-lib-cni-multus\") pod \"multus-rrqzz\" (UID: \"3d80051d-4dd9-44d5-9be4-64a500f8c435\") " pod="openshift-multus/multus-rrqzz" Jul 06 16:24:11.859909 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:11.842752 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/6a8a4796-dbbc-43e2-a459-7acf11f9468d-log-socket\") pod \"ovnkube-node-294bx\" (UID: \"6a8a4796-dbbc-43e2-a459-7acf11f9468d\") " pod="openshift-ovn-kubernetes/ovnkube-node-294bx" Jul 06 16:24:11.859909 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:11.842815 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/6d4323bb-289e-490c-938f-d4501d0fd81b-host-slash\") pod \"iptables-alerter-smm2d\" (UID: \"6d4323bb-289e-490c-938f-d4501d0fd81b\") " pod="openshift-network-operator/iptables-alerter-smm2d" Jul 06 16:24:11.859909 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:11.842856 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/28844609-46d4-4ebf-9c6a-e714ef3bba00-lib-modules\") pod \"tuned-gq74s\" (UID: \"28844609-46d4-4ebf-9c6a-e714ef3bba00\") " pod="openshift-cluster-node-tuning-operator/tuned-gq74s" Jul 06 16:24:11.859909 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:11.842883 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/28844609-46d4-4ebf-9c6a-e714ef3bba00-host\") pod \"tuned-gq74s\" (UID: \"28844609-46d4-4ebf-9c6a-e714ef3bba00\") " pod="openshift-cluster-node-tuning-operator/tuned-gq74s" Jul 06 16:24:11.860348 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:11.843826 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 06 16:24:11.860348 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:11.845415 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 06 16:24:11.860348 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:11.845598 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-vstl9\"" Jul 06 16:24:11.860348 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:11.845607 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 06 16:24:11.860348 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:11.854086 2584 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 06 16:24:11.881233 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:11.881203 2584 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-zmsft" Jul 06 16:24:11.889832 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:11.889804 2584 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-zmsft" Jul 06 16:24:11.928702 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:11.928675 2584 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 06 16:24:11.946307 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:11.943796 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a40a609b-06a9-48c2-a98a-dc948e54049c-tuning-conf-dir\") pod \"multus-additional-cni-plugins-55rq5\" (UID: \"a40a609b-06a9-48c2-a98a-dc948e54049c\") " pod="openshift-multus/multus-additional-cni-plugins-55rq5" Jul 06 16:24:11.946307 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:11.943841 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vqnr6\" (UniqueName: \"kubernetes.io/projected/3106072a-c698-44ac-9b92-8a7fd1e08d8a-kube-api-access-vqnr6\") pod \"network-metrics-daemon-2zdgn\" (UID: \"3106072a-c698-44ac-9b92-8a7fd1e08d8a\") " pod="openshift-multus/network-metrics-daemon-2zdgn" Jul 06 16:24:11.946307 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:11.943879 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/6a8a4796-dbbc-43e2-a459-7acf11f9468d-node-log\") pod \"ovnkube-node-294bx\" (UID: \"6a8a4796-dbbc-43e2-a459-7acf11f9468d\") " pod="openshift-ovn-kubernetes/ovnkube-node-294bx" Jul 06 16:24:11.946307 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:11.943907 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/28844609-46d4-4ebf-9c6a-e714ef3bba00-etc-sysctl-conf\") pod \"tuned-gq74s\" (UID: \"28844609-46d4-4ebf-9c6a-e714ef3bba00\") " pod="openshift-cluster-node-tuning-operator/tuned-gq74s" Jul 06 16:24:11.946307 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:11.943934 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a40a609b-06a9-48c2-a98a-dc948e54049c-system-cni-dir\") pod \"multus-additional-cni-plugins-55rq5\" (UID: \"a40a609b-06a9-48c2-a98a-dc948e54049c\") " pod="openshift-multus/multus-additional-cni-plugins-55rq5" Jul 06 16:24:11.946307 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:11.943961 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/3d80051d-4dd9-44d5-9be4-64a500f8c435-host-run-multus-certs\") pod \"multus-rrqzz\" (UID: \"3d80051d-4dd9-44d5-9be4-64a500f8c435\") " pod="openshift-multus/multus-rrqzz" Jul 06 16:24:11.946307 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:11.943980 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/6a8a4796-dbbc-43e2-a459-7acf11f9468d-node-log\") pod \"ovnkube-node-294bx\" (UID: \"6a8a4796-dbbc-43e2-a459-7acf11f9468d\") " pod="openshift-ovn-kubernetes/ovnkube-node-294bx" Jul 06 16:24:11.946307 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:11.944029 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/6a8a4796-dbbc-43e2-a459-7acf11f9468d-systemd-units\") pod \"ovnkube-node-294bx\" (UID: \"6a8a4796-dbbc-43e2-a459-7acf11f9468d\") " pod="openshift-ovn-kubernetes/ovnkube-node-294bx" Jul 06 16:24:11.946307 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:11.943986 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/6a8a4796-dbbc-43e2-a459-7acf11f9468d-systemd-units\") pod \"ovnkube-node-294bx\" (UID: \"6a8a4796-dbbc-43e2-a459-7acf11f9468d\") " pod="openshift-ovn-kubernetes/ovnkube-node-294bx" Jul 06 16:24:11.946307 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:11.944060 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/6a8a4796-dbbc-43e2-a459-7acf11f9468d-host-slash\") pod \"ovnkube-node-294bx\" (UID: \"6a8a4796-dbbc-43e2-a459-7acf11f9468d\") " pod="openshift-ovn-kubernetes/ovnkube-node-294bx" Jul 06 16:24:11.946307 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:11.944084 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4c862983-24dd-471d-928b-aa352b202fd9-tmp-dir\") pod \"node-resolver-wg98x\" (UID: \"4c862983-24dd-471d-928b-aa352b202fd9\") " pod="openshift-dns/node-resolver-wg98x" Jul 06 16:24:11.946307 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:11.944106 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-psnvc\" (UniqueName: \"kubernetes.io/projected/28844609-46d4-4ebf-9c6a-e714ef3bba00-kube-api-access-psnvc\") pod \"tuned-gq74s\" (UID: \"28844609-46d4-4ebf-9c6a-e714ef3bba00\") " pod="openshift-cluster-node-tuning-operator/tuned-gq74s" Jul 06 16:24:11.946307 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:11.944138 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-shdb5\" (UniqueName: \"kubernetes.io/projected/656a2b09-7d06-48e6-9ddf-c261858ab728-kube-api-access-shdb5\") pod \"aws-ebs-csi-driver-node-8mtdq\" (UID: \"656a2b09-7d06-48e6-9ddf-c261858ab728\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8mtdq" Jul 06 16:24:11.946307 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:11.944152 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/28844609-46d4-4ebf-9c6a-e714ef3bba00-etc-sysctl-conf\") pod \"tuned-gq74s\" (UID: \"28844609-46d4-4ebf-9c6a-e714ef3bba00\") " pod="openshift-cluster-node-tuning-operator/tuned-gq74s" Jul 06 16:24:11.947147 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:11.944168 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/3d80051d-4dd9-44d5-9be4-64a500f8c435-os-release\") pod \"multus-rrqzz\" (UID: \"3d80051d-4dd9-44d5-9be4-64a500f8c435\") " pod="openshift-multus/multus-rrqzz" Jul 06 16:24:11.947147 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:11.944197 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/3d80051d-4dd9-44d5-9be4-64a500f8c435-cni-binary-copy\") pod \"multus-rrqzz\" (UID: \"3d80051d-4dd9-44d5-9be4-64a500f8c435\") " pod="openshift-multus/multus-rrqzz" Jul 06 16:24:11.947147 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:11.944224 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/3d80051d-4dd9-44d5-9be4-64a500f8c435-host-run-multus-certs\") pod \"multus-rrqzz\" (UID: \"3d80051d-4dd9-44d5-9be4-64a500f8c435\") " pod="openshift-multus/multus-rrqzz" Jul 06 16:24:11.947147 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:11.944226 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3d80051d-4dd9-44d5-9be4-64a500f8c435-host-var-lib-kubelet\") pod \"multus-rrqzz\" (UID: \"3d80051d-4dd9-44d5-9be4-64a500f8c435\") " pod="openshift-multus/multus-rrqzz" Jul 06 16:24:11.947147 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:11.944264 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/a40a609b-06a9-48c2-a98a-dc948e54049c-os-release\") pod \"multus-additional-cni-plugins-55rq5\" (UID: \"a40a609b-06a9-48c2-a98a-dc948e54049c\") " pod="openshift-multus/multus-additional-cni-plugins-55rq5" Jul 06 16:24:11.947147 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:11.944293 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/d695ec8b-3624-40aa-afe2-841d320d0a66-dbus\") pod \"global-pull-secret-syncer-wvcq5\" (UID: \"d695ec8b-3624-40aa-afe2-841d320d0a66\") " pod="kube-system/global-pull-secret-syncer-wvcq5" Jul 06 16:24:11.947147 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:11.944298 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/6a8a4796-dbbc-43e2-a459-7acf11f9468d-host-slash\") pod \"ovnkube-node-294bx\" (UID: \"6a8a4796-dbbc-43e2-a459-7acf11f9468d\") " pod="openshift-ovn-kubernetes/ovnkube-node-294bx" Jul 06 16:24:11.947147 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:11.944321 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dtsdb\" (UniqueName: \"kubernetes.io/projected/3d80051d-4dd9-44d5-9be4-64a500f8c435-kube-api-access-dtsdb\") pod \"multus-rrqzz\" (UID: \"3d80051d-4dd9-44d5-9be4-64a500f8c435\") " pod="openshift-multus/multus-rrqzz" Jul 06 16:24:11.947147 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:11.944347 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cbhjk\" (UniqueName: \"kubernetes.io/projected/4c862983-24dd-471d-928b-aa352b202fd9-kube-api-access-cbhjk\") pod \"node-resolver-wg98x\" (UID: \"4c862983-24dd-471d-928b-aa352b202fd9\") " pod="openshift-dns/node-resolver-wg98x" Jul 06 16:24:11.947147 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:11.944371 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/28844609-46d4-4ebf-9c6a-e714ef3bba00-sys\") pod \"tuned-gq74s\" (UID: \"28844609-46d4-4ebf-9c6a-e714ef3bba00\") " pod="openshift-cluster-node-tuning-operator/tuned-gq74s" Jul 06 16:24:11.947147 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:11.944396 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d695ec8b-3624-40aa-afe2-841d320d0a66-original-pull-secret\") pod \"global-pull-secret-syncer-wvcq5\" (UID: \"d695ec8b-3624-40aa-afe2-841d320d0a66\") " pod="kube-system/global-pull-secret-syncer-wvcq5" Jul 06 16:24:11.947147 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:11.944423 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/6d4323bb-289e-490c-938f-d4501d0fd81b-iptables-alerter-script\") pod \"iptables-alerter-smm2d\" (UID: \"6d4323bb-289e-490c-938f-d4501d0fd81b\") " pod="openshift-network-operator/iptables-alerter-smm2d" Jul 06 16:24:11.947147 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:11.944448 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/28844609-46d4-4ebf-9c6a-e714ef3bba00-etc-tuned\") pod \"tuned-gq74s\" (UID: \"28844609-46d4-4ebf-9c6a-e714ef3bba00\") " pod="openshift-cluster-node-tuning-operator/tuned-gq74s" Jul 06 16:24:11.947147 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:11.944473 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/28844609-46d4-4ebf-9c6a-e714ef3bba00-tmp\") pod \"tuned-gq74s\" (UID: \"28844609-46d4-4ebf-9c6a-e714ef3bba00\") " pod="openshift-cluster-node-tuning-operator/tuned-gq74s" Jul 06 16:24:11.947904 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:11.944501 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/9bf128db-4ed4-483d-a886-cb334e6fa4ef-konnectivity-ca\") pod \"konnectivity-agent-k4gbb\" (UID: \"9bf128db-4ed4-483d-a886-cb334e6fa4ef\") " pod="kube-system/konnectivity-agent-k4gbb" Jul 06 16:24:11.947904 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:11.944530 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/656a2b09-7d06-48e6-9ddf-c261858ab728-socket-dir\") pod \"aws-ebs-csi-driver-node-8mtdq\" (UID: \"656a2b09-7d06-48e6-9ddf-c261858ab728\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8mtdq" Jul 06 16:24:11.947904 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:11.944556 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/3d80051d-4dd9-44d5-9be4-64a500f8c435-multus-socket-dir-parent\") pod \"multus-rrqzz\" (UID: \"3d80051d-4dd9-44d5-9be4-64a500f8c435\") " pod="openshift-multus/multus-rrqzz" Jul 06 16:24:11.947904 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:11.944581 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/3d80051d-4dd9-44d5-9be4-64a500f8c435-multus-daemon-config\") pod \"multus-rrqzz\" (UID: \"3d80051d-4dd9-44d5-9be4-64a500f8c435\") " pod="openshift-multus/multus-rrqzz" Jul 06 16:24:11.947904 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:11.944610 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/6a8a4796-dbbc-43e2-a459-7acf11f9468d-run-ovn\") pod \"ovnkube-node-294bx\" (UID: \"6a8a4796-dbbc-43e2-a459-7acf11f9468d\") " pod="openshift-ovn-kubernetes/ovnkube-node-294bx" Jul 06 16:24:11.947904 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:11.944636 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-glzns\" (UniqueName: \"kubernetes.io/projected/6d4323bb-289e-490c-938f-d4501d0fd81b-kube-api-access-glzns\") pod \"iptables-alerter-smm2d\" (UID: \"6d4323bb-289e-490c-938f-d4501d0fd81b\") " pod="openshift-network-operator/iptables-alerter-smm2d" Jul 06 16:24:11.947904 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:11.944642 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4c862983-24dd-471d-928b-aa352b202fd9-tmp-dir\") pod \"node-resolver-wg98x\" (UID: \"4c862983-24dd-471d-928b-aa352b202fd9\") " pod="openshift-dns/node-resolver-wg98x" Jul 06 16:24:11.947904 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:11.944661 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/28844609-46d4-4ebf-9c6a-e714ef3bba00-etc-modprobe-d\") pod \"tuned-gq74s\" (UID: \"28844609-46d4-4ebf-9c6a-e714ef3bba00\") " pod="openshift-cluster-node-tuning-operator/tuned-gq74s" Jul 06 16:24:11.947904 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:11.944710 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/28844609-46d4-4ebf-9c6a-e714ef3bba00-etc-kubernetes\") pod \"tuned-gq74s\" (UID: \"28844609-46d4-4ebf-9c6a-e714ef3bba00\") " pod="openshift-cluster-node-tuning-operator/tuned-gq74s" Jul 06 16:24:11.947904 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:11.944736 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/28844609-46d4-4ebf-9c6a-e714ef3bba00-run\") pod \"tuned-gq74s\" (UID: \"28844609-46d4-4ebf-9c6a-e714ef3bba00\") " pod="openshift-cluster-node-tuning-operator/tuned-gq74s" Jul 06 16:24:11.947904 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:11.944768 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/a40a609b-06a9-48c2-a98a-dc948e54049c-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-55rq5\" (UID: \"a40a609b-06a9-48c2-a98a-dc948e54049c\") " pod="openshift-multus/multus-additional-cni-plugins-55rq5" Jul 06 16:24:11.947904 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:11.944797 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/3d80051d-4dd9-44d5-9be4-64a500f8c435-multus-conf-dir\") pod \"multus-rrqzz\" (UID: \"3d80051d-4dd9-44d5-9be4-64a500f8c435\") " pod="openshift-multus/multus-rrqzz" Jul 06 16:24:11.947904 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:11.944838 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/6a8a4796-dbbc-43e2-a459-7acf11f9468d-host-kubelet\") pod \"ovnkube-node-294bx\" (UID: \"6a8a4796-dbbc-43e2-a459-7acf11f9468d\") " pod="openshift-ovn-kubernetes/ovnkube-node-294bx" Jul 06 16:24:11.947904 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:11.944872 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6a8a4796-dbbc-43e2-a459-7acf11f9468d-etc-openvswitch\") pod \"ovnkube-node-294bx\" (UID: \"6a8a4796-dbbc-43e2-a459-7acf11f9468d\") " pod="openshift-ovn-kubernetes/ovnkube-node-294bx" Jul 06 16:24:11.948513 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:11.944900 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/28844609-46d4-4ebf-9c6a-e714ef3bba00-etc-sysctl-d\") pod \"tuned-gq74s\" (UID: \"28844609-46d4-4ebf-9c6a-e714ef3bba00\") " pod="openshift-cluster-node-tuning-operator/tuned-gq74s" Jul 06 16:24:11.948513 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:11.944926 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/28844609-46d4-4ebf-9c6a-e714ef3bba00-etc-systemd\") pod \"tuned-gq74s\" (UID: \"28844609-46d4-4ebf-9c6a-e714ef3bba00\") " pod="openshift-cluster-node-tuning-operator/tuned-gq74s" Jul 06 16:24:11.948513 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:11.944956 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bv7nz\" (UniqueName: \"kubernetes.io/projected/a40a609b-06a9-48c2-a98a-dc948e54049c-kube-api-access-bv7nz\") pod \"multus-additional-cni-plugins-55rq5\" (UID: \"a40a609b-06a9-48c2-a98a-dc948e54049c\") " pod="openshift-multus/multus-additional-cni-plugins-55rq5" Jul 06 16:24:11.948513 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:11.945007 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3106072a-c698-44ac-9b92-8a7fd1e08d8a-metrics-certs\") pod \"network-metrics-daemon-2zdgn\" (UID: \"3106072a-c698-44ac-9b92-8a7fd1e08d8a\") " pod="openshift-multus/network-metrics-daemon-2zdgn" Jul 06 16:24:11.948513 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:11.945035 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/656a2b09-7d06-48e6-9ddf-c261858ab728-device-dir\") pod \"aws-ebs-csi-driver-node-8mtdq\" (UID: \"656a2b09-7d06-48e6-9ddf-c261858ab728\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8mtdq" Jul 06 16:24:11.948513 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:11.945062 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jz4gp\" (UniqueName: \"kubernetes.io/projected/6c5de78f-76c0-4c6a-9c41-787f8452e5b4-kube-api-access-jz4gp\") pod \"network-check-target-v8kjk\" (UID: \"6c5de78f-76c0-4c6a-9c41-787f8452e5b4\") " pod="openshift-network-diagnostics/network-check-target-v8kjk" Jul 06 16:24:11.948513 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:11.945101 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/3d80051d-4dd9-44d5-9be4-64a500f8c435-os-release\") pod \"multus-rrqzz\" (UID: \"3d80051d-4dd9-44d5-9be4-64a500f8c435\") " pod="openshift-multus/multus-rrqzz" Jul 06 16:24:11.948513 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:11.944266 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3d80051d-4dd9-44d5-9be4-64a500f8c435-host-var-lib-kubelet\") pod \"multus-rrqzz\" (UID: \"3d80051d-4dd9-44d5-9be4-64a500f8c435\") " pod="openshift-multus/multus-rrqzz" Jul 06 16:24:11.948513 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:11.945105 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/9bf128db-4ed4-483d-a886-cb334e6fa4ef-agent-certs\") pod \"konnectivity-agent-k4gbb\" (UID: \"9bf128db-4ed4-483d-a886-cb334e6fa4ef\") " pod="kube-system/konnectivity-agent-k4gbb" Jul 06 16:24:11.948513 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:11.945391 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/d695ec8b-3624-40aa-afe2-841d320d0a66-kubelet-config\") pod \"global-pull-secret-syncer-wvcq5\" (UID: \"d695ec8b-3624-40aa-afe2-841d320d0a66\") " pod="kube-system/global-pull-secret-syncer-wvcq5" Jul 06 16:24:11.948513 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:11.945428 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/3d80051d-4dd9-44d5-9be4-64a500f8c435-host-run-netns\") pod \"multus-rrqzz\" (UID: \"3d80051d-4dd9-44d5-9be4-64a500f8c435\") " pod="openshift-multus/multus-rrqzz" Jul 06 16:24:11.948513 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:11.945459 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a40a609b-06a9-48c2-a98a-dc948e54049c-cni-binary-copy\") pod \"multus-additional-cni-plugins-55rq5\" (UID: \"a40a609b-06a9-48c2-a98a-dc948e54049c\") " pod="openshift-multus/multus-additional-cni-plugins-55rq5" Jul 06 16:24:11.948513 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:11.945484 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/e98b6fdb-3725-4c37-8a97-6a59fbc25f76-host\") pod \"node-ca-27rkm\" (UID: \"e98b6fdb-3725-4c37-8a97-6a59fbc25f76\") " pod="openshift-image-registry/node-ca-27rkm" Jul 06 16:24:11.948513 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:11.945522 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/656a2b09-7d06-48e6-9ddf-c261858ab728-etc-selinux\") pod \"aws-ebs-csi-driver-node-8mtdq\" (UID: \"656a2b09-7d06-48e6-9ddf-c261858ab728\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8mtdq" Jul 06 16:24:11.949193 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:11.945553 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/3d80051d-4dd9-44d5-9be4-64a500f8c435-cnibin\") pod \"multus-rrqzz\" (UID: \"3d80051d-4dd9-44d5-9be4-64a500f8c435\") " pod="openshift-multus/multus-rrqzz" Jul 06 16:24:11.949193 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:11.945580 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/3d80051d-4dd9-44d5-9be4-64a500f8c435-host-run-k8s-cni-cncf-io\") pod \"multus-rrqzz\" (UID: \"3d80051d-4dd9-44d5-9be4-64a500f8c435\") " pod="openshift-multus/multus-rrqzz" Jul 06 16:24:11.949193 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:11.945605 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/3d80051d-4dd9-44d5-9be4-64a500f8c435-hostroot\") pod \"multus-rrqzz\" (UID: \"3d80051d-4dd9-44d5-9be4-64a500f8c435\") " pod="openshift-multus/multus-rrqzz" Jul 06 16:24:11.949193 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:11.945629 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/28844609-46d4-4ebf-9c6a-e714ef3bba00-etc-kubernetes\") pod \"tuned-gq74s\" (UID: \"28844609-46d4-4ebf-9c6a-e714ef3bba00\") " pod="openshift-cluster-node-tuning-operator/tuned-gq74s" Jul 06 16:24:11.949193 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:11.945657 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3d80051d-4dd9-44d5-9be4-64a500f8c435-etc-kubernetes\") pod \"multus-rrqzz\" (UID: \"3d80051d-4dd9-44d5-9be4-64a500f8c435\") " pod="openshift-multus/multus-rrqzz" Jul 06 16:24:11.949193 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:11.945686 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/6a8a4796-dbbc-43e2-a459-7acf11f9468d-ovn-node-metrics-cert\") pod \"ovnkube-node-294bx\" (UID: \"6a8a4796-dbbc-43e2-a459-7acf11f9468d\") " pod="openshift-ovn-kubernetes/ovnkube-node-294bx" Jul 06 16:24:11.949193 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:11.945714 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/28844609-46d4-4ebf-9c6a-e714ef3bba00-var-lib-kubelet\") pod \"tuned-gq74s\" (UID: \"28844609-46d4-4ebf-9c6a-e714ef3bba00\") " pod="openshift-cluster-node-tuning-operator/tuned-gq74s" Jul 06 16:24:11.949193 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:11.945743 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/a40a609b-06a9-48c2-a98a-dc948e54049c-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-55rq5\" (UID: \"a40a609b-06a9-48c2-a98a-dc948e54049c\") " pod="openshift-multus/multus-additional-cni-plugins-55rq5" Jul 06 16:24:11.949193 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:11.945777 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6a8a4796-dbbc-43e2-a459-7acf11f9468d-host-run-ovn-kubernetes\") pod \"ovnkube-node-294bx\" (UID: \"6a8a4796-dbbc-43e2-a459-7acf11f9468d\") " pod="openshift-ovn-kubernetes/ovnkube-node-294bx" Jul 06 16:24:11.949193 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:11.945822 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/6a8a4796-dbbc-43e2-a459-7acf11f9468d-host-cni-bin\") pod \"ovnkube-node-294bx\" (UID: \"6a8a4796-dbbc-43e2-a459-7acf11f9468d\") " pod="openshift-ovn-kubernetes/ovnkube-node-294bx" Jul 06 16:24:11.949193 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:11.945852 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6a8a4796-dbbc-43e2-a459-7acf11f9468d-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-294bx\" (UID: \"6a8a4796-dbbc-43e2-a459-7acf11f9468d\") " pod="openshift-ovn-kubernetes/ovnkube-node-294bx" Jul 06 16:24:11.949193 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:11.945883 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cjj6n\" (UniqueName: \"kubernetes.io/projected/e98b6fdb-3725-4c37-8a97-6a59fbc25f76-kube-api-access-cjj6n\") pod \"node-ca-27rkm\" (UID: \"e98b6fdb-3725-4c37-8a97-6a59fbc25f76\") " pod="openshift-image-registry/node-ca-27rkm" Jul 06 16:24:11.949193 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:11.945895 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/28844609-46d4-4ebf-9c6a-e714ef3bba00-sys\") pod \"tuned-gq74s\" (UID: \"28844609-46d4-4ebf-9c6a-e714ef3bba00\") " pod="openshift-cluster-node-tuning-operator/tuned-gq74s" Jul 06 16:24:11.949193 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:11.945910 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/656a2b09-7d06-48e6-9ddf-c261858ab728-kubelet-dir\") pod \"aws-ebs-csi-driver-node-8mtdq\" (UID: \"656a2b09-7d06-48e6-9ddf-c261858ab728\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8mtdq" Jul 06 16:24:11.949895 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:11.945947 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/6a8a4796-dbbc-43e2-a459-7acf11f9468d-run-systemd\") pod \"ovnkube-node-294bx\" (UID: \"6a8a4796-dbbc-43e2-a459-7acf11f9468d\") " pod="openshift-ovn-kubernetes/ovnkube-node-294bx" Jul 06 16:24:11.949895 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:11.945979 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6a8a4796-dbbc-43e2-a459-7acf11f9468d-run-openvswitch\") pod \"ovnkube-node-294bx\" (UID: \"6a8a4796-dbbc-43e2-a459-7acf11f9468d\") " pod="openshift-ovn-kubernetes/ovnkube-node-294bx" Jul 06 16:24:11.949895 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:11.946238 2584 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 06 16:24:11.949895 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:11.946651 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/6d4323bb-289e-490c-938f-d4501d0fd81b-iptables-alerter-script\") pod \"iptables-alerter-smm2d\" (UID: \"6d4323bb-289e-490c-938f-d4501d0fd81b\") " pod="openshift-network-operator/iptables-alerter-smm2d" Jul 06 16:24:11.949895 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:11.946709 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-59ftq\" (UniqueName: \"kubernetes.io/projected/6a8a4796-dbbc-43e2-a459-7acf11f9468d-kube-api-access-59ftq\") pod \"ovnkube-node-294bx\" (UID: \"6a8a4796-dbbc-43e2-a459-7acf11f9468d\") " pod="openshift-ovn-kubernetes/ovnkube-node-294bx" Jul 06 16:24:11.949895 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:11.946745 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/28844609-46d4-4ebf-9c6a-e714ef3bba00-run\") pod \"tuned-gq74s\" (UID: \"28844609-46d4-4ebf-9c6a-e714ef3bba00\") " pod="openshift-cluster-node-tuning-operator/tuned-gq74s" Jul 06 16:24:11.949895 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:11.946747 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/4c862983-24dd-471d-928b-aa352b202fd9-hosts-file\") pod \"node-resolver-wg98x\" (UID: \"4c862983-24dd-471d-928b-aa352b202fd9\") " pod="openshift-dns/node-resolver-wg98x" Jul 06 16:24:11.949895 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:11.946792 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/3d80051d-4dd9-44d5-9be4-64a500f8c435-multus-conf-dir\") pod \"multus-rrqzz\" (UID: \"3d80051d-4dd9-44d5-9be4-64a500f8c435\") " pod="openshift-multus/multus-rrqzz" Jul 06 16:24:11.949895 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:11.946816 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/28844609-46d4-4ebf-9c6a-e714ef3bba00-etc-sysconfig\") pod \"tuned-gq74s\" (UID: \"28844609-46d4-4ebf-9c6a-e714ef3bba00\") " pod="openshift-cluster-node-tuning-operator/tuned-gq74s" Jul 06 16:24:11.949895 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:11.946848 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/656a2b09-7d06-48e6-9ddf-c261858ab728-sys-fs\") pod \"aws-ebs-csi-driver-node-8mtdq\" (UID: \"656a2b09-7d06-48e6-9ddf-c261858ab728\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8mtdq" Jul 06 16:24:11.949895 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:11.946853 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/6a8a4796-dbbc-43e2-a459-7acf11f9468d-host-kubelet\") pod \"ovnkube-node-294bx\" (UID: \"6a8a4796-dbbc-43e2-a459-7acf11f9468d\") " pod="openshift-ovn-kubernetes/ovnkube-node-294bx" Jul 06 16:24:11.949895 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:11.946876 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/3d80051d-4dd9-44d5-9be4-64a500f8c435-host-var-lib-cni-multus\") pod \"multus-rrqzz\" (UID: \"3d80051d-4dd9-44d5-9be4-64a500f8c435\") " pod="openshift-multus/multus-rrqzz" Jul 06 16:24:11.949895 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:11.946889 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6a8a4796-dbbc-43e2-a459-7acf11f9468d-etc-openvswitch\") pod \"ovnkube-node-294bx\" (UID: \"6a8a4796-dbbc-43e2-a459-7acf11f9468d\") " pod="openshift-ovn-kubernetes/ovnkube-node-294bx" Jul 06 16:24:11.949895 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:11.946893 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/28844609-46d4-4ebf-9c6a-e714ef3bba00-etc-modprobe-d\") pod \"tuned-gq74s\" (UID: \"28844609-46d4-4ebf-9c6a-e714ef3bba00\") " pod="openshift-cluster-node-tuning-operator/tuned-gq74s" Jul 06 16:24:11.949895 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:11.946904 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/6a8a4796-dbbc-43e2-a459-7acf11f9468d-log-socket\") pod \"ovnkube-node-294bx\" (UID: \"6a8a4796-dbbc-43e2-a459-7acf11f9468d\") " pod="openshift-ovn-kubernetes/ovnkube-node-294bx" Jul 06 16:24:11.950591 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:11.946935 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/6d4323bb-289e-490c-938f-d4501d0fd81b-host-slash\") pod \"iptables-alerter-smm2d\" (UID: \"6d4323bb-289e-490c-938f-d4501d0fd81b\") " pod="openshift-network-operator/iptables-alerter-smm2d" Jul 06 16:24:11.950591 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:11.946940 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/3d80051d-4dd9-44d5-9be4-64a500f8c435-cnibin\") pod \"multus-rrqzz\" (UID: \"3d80051d-4dd9-44d5-9be4-64a500f8c435\") " pod="openshift-multus/multus-rrqzz" Jul 06 16:24:11.950591 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:11.946978 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/28844609-46d4-4ebf-9c6a-e714ef3bba00-lib-modules\") pod \"tuned-gq74s\" (UID: \"28844609-46d4-4ebf-9c6a-e714ef3bba00\") " pod="openshift-cluster-node-tuning-operator/tuned-gq74s" Jul 06 16:24:11.950591 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:11.946995 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/28844609-46d4-4ebf-9c6a-e714ef3bba00-etc-sysctl-d\") pod \"tuned-gq74s\" (UID: \"28844609-46d4-4ebf-9c6a-e714ef3bba00\") " pod="openshift-cluster-node-tuning-operator/tuned-gq74s" Jul 06 16:24:11.950591 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:11.947007 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/28844609-46d4-4ebf-9c6a-e714ef3bba00-host\") pod \"tuned-gq74s\" (UID: \"28844609-46d4-4ebf-9c6a-e714ef3bba00\") " pod="openshift-cluster-node-tuning-operator/tuned-gq74s" Jul 06 16:24:11.950591 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:11.947037 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/e98b6fdb-3725-4c37-8a97-6a59fbc25f76-serviceca\") pod \"node-ca-27rkm\" (UID: \"e98b6fdb-3725-4c37-8a97-6a59fbc25f76\") " pod="openshift-image-registry/node-ca-27rkm" Jul 06 16:24:11.950591 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:11.947068 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/656a2b09-7d06-48e6-9ddf-c261858ab728-registration-dir\") pod \"aws-ebs-csi-driver-node-8mtdq\" (UID: \"656a2b09-7d06-48e6-9ddf-c261858ab728\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8mtdq" Jul 06 16:24:11.950591 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:11.947102 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3d80051d-4dd9-44d5-9be4-64a500f8c435-system-cni-dir\") pod \"multus-rrqzz\" (UID: \"3d80051d-4dd9-44d5-9be4-64a500f8c435\") " pod="openshift-multus/multus-rrqzz" Jul 06 16:24:11.950591 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:11.947150 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3d80051d-4dd9-44d5-9be4-64a500f8c435-multus-cni-dir\") pod \"multus-rrqzz\" (UID: \"3d80051d-4dd9-44d5-9be4-64a500f8c435\") " pod="openshift-multus/multus-rrqzz" Jul 06 16:24:11.950591 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:11.947182 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/6a8a4796-dbbc-43e2-a459-7acf11f9468d-ovnkube-config\") pod \"ovnkube-node-294bx\" (UID: \"6a8a4796-dbbc-43e2-a459-7acf11f9468d\") " pod="openshift-ovn-kubernetes/ovnkube-node-294bx" Jul 06 16:24:11.950591 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:11.947210 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/6a8a4796-dbbc-43e2-a459-7acf11f9468d-env-overrides\") pod \"ovnkube-node-294bx\" (UID: \"6a8a4796-dbbc-43e2-a459-7acf11f9468d\") " pod="openshift-ovn-kubernetes/ovnkube-node-294bx" Jul 06 16:24:11.950591 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:11.947273 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/3d80051d-4dd9-44d5-9be4-64a500f8c435-host-run-k8s-cni-cncf-io\") pod \"multus-rrqzz\" (UID: \"3d80051d-4dd9-44d5-9be4-64a500f8c435\") " pod="openshift-multus/multus-rrqzz" Jul 06 16:24:11.950591 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:11.947269 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/3d80051d-4dd9-44d5-9be4-64a500f8c435-host-run-netns\") pod \"multus-rrqzz\" (UID: \"3d80051d-4dd9-44d5-9be4-64a500f8c435\") " pod="openshift-multus/multus-rrqzz" Jul 06 16:24:11.950591 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:11.947305 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/6a8a4796-dbbc-43e2-a459-7acf11f9468d-ovnkube-script-lib\") pod \"ovnkube-node-294bx\" (UID: \"6a8a4796-dbbc-43e2-a459-7acf11f9468d\") " pod="openshift-ovn-kubernetes/ovnkube-node-294bx" Jul 06 16:24:11.950591 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:11.947040 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/28844609-46d4-4ebf-9c6a-e714ef3bba00-etc-systemd\") pod \"tuned-gq74s\" (UID: \"28844609-46d4-4ebf-9c6a-e714ef3bba00\") " pod="openshift-cluster-node-tuning-operator/tuned-gq74s" Jul 06 16:24:11.951368 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:11.947624 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/3d80051d-4dd9-44d5-9be4-64a500f8c435-cni-binary-copy\") pod \"multus-rrqzz\" (UID: \"3d80051d-4dd9-44d5-9be4-64a500f8c435\") " pod="openshift-multus/multus-rrqzz" Jul 06 16:24:11.951368 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:11.947688 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/3d80051d-4dd9-44d5-9be4-64a500f8c435-hostroot\") pod \"multus-rrqzz\" (UID: \"3d80051d-4dd9-44d5-9be4-64a500f8c435\") " pod="openshift-multus/multus-rrqzz" Jul 06 16:24:11.951368 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:11.947716 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3d80051d-4dd9-44d5-9be4-64a500f8c435-etc-kubernetes\") pod \"multus-rrqzz\" (UID: \"3d80051d-4dd9-44d5-9be4-64a500f8c435\") " pod="openshift-multus/multus-rrqzz" Jul 06 16:24:11.951368 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:11.947847 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/6a8a4796-dbbc-43e2-a459-7acf11f9468d-env-overrides\") pod \"ovnkube-node-294bx\" (UID: \"6a8a4796-dbbc-43e2-a459-7acf11f9468d\") " pod="openshift-ovn-kubernetes/ovnkube-node-294bx" Jul 06 16:24:11.951368 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:11.947896 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/6a8a4796-dbbc-43e2-a459-7acf11f9468d-ovnkube-config\") pod \"ovnkube-node-294bx\" (UID: \"6a8a4796-dbbc-43e2-a459-7acf11f9468d\") " pod="openshift-ovn-kubernetes/ovnkube-node-294bx" Jul 06 16:24:11.951368 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:11.947978 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/28844609-46d4-4ebf-9c6a-e714ef3bba00-lib-modules\") pod \"tuned-gq74s\" (UID: \"28844609-46d4-4ebf-9c6a-e714ef3bba00\") " pod="openshift-cluster-node-tuning-operator/tuned-gq74s" Jul 06 16:24:11.951368 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:11.948025 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/28844609-46d4-4ebf-9c6a-e714ef3bba00-host\") pod \"tuned-gq74s\" (UID: \"28844609-46d4-4ebf-9c6a-e714ef3bba00\") " pod="openshift-cluster-node-tuning-operator/tuned-gq74s" Jul 06 16:24:11.951368 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:11.948066 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a40a609b-06a9-48c2-a98a-dc948e54049c-cnibin\") pod \"multus-additional-cni-plugins-55rq5\" (UID: \"a40a609b-06a9-48c2-a98a-dc948e54049c\") " pod="openshift-multus/multus-additional-cni-plugins-55rq5" Jul 06 16:24:11.951368 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:11.948099 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/3d80051d-4dd9-44d5-9be4-64a500f8c435-host-var-lib-cni-bin\") pod \"multus-rrqzz\" (UID: \"3d80051d-4dd9-44d5-9be4-64a500f8c435\") " pod="openshift-multus/multus-rrqzz" Jul 06 16:24:11.951368 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:11.948130 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/6a8a4796-dbbc-43e2-a459-7acf11f9468d-host-run-netns\") pod \"ovnkube-node-294bx\" (UID: \"6a8a4796-dbbc-43e2-a459-7acf11f9468d\") " pod="openshift-ovn-kubernetes/ovnkube-node-294bx" Jul 06 16:24:11.951368 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:11.948158 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6a8a4796-dbbc-43e2-a459-7acf11f9468d-var-lib-openvswitch\") pod \"ovnkube-node-294bx\" (UID: \"6a8a4796-dbbc-43e2-a459-7acf11f9468d\") " pod="openshift-ovn-kubernetes/ovnkube-node-294bx" Jul 06 16:24:11.951368 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:11.948192 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/6a8a4796-dbbc-43e2-a459-7acf11f9468d-host-cni-netd\") pod \"ovnkube-node-294bx\" (UID: \"6a8a4796-dbbc-43e2-a459-7acf11f9468d\") " pod="openshift-ovn-kubernetes/ovnkube-node-294bx" Jul 06 16:24:11.951368 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:11.948276 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/6a8a4796-dbbc-43e2-a459-7acf11f9468d-host-cni-netd\") pod \"ovnkube-node-294bx\" (UID: \"6a8a4796-dbbc-43e2-a459-7acf11f9468d\") " pod="openshift-ovn-kubernetes/ovnkube-node-294bx" Jul 06 16:24:11.951368 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:11.948327 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3d80051d-4dd9-44d5-9be4-64a500f8c435-system-cni-dir\") pod \"multus-rrqzz\" (UID: \"3d80051d-4dd9-44d5-9be4-64a500f8c435\") " pod="openshift-multus/multus-rrqzz" Jul 06 16:24:11.951368 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:11.948386 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/6a8a4796-dbbc-43e2-a459-7acf11f9468d-ovnkube-script-lib\") pod \"ovnkube-node-294bx\" (UID: \"6a8a4796-dbbc-43e2-a459-7acf11f9468d\") " pod="openshift-ovn-kubernetes/ovnkube-node-294bx" Jul 06 16:24:11.952126 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:11.948438 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/6d4323bb-289e-490c-938f-d4501d0fd81b-host-slash\") pod \"iptables-alerter-smm2d\" (UID: \"6d4323bb-289e-490c-938f-d4501d0fd81b\") " pod="openshift-network-operator/iptables-alerter-smm2d" Jul 06 16:24:11.952126 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:11.948441 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3d80051d-4dd9-44d5-9be4-64a500f8c435-multus-cni-dir\") pod \"multus-rrqzz\" (UID: \"3d80051d-4dd9-44d5-9be4-64a500f8c435\") " pod="openshift-multus/multus-rrqzz" Jul 06 16:24:11.952126 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:11.948474 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/6a8a4796-dbbc-43e2-a459-7acf11f9468d-run-systemd\") pod \"ovnkube-node-294bx\" (UID: \"6a8a4796-dbbc-43e2-a459-7acf11f9468d\") " pod="openshift-ovn-kubernetes/ovnkube-node-294bx" Jul 06 16:24:11.952126 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:11.948490 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/3d80051d-4dd9-44d5-9be4-64a500f8c435-host-var-lib-cni-bin\") pod \"multus-rrqzz\" (UID: \"3d80051d-4dd9-44d5-9be4-64a500f8c435\") " pod="openshift-multus/multus-rrqzz" Jul 06 16:24:11.952126 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:11.946751 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6a8a4796-dbbc-43e2-a459-7acf11f9468d-run-openvswitch\") pod \"ovnkube-node-294bx\" (UID: \"6a8a4796-dbbc-43e2-a459-7acf11f9468d\") " pod="openshift-ovn-kubernetes/ovnkube-node-294bx" Jul 06 16:24:11.952126 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:11.948537 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/6a8a4796-dbbc-43e2-a459-7acf11f9468d-host-run-netns\") pod \"ovnkube-node-294bx\" (UID: \"6a8a4796-dbbc-43e2-a459-7acf11f9468d\") " pod="openshift-ovn-kubernetes/ovnkube-node-294bx" Jul 06 16:24:11.952126 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:11.948558 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/28844609-46d4-4ebf-9c6a-e714ef3bba00-etc-sysconfig\") pod \"tuned-gq74s\" (UID: \"28844609-46d4-4ebf-9c6a-e714ef3bba00\") " pod="openshift-cluster-node-tuning-operator/tuned-gq74s" Jul 06 16:24:11.952126 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:11.948574 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6a8a4796-dbbc-43e2-a459-7acf11f9468d-var-lib-openvswitch\") pod \"ovnkube-node-294bx\" (UID: \"6a8a4796-dbbc-43e2-a459-7acf11f9468d\") " pod="openshift-ovn-kubernetes/ovnkube-node-294bx" Jul 06 16:24:11.952126 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:11.948615 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/6a8a4796-dbbc-43e2-a459-7acf11f9468d-run-ovn\") pod \"ovnkube-node-294bx\" (UID: \"6a8a4796-dbbc-43e2-a459-7acf11f9468d\") " pod="openshift-ovn-kubernetes/ovnkube-node-294bx" Jul 06 16:24:11.952126 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:11.948620 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/3d80051d-4dd9-44d5-9be4-64a500f8c435-multus-socket-dir-parent\") pod \"multus-rrqzz\" (UID: \"3d80051d-4dd9-44d5-9be4-64a500f8c435\") " pod="openshift-multus/multus-rrqzz" Jul 06 16:24:11.952126 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:11.948693 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/4c862983-24dd-471d-928b-aa352b202fd9-hosts-file\") pod \"node-resolver-wg98x\" (UID: \"4c862983-24dd-471d-928b-aa352b202fd9\") " pod="openshift-dns/node-resolver-wg98x" Jul 06 16:24:11.952126 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:11.948851 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/3d80051d-4dd9-44d5-9be4-64a500f8c435-host-var-lib-cni-multus\") pod \"multus-rrqzz\" (UID: \"3d80051d-4dd9-44d5-9be4-64a500f8c435\") " pod="openshift-multus/multus-rrqzz" Jul 06 16:24:11.952126 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:11.948879 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/6a8a4796-dbbc-43e2-a459-7acf11f9468d-log-socket\") pod \"ovnkube-node-294bx\" (UID: \"6a8a4796-dbbc-43e2-a459-7acf11f9468d\") " pod="openshift-ovn-kubernetes/ovnkube-node-294bx" Jul 06 16:24:11.952126 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:11.948932 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6a8a4796-dbbc-43e2-a459-7acf11f9468d-host-run-ovn-kubernetes\") pod \"ovnkube-node-294bx\" (UID: \"6a8a4796-dbbc-43e2-a459-7acf11f9468d\") " pod="openshift-ovn-kubernetes/ovnkube-node-294bx" Jul 06 16:24:11.952126 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:11.948969 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/28844609-46d4-4ebf-9c6a-e714ef3bba00-var-lib-kubelet\") pod \"tuned-gq74s\" (UID: \"28844609-46d4-4ebf-9c6a-e714ef3bba00\") " pod="openshift-cluster-node-tuning-operator/tuned-gq74s" Jul 06 16:24:11.952915 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:11.949006 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/6a8a4796-dbbc-43e2-a459-7acf11f9468d-host-cni-bin\") pod \"ovnkube-node-294bx\" (UID: \"6a8a4796-dbbc-43e2-a459-7acf11f9468d\") " pod="openshift-ovn-kubernetes/ovnkube-node-294bx" Jul 06 16:24:11.952915 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:11.949040 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6a8a4796-dbbc-43e2-a459-7acf11f9468d-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-294bx\" (UID: \"6a8a4796-dbbc-43e2-a459-7acf11f9468d\") " pod="openshift-ovn-kubernetes/ovnkube-node-294bx" Jul 06 16:24:11.952915 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:11.949154 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/3d80051d-4dd9-44d5-9be4-64a500f8c435-multus-daemon-config\") pod \"multus-rrqzz\" (UID: \"3d80051d-4dd9-44d5-9be4-64a500f8c435\") " pod="openshift-multus/multus-rrqzz" Jul 06 16:24:11.952915 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:11.952224 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/6a8a4796-dbbc-43e2-a459-7acf11f9468d-ovn-node-metrics-cert\") pod \"ovnkube-node-294bx\" (UID: \"6a8a4796-dbbc-43e2-a459-7acf11f9468d\") " pod="openshift-ovn-kubernetes/ovnkube-node-294bx" Jul 06 16:24:11.952915 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:11.952389 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/28844609-46d4-4ebf-9c6a-e714ef3bba00-etc-tuned\") pod \"tuned-gq74s\" (UID: \"28844609-46d4-4ebf-9c6a-e714ef3bba00\") " pod="openshift-cluster-node-tuning-operator/tuned-gq74s" Jul 06 16:24:11.953935 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:11.953911 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/28844609-46d4-4ebf-9c6a-e714ef3bba00-tmp\") pod \"tuned-gq74s\" (UID: \"28844609-46d4-4ebf-9c6a-e714ef3bba00\") " pod="openshift-cluster-node-tuning-operator/tuned-gq74s" Jul 06 16:24:11.960996 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:11.960968 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-psnvc\" (UniqueName: \"kubernetes.io/projected/28844609-46d4-4ebf-9c6a-e714ef3bba00-kube-api-access-psnvc\") pod \"tuned-gq74s\" (UID: \"28844609-46d4-4ebf-9c6a-e714ef3bba00\") " pod="openshift-cluster-node-tuning-operator/tuned-gq74s" Jul 06 16:24:11.960996 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:11.960977 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-59ftq\" (UniqueName: \"kubernetes.io/projected/6a8a4796-dbbc-43e2-a459-7acf11f9468d-kube-api-access-59ftq\") pod \"ovnkube-node-294bx\" (UID: \"6a8a4796-dbbc-43e2-a459-7acf11f9468d\") " pod="openshift-ovn-kubernetes/ovnkube-node-294bx" Jul 06 16:24:11.964577 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:11.964516 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-glzns\" (UniqueName: \"kubernetes.io/projected/6d4323bb-289e-490c-938f-d4501d0fd81b-kube-api-access-glzns\") pod \"iptables-alerter-smm2d\" (UID: \"6d4323bb-289e-490c-938f-d4501d0fd81b\") " pod="openshift-network-operator/iptables-alerter-smm2d" Jul 06 16:24:11.968531 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:11.968500 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cbhjk\" (UniqueName: \"kubernetes.io/projected/4c862983-24dd-471d-928b-aa352b202fd9-kube-api-access-cbhjk\") pod \"node-resolver-wg98x\" (UID: \"4c862983-24dd-471d-928b-aa352b202fd9\") " pod="openshift-dns/node-resolver-wg98x" Jul 06 16:24:11.968759 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:11.968732 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dtsdb\" (UniqueName: \"kubernetes.io/projected/3d80051d-4dd9-44d5-9be4-64a500f8c435-kube-api-access-dtsdb\") pod \"multus-rrqzz\" (UID: \"3d80051d-4dd9-44d5-9be4-64a500f8c435\") " pod="openshift-multus/multus-rrqzz" Jul 06 16:24:12.049303 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:12.049225 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/656a2b09-7d06-48e6-9ddf-c261858ab728-sys-fs\") pod \"aws-ebs-csi-driver-node-8mtdq\" (UID: \"656a2b09-7d06-48e6-9ddf-c261858ab728\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8mtdq" Jul 06 16:24:12.049303 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:12.049263 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/e98b6fdb-3725-4c37-8a97-6a59fbc25f76-serviceca\") pod \"node-ca-27rkm\" (UID: \"e98b6fdb-3725-4c37-8a97-6a59fbc25f76\") " pod="openshift-image-registry/node-ca-27rkm" Jul 06 16:24:12.049303 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:12.049282 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/656a2b09-7d06-48e6-9ddf-c261858ab728-registration-dir\") pod \"aws-ebs-csi-driver-node-8mtdq\" (UID: \"656a2b09-7d06-48e6-9ddf-c261858ab728\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8mtdq" Jul 06 16:24:12.049530 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:12.049308 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a40a609b-06a9-48c2-a98a-dc948e54049c-cnibin\") pod \"multus-additional-cni-plugins-55rq5\" (UID: \"a40a609b-06a9-48c2-a98a-dc948e54049c\") " pod="openshift-multus/multus-additional-cni-plugins-55rq5" Jul 06 16:24:12.049530 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:12.049347 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a40a609b-06a9-48c2-a98a-dc948e54049c-tuning-conf-dir\") pod \"multus-additional-cni-plugins-55rq5\" (UID: \"a40a609b-06a9-48c2-a98a-dc948e54049c\") " pod="openshift-multus/multus-additional-cni-plugins-55rq5" Jul 06 16:24:12.049530 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:12.049362 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/656a2b09-7d06-48e6-9ddf-c261858ab728-sys-fs\") pod \"aws-ebs-csi-driver-node-8mtdq\" (UID: \"656a2b09-7d06-48e6-9ddf-c261858ab728\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8mtdq" Jul 06 16:24:12.049530 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:12.049382 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/656a2b09-7d06-48e6-9ddf-c261858ab728-registration-dir\") pod \"aws-ebs-csi-driver-node-8mtdq\" (UID: \"656a2b09-7d06-48e6-9ddf-c261858ab728\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8mtdq" Jul 06 16:24:12.049530 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:12.049365 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vqnr6\" (UniqueName: \"kubernetes.io/projected/3106072a-c698-44ac-9b92-8a7fd1e08d8a-kube-api-access-vqnr6\") pod \"network-metrics-daemon-2zdgn\" (UID: \"3106072a-c698-44ac-9b92-8a7fd1e08d8a\") " pod="openshift-multus/network-metrics-daemon-2zdgn" Jul 06 16:24:12.049530 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:12.049425 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a40a609b-06a9-48c2-a98a-dc948e54049c-cnibin\") pod \"multus-additional-cni-plugins-55rq5\" (UID: \"a40a609b-06a9-48c2-a98a-dc948e54049c\") " pod="openshift-multus/multus-additional-cni-plugins-55rq5" Jul 06 16:24:12.049530 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:12.049445 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a40a609b-06a9-48c2-a98a-dc948e54049c-system-cni-dir\") pod \"multus-additional-cni-plugins-55rq5\" (UID: \"a40a609b-06a9-48c2-a98a-dc948e54049c\") " pod="openshift-multus/multus-additional-cni-plugins-55rq5" Jul 06 16:24:12.049530 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:12.049470 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-shdb5\" (UniqueName: \"kubernetes.io/projected/656a2b09-7d06-48e6-9ddf-c261858ab728-kube-api-access-shdb5\") pod \"aws-ebs-csi-driver-node-8mtdq\" (UID: \"656a2b09-7d06-48e6-9ddf-c261858ab728\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8mtdq" Jul 06 16:24:12.049530 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:12.049493 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/a40a609b-06a9-48c2-a98a-dc948e54049c-os-release\") pod \"multus-additional-cni-plugins-55rq5\" (UID: \"a40a609b-06a9-48c2-a98a-dc948e54049c\") " pod="openshift-multus/multus-additional-cni-plugins-55rq5" Jul 06 16:24:12.049530 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:12.049510 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/d695ec8b-3624-40aa-afe2-841d320d0a66-dbus\") pod \"global-pull-secret-syncer-wvcq5\" (UID: \"d695ec8b-3624-40aa-afe2-841d320d0a66\") " pod="kube-system/global-pull-secret-syncer-wvcq5" Jul 06 16:24:12.049530 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:12.049534 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d695ec8b-3624-40aa-afe2-841d320d0a66-original-pull-secret\") pod \"global-pull-secret-syncer-wvcq5\" (UID: \"d695ec8b-3624-40aa-afe2-841d320d0a66\") " pod="kube-system/global-pull-secret-syncer-wvcq5" Jul 06 16:24:12.050068 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:12.049541 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a40a609b-06a9-48c2-a98a-dc948e54049c-tuning-conf-dir\") pod \"multus-additional-cni-plugins-55rq5\" (UID: \"a40a609b-06a9-48c2-a98a-dc948e54049c\") " pod="openshift-multus/multus-additional-cni-plugins-55rq5" Jul 06 16:24:12.050068 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:12.049565 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/9bf128db-4ed4-483d-a886-cb334e6fa4ef-konnectivity-ca\") pod \"konnectivity-agent-k4gbb\" (UID: \"9bf128db-4ed4-483d-a886-cb334e6fa4ef\") " pod="kube-system/konnectivity-agent-k4gbb" Jul 06 16:24:12.050068 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:12.049619 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/a40a609b-06a9-48c2-a98a-dc948e54049c-os-release\") pod \"multus-additional-cni-plugins-55rq5\" (UID: \"a40a609b-06a9-48c2-a98a-dc948e54049c\") " pod="openshift-multus/multus-additional-cni-plugins-55rq5" Jul 06 16:24:12.050068 ip-10-0-128-22 kubenswrapper[2584]: E0706 16:24:12.049641 2584 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 06 16:24:12.050068 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:12.049668 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/656a2b09-7d06-48e6-9ddf-c261858ab728-socket-dir\") pod \"aws-ebs-csi-driver-node-8mtdq\" (UID: \"656a2b09-7d06-48e6-9ddf-c261858ab728\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8mtdq" Jul 06 16:24:12.050068 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:12.049743 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/a40a609b-06a9-48c2-a98a-dc948e54049c-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-55rq5\" (UID: \"a40a609b-06a9-48c2-a98a-dc948e54049c\") " pod="openshift-multus/multus-additional-cni-plugins-55rq5" Jul 06 16:24:12.050068 ip-10-0-128-22 kubenswrapper[2584]: E0706 16:24:12.049759 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d695ec8b-3624-40aa-afe2-841d320d0a66-original-pull-secret podName:d695ec8b-3624-40aa-afe2-841d320d0a66 nodeName:}" failed. No retries permitted until 2026-07-06 16:24:12.549712661 +0000 UTC m=+3.163852784 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d695ec8b-3624-40aa-afe2-841d320d0a66-original-pull-secret") pod "global-pull-secret-syncer-wvcq5" (UID: "d695ec8b-3624-40aa-afe2-841d320d0a66") : object "kube-system"/"original-pull-secret" not registered Jul 06 16:24:12.050068 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:12.049797 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a40a609b-06a9-48c2-a98a-dc948e54049c-system-cni-dir\") pod \"multus-additional-cni-plugins-55rq5\" (UID: \"a40a609b-06a9-48c2-a98a-dc948e54049c\") " pod="openshift-multus/multus-additional-cni-plugins-55rq5" Jul 06 16:24:12.050068 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:12.049813 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/656a2b09-7d06-48e6-9ddf-c261858ab728-socket-dir\") pod \"aws-ebs-csi-driver-node-8mtdq\" (UID: \"656a2b09-7d06-48e6-9ddf-c261858ab728\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8mtdq" Jul 06 16:24:12.050068 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:12.049812 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bv7nz\" (UniqueName: \"kubernetes.io/projected/a40a609b-06a9-48c2-a98a-dc948e54049c-kube-api-access-bv7nz\") pod \"multus-additional-cni-plugins-55rq5\" (UID: \"a40a609b-06a9-48c2-a98a-dc948e54049c\") " pod="openshift-multus/multus-additional-cni-plugins-55rq5" Jul 06 16:24:12.050068 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:12.049849 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/d695ec8b-3624-40aa-afe2-841d320d0a66-dbus\") pod \"global-pull-secret-syncer-wvcq5\" (UID: \"d695ec8b-3624-40aa-afe2-841d320d0a66\") " pod="kube-system/global-pull-secret-syncer-wvcq5" Jul 06 16:24:12.050068 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:12.049853 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3106072a-c698-44ac-9b92-8a7fd1e08d8a-metrics-certs\") pod \"network-metrics-daemon-2zdgn\" (UID: \"3106072a-c698-44ac-9b92-8a7fd1e08d8a\") " pod="openshift-multus/network-metrics-daemon-2zdgn" Jul 06 16:24:12.050068 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:12.049880 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/656a2b09-7d06-48e6-9ddf-c261858ab728-device-dir\") pod \"aws-ebs-csi-driver-node-8mtdq\" (UID: \"656a2b09-7d06-48e6-9ddf-c261858ab728\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8mtdq" Jul 06 16:24:12.050068 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:12.049910 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jz4gp\" (UniqueName: \"kubernetes.io/projected/6c5de78f-76c0-4c6a-9c41-787f8452e5b4-kube-api-access-jz4gp\") pod \"network-check-target-v8kjk\" (UID: \"6c5de78f-76c0-4c6a-9c41-787f8452e5b4\") " pod="openshift-network-diagnostics/network-check-target-v8kjk" Jul 06 16:24:12.050673 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:12.049944 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/9bf128db-4ed4-483d-a886-cb334e6fa4ef-agent-certs\") pod \"konnectivity-agent-k4gbb\" (UID: \"9bf128db-4ed4-483d-a886-cb334e6fa4ef\") " pod="kube-system/konnectivity-agent-k4gbb" Jul 06 16:24:12.050673 ip-10-0-128-22 kubenswrapper[2584]: E0706 16:24:12.049964 2584 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 16:24:12.050673 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:12.049974 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/d695ec8b-3624-40aa-afe2-841d320d0a66-kubelet-config\") pod \"global-pull-secret-syncer-wvcq5\" (UID: \"d695ec8b-3624-40aa-afe2-841d320d0a66\") " pod="kube-system/global-pull-secret-syncer-wvcq5" Jul 06 16:24:12.050673 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:12.050015 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/656a2b09-7d06-48e6-9ddf-c261858ab728-device-dir\") pod \"aws-ebs-csi-driver-node-8mtdq\" (UID: \"656a2b09-7d06-48e6-9ddf-c261858ab728\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8mtdq" Jul 06 16:24:12.050673 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:12.050027 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a40a609b-06a9-48c2-a98a-dc948e54049c-cni-binary-copy\") pod \"multus-additional-cni-plugins-55rq5\" (UID: \"a40a609b-06a9-48c2-a98a-dc948e54049c\") " pod="openshift-multus/multus-additional-cni-plugins-55rq5" Jul 06 16:24:12.050673 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:12.050061 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/e98b6fdb-3725-4c37-8a97-6a59fbc25f76-host\") pod \"node-ca-27rkm\" (UID: \"e98b6fdb-3725-4c37-8a97-6a59fbc25f76\") " pod="openshift-image-registry/node-ca-27rkm" Jul 06 16:24:12.050673 ip-10-0-128-22 kubenswrapper[2584]: E0706 16:24:12.050115 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3106072a-c698-44ac-9b92-8a7fd1e08d8a-metrics-certs podName:3106072a-c698-44ac-9b92-8a7fd1e08d8a nodeName:}" failed. No retries permitted until 2026-07-06 16:24:12.550073454 +0000 UTC m=+3.164213569 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3106072a-c698-44ac-9b92-8a7fd1e08d8a-metrics-certs") pod "network-metrics-daemon-2zdgn" (UID: "3106072a-c698-44ac-9b92-8a7fd1e08d8a") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 16:24:12.050673 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:12.050153 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/656a2b09-7d06-48e6-9ddf-c261858ab728-etc-selinux\") pod \"aws-ebs-csi-driver-node-8mtdq\" (UID: \"656a2b09-7d06-48e6-9ddf-c261858ab728\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8mtdq" Jul 06 16:24:12.050673 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:12.050175 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/9bf128db-4ed4-483d-a886-cb334e6fa4ef-konnectivity-ca\") pod \"konnectivity-agent-k4gbb\" (UID: \"9bf128db-4ed4-483d-a886-cb334e6fa4ef\") " pod="kube-system/konnectivity-agent-k4gbb" Jul 06 16:24:12.050673 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:12.050113 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/e98b6fdb-3725-4c37-8a97-6a59fbc25f76-host\") pod \"node-ca-27rkm\" (UID: \"e98b6fdb-3725-4c37-8a97-6a59fbc25f76\") " pod="openshift-image-registry/node-ca-27rkm" Jul 06 16:24:12.050673 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:12.050196 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/a40a609b-06a9-48c2-a98a-dc948e54049c-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-55rq5\" (UID: \"a40a609b-06a9-48c2-a98a-dc948e54049c\") " pod="openshift-multus/multus-additional-cni-plugins-55rq5" Jul 06 16:24:12.050673 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:12.050177 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/d695ec8b-3624-40aa-afe2-841d320d0a66-kubelet-config\") pod \"global-pull-secret-syncer-wvcq5\" (UID: \"d695ec8b-3624-40aa-afe2-841d320d0a66\") " pod="kube-system/global-pull-secret-syncer-wvcq5" Jul 06 16:24:12.050673 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:12.050349 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cjj6n\" (UniqueName: \"kubernetes.io/projected/e98b6fdb-3725-4c37-8a97-6a59fbc25f76-kube-api-access-cjj6n\") pod \"node-ca-27rkm\" (UID: \"e98b6fdb-3725-4c37-8a97-6a59fbc25f76\") " pod="openshift-image-registry/node-ca-27rkm" Jul 06 16:24:12.050673 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:12.050384 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/656a2b09-7d06-48e6-9ddf-c261858ab728-kubelet-dir\") pod \"aws-ebs-csi-driver-node-8mtdq\" (UID: \"656a2b09-7d06-48e6-9ddf-c261858ab728\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8mtdq" Jul 06 16:24:12.051123 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:12.050397 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/656a2b09-7d06-48e6-9ddf-c261858ab728-etc-selinux\") pod \"aws-ebs-csi-driver-node-8mtdq\" (UID: \"656a2b09-7d06-48e6-9ddf-c261858ab728\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8mtdq" Jul 06 16:24:12.051123 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:12.050434 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/a40a609b-06a9-48c2-a98a-dc948e54049c-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-55rq5\" (UID: \"a40a609b-06a9-48c2-a98a-dc948e54049c\") " pod="openshift-multus/multus-additional-cni-plugins-55rq5" Jul 06 16:24:12.051123 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:12.050541 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/e98b6fdb-3725-4c37-8a97-6a59fbc25f76-serviceca\") pod \"node-ca-27rkm\" (UID: \"e98b6fdb-3725-4c37-8a97-6a59fbc25f76\") " pod="openshift-image-registry/node-ca-27rkm" Jul 06 16:24:12.051123 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:12.050565 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/656a2b09-7d06-48e6-9ddf-c261858ab728-kubelet-dir\") pod \"aws-ebs-csi-driver-node-8mtdq\" (UID: \"656a2b09-7d06-48e6-9ddf-c261858ab728\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8mtdq" Jul 06 16:24:12.051123 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:12.050743 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a40a609b-06a9-48c2-a98a-dc948e54049c-cni-binary-copy\") pod \"multus-additional-cni-plugins-55rq5\" (UID: \"a40a609b-06a9-48c2-a98a-dc948e54049c\") " pod="openshift-multus/multus-additional-cni-plugins-55rq5" Jul 06 16:24:12.051123 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:12.050869 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/a40a609b-06a9-48c2-a98a-dc948e54049c-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-55rq5\" (UID: \"a40a609b-06a9-48c2-a98a-dc948e54049c\") " pod="openshift-multus/multus-additional-cni-plugins-55rq5" Jul 06 16:24:12.052676 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:12.052637 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/9bf128db-4ed4-483d-a886-cb334e6fa4ef-agent-certs\") pod \"konnectivity-agent-k4gbb\" (UID: \"9bf128db-4ed4-483d-a886-cb334e6fa4ef\") " pod="kube-system/konnectivity-agent-k4gbb" Jul 06 16:24:12.069323 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:12.069288 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-shdb5\" (UniqueName: \"kubernetes.io/projected/656a2b09-7d06-48e6-9ddf-c261858ab728-kube-api-access-shdb5\") pod \"aws-ebs-csi-driver-node-8mtdq\" (UID: \"656a2b09-7d06-48e6-9ddf-c261858ab728\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8mtdq" Jul 06 16:24:12.069548 ip-10-0-128-22 kubenswrapper[2584]: E0706 16:24:12.069527 2584 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 06 16:24:12.069596 ip-10-0-128-22 kubenswrapper[2584]: E0706 16:24:12.069558 2584 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 06 16:24:12.069596 ip-10-0-128-22 kubenswrapper[2584]: E0706 16:24:12.069583 2584 projected.go:196] Error preparing data for projected volume kube-api-access-jz4gp for pod openshift-network-diagnostics/network-check-target-v8kjk: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 16:24:12.069697 ip-10-0-128-22 kubenswrapper[2584]: E0706 16:24:12.069682 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6c5de78f-76c0-4c6a-9c41-787f8452e5b4-kube-api-access-jz4gp podName:6c5de78f-76c0-4c6a-9c41-787f8452e5b4 nodeName:}" failed. No retries permitted until 2026-07-06 16:24:12.569658017 +0000 UTC m=+3.183798135 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-jz4gp" (UniqueName: "kubernetes.io/projected/6c5de78f-76c0-4c6a-9c41-787f8452e5b4-kube-api-access-jz4gp") pod "network-check-target-v8kjk" (UID: "6c5de78f-76c0-4c6a-9c41-787f8452e5b4") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 16:24:12.071510 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:12.071479 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bv7nz\" (UniqueName: \"kubernetes.io/projected/a40a609b-06a9-48c2-a98a-dc948e54049c-kube-api-access-bv7nz\") pod \"multus-additional-cni-plugins-55rq5\" (UID: \"a40a609b-06a9-48c2-a98a-dc948e54049c\") " pod="openshift-multus/multus-additional-cni-plugins-55rq5" Jul 06 16:24:12.072300 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:12.072257 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cjj6n\" (UniqueName: \"kubernetes.io/projected/e98b6fdb-3725-4c37-8a97-6a59fbc25f76-kube-api-access-cjj6n\") pod \"node-ca-27rkm\" (UID: \"e98b6fdb-3725-4c37-8a97-6a59fbc25f76\") " pod="openshift-image-registry/node-ca-27rkm" Jul 06 16:24:12.077936 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:12.073654 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vqnr6\" (UniqueName: \"kubernetes.io/projected/3106072a-c698-44ac-9b92-8a7fd1e08d8a-kube-api-access-vqnr6\") pod \"network-metrics-daemon-2zdgn\" (UID: \"3106072a-c698-44ac-9b92-8a7fd1e08d8a\") " pod="openshift-multus/network-metrics-daemon-2zdgn" Jul 06 16:24:12.117583 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:12.117514 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-22.ec2.internal" event={"ID":"681ecc11fd0a8e367cd02518060abe2d","Type":"ContainerStarted","Data":"b28ad8e4d399f8078b0171cefb47cb8566817aea95f55017fedffe24ba47eb22"} Jul 06 16:24:12.119077 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:12.119044 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-22.ec2.internal" event={"ID":"a4d219c131416ebd840bc1507523f196","Type":"ContainerStarted","Data":"d84d6c434ae977dd9a488d3549e8844a1c0ec4b04bad2c5382106e076a210b6b"} Jul 06 16:24:12.128557 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:12.128476 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-rrqzz" Jul 06 16:24:12.137286 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:12.137068 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-294bx" Jul 06 16:24:12.140638 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:12.140599 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3d80051d_4dd9_44d5_9be4_64a500f8c435.slice/crio-8a5aefbd139f73988381d5b8580608473750d63ca03311eb3378115129a02de0 WatchSource:0}: Error finding container 8a5aefbd139f73988381d5b8580608473750d63ca03311eb3378115129a02de0: Status 404 returned error can't find the container with id 8a5aefbd139f73988381d5b8580608473750d63ca03311eb3378115129a02de0 Jul 06 16:24:12.151302 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:12.151255 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6a8a4796_dbbc_43e2_a459_7acf11f9468d.slice/crio-db925100e854f945db55cb328befb74fea517a0d4c814970010807d25bb8d30e WatchSource:0}: Error finding container db925100e854f945db55cb328befb74fea517a0d4c814970010807d25bb8d30e: Status 404 returned error can't find the container with id db925100e854f945db55cb328befb74fea517a0d4c814970010807d25bb8d30e Jul 06 16:24:12.167533 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:12.167505 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-wg98x" Jul 06 16:24:12.176192 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:12.176160 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-smm2d" Jul 06 16:24:12.177787 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:12.177730 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4c862983_24dd_471d_928b_aa352b202fd9.slice/crio-ef79b225096ec8e5a924c6fe9759e018da4978ab810c84d7e81a4483a1c07218 WatchSource:0}: Error finding container ef79b225096ec8e5a924c6fe9759e018da4978ab810c84d7e81a4483a1c07218: Status 404 returned error can't find the container with id ef79b225096ec8e5a924c6fe9759e018da4978ab810c84d7e81a4483a1c07218 Jul 06 16:24:12.185843 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:12.185615 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-gq74s" Jul 06 16:24:12.185995 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:12.185964 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6d4323bb_289e_490c_938f_d4501d0fd81b.slice/crio-114d36c212103029e614189ba4c39311c545fbe85497600b0891acd41af6d07b WatchSource:0}: Error finding container 114d36c212103029e614189ba4c39311c545fbe85497600b0891acd41af6d07b: Status 404 returned error can't find the container with id 114d36c212103029e614189ba4c39311c545fbe85497600b0891acd41af6d07b Jul 06 16:24:12.194280 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:12.194256 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-k4gbb" Jul 06 16:24:12.196704 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:12.196675 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod28844609_46d4_4ebf_9c6a_e714ef3bba00.slice/crio-5d2d4f5ae6d701ddca694e19fe511bb06c7d692d1ae33e3e1d37bab180b37430 WatchSource:0}: Error finding container 5d2d4f5ae6d701ddca694e19fe511bb06c7d692d1ae33e3e1d37bab180b37430: Status 404 returned error can't find the container with id 5d2d4f5ae6d701ddca694e19fe511bb06c7d692d1ae33e3e1d37bab180b37430 Jul 06 16:24:12.203353 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:12.203326 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9bf128db_4ed4_483d_a886_cb334e6fa4ef.slice/crio-c1fe5f29caa842a1e29a82b8022893a8e064046270bb745972b53ad57f2a5c8f WatchSource:0}: Error finding container c1fe5f29caa842a1e29a82b8022893a8e064046270bb745972b53ad57f2a5c8f: Status 404 returned error can't find the container with id c1fe5f29caa842a1e29a82b8022893a8e064046270bb745972b53ad57f2a5c8f Jul 06 16:24:12.207517 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:12.207494 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-55rq5" Jul 06 16:24:12.216060 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:12.216037 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-27rkm" Jul 06 16:24:12.218710 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:12.218678 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda40a609b_06a9_48c2_a98a_dc948e54049c.slice/crio-beec3eeeb493a574619e2bfef6c76215cac0156d25350d2dc30cd5551cc0b076 WatchSource:0}: Error finding container beec3eeeb493a574619e2bfef6c76215cac0156d25350d2dc30cd5551cc0b076: Status 404 returned error can't find the container with id beec3eeeb493a574619e2bfef6c76215cac0156d25350d2dc30cd5551cc0b076 Jul 06 16:24:12.222742 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:12.222691 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8mtdq" Jul 06 16:24:12.224856 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:12.224825 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode98b6fdb_3725_4c37_8a97_6a59fbc25f76.slice/crio-7ac5c72a273df66be5e8d2344ba10815599204babc73d25eedf67893a63319d8 WatchSource:0}: Error finding container 7ac5c72a273df66be5e8d2344ba10815599204babc73d25eedf67893a63319d8: Status 404 returned error can't find the container with id 7ac5c72a273df66be5e8d2344ba10815599204babc73d25eedf67893a63319d8 Jul 06 16:24:12.233141 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:12.233116 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod656a2b09_7d06_48e6_9ddf_c261858ab728.slice/crio-379c62a931c54e9a74d50e7ca42ef23a73479c724eec376dd3bf5e40ee28d57f WatchSource:0}: Error finding container 379c62a931c54e9a74d50e7ca42ef23a73479c724eec376dd3bf5e40ee28d57f: Status 404 returned error can't find the container with id 379c62a931c54e9a74d50e7ca42ef23a73479c724eec376dd3bf5e40ee28d57f Jul 06 16:24:12.554074 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:12.554031 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d695ec8b-3624-40aa-afe2-841d320d0a66-original-pull-secret\") pod \"global-pull-secret-syncer-wvcq5\" (UID: \"d695ec8b-3624-40aa-afe2-841d320d0a66\") " pod="kube-system/global-pull-secret-syncer-wvcq5" Jul 06 16:24:12.554271 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:12.554093 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3106072a-c698-44ac-9b92-8a7fd1e08d8a-metrics-certs\") pod \"network-metrics-daemon-2zdgn\" (UID: \"3106072a-c698-44ac-9b92-8a7fd1e08d8a\") " pod="openshift-multus/network-metrics-daemon-2zdgn" Jul 06 16:24:12.554271 ip-10-0-128-22 kubenswrapper[2584]: E0706 16:24:12.554215 2584 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 06 16:24:12.554403 ip-10-0-128-22 kubenswrapper[2584]: E0706 16:24:12.554293 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d695ec8b-3624-40aa-afe2-841d320d0a66-original-pull-secret podName:d695ec8b-3624-40aa-afe2-841d320d0a66 nodeName:}" failed. No retries permitted until 2026-07-06 16:24:13.554273475 +0000 UTC m=+4.168413576 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d695ec8b-3624-40aa-afe2-841d320d0a66-original-pull-secret") pod "global-pull-secret-syncer-wvcq5" (UID: "d695ec8b-3624-40aa-afe2-841d320d0a66") : object "kube-system"/"original-pull-secret" not registered Jul 06 16:24:12.554403 ip-10-0-128-22 kubenswrapper[2584]: E0706 16:24:12.554221 2584 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 16:24:12.554403 ip-10-0-128-22 kubenswrapper[2584]: E0706 16:24:12.554345 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3106072a-c698-44ac-9b92-8a7fd1e08d8a-metrics-certs podName:3106072a-c698-44ac-9b92-8a7fd1e08d8a nodeName:}" failed. No retries permitted until 2026-07-06 16:24:13.554333288 +0000 UTC m=+4.168473393 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3106072a-c698-44ac-9b92-8a7fd1e08d8a-metrics-certs") pod "network-metrics-daemon-2zdgn" (UID: "3106072a-c698-44ac-9b92-8a7fd1e08d8a") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 16:24:12.655243 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:12.655203 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jz4gp\" (UniqueName: \"kubernetes.io/projected/6c5de78f-76c0-4c6a-9c41-787f8452e5b4-kube-api-access-jz4gp\") pod \"network-check-target-v8kjk\" (UID: \"6c5de78f-76c0-4c6a-9c41-787f8452e5b4\") " pod="openshift-network-diagnostics/network-check-target-v8kjk" Jul 06 16:24:12.655439 ip-10-0-128-22 kubenswrapper[2584]: E0706 16:24:12.655376 2584 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 06 16:24:12.655439 ip-10-0-128-22 kubenswrapper[2584]: E0706 16:24:12.655404 2584 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 06 16:24:12.655439 ip-10-0-128-22 kubenswrapper[2584]: E0706 16:24:12.655417 2584 projected.go:196] Error preparing data for projected volume kube-api-access-jz4gp for pod openshift-network-diagnostics/network-check-target-v8kjk: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 16:24:12.655602 ip-10-0-128-22 kubenswrapper[2584]: E0706 16:24:12.655493 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6c5de78f-76c0-4c6a-9c41-787f8452e5b4-kube-api-access-jz4gp podName:6c5de78f-76c0-4c6a-9c41-787f8452e5b4 nodeName:}" failed. No retries permitted until 2026-07-06 16:24:13.655470949 +0000 UTC m=+4.269611068 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-jz4gp" (UniqueName: "kubernetes.io/projected/6c5de78f-76c0-4c6a-9c41-787f8452e5b4-kube-api-access-jz4gp") pod "network-check-target-v8kjk" (UID: "6c5de78f-76c0-4c6a-9c41-787f8452e5b4") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 16:24:12.892654 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:12.892531 2584 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-05 16:19:12 +0000 UTC" deadline="2028-04-13 04:44:41.822224408 +0000 UTC" Jul 06 16:24:12.892654 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:12.892594 2584 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15516h20m28.929635511s" Jul 06 16:24:13.122557 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:13.122519 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-27rkm" event={"ID":"e98b6fdb-3725-4c37-8a97-6a59fbc25f76","Type":"ContainerStarted","Data":"7ac5c72a273df66be5e8d2344ba10815599204babc73d25eedf67893a63319d8"} Jul 06 16:24:13.123773 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:13.123735 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-55rq5" event={"ID":"a40a609b-06a9-48c2-a98a-dc948e54049c","Type":"ContainerStarted","Data":"beec3eeeb493a574619e2bfef6c76215cac0156d25350d2dc30cd5551cc0b076"} Jul 06 16:24:13.124920 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:13.124891 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-k4gbb" event={"ID":"9bf128db-4ed4-483d-a886-cb334e6fa4ef","Type":"ContainerStarted","Data":"c1fe5f29caa842a1e29a82b8022893a8e064046270bb745972b53ad57f2a5c8f"} Jul 06 16:24:13.126207 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:13.126167 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-smm2d" event={"ID":"6d4323bb-289e-490c-938f-d4501d0fd81b","Type":"ContainerStarted","Data":"114d36c212103029e614189ba4c39311c545fbe85497600b0891acd41af6d07b"} Jul 06 16:24:13.127431 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:13.127406 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8mtdq" event={"ID":"656a2b09-7d06-48e6-9ddf-c261858ab728","Type":"ContainerStarted","Data":"379c62a931c54e9a74d50e7ca42ef23a73479c724eec376dd3bf5e40ee28d57f"} Jul 06 16:24:13.128905 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:13.128878 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-gq74s" event={"ID":"28844609-46d4-4ebf-9c6a-e714ef3bba00","Type":"ContainerStarted","Data":"5d2d4f5ae6d701ddca694e19fe511bb06c7d692d1ae33e3e1d37bab180b37430"} Jul 06 16:24:13.130577 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:13.130520 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-wg98x" event={"ID":"4c862983-24dd-471d-928b-aa352b202fd9","Type":"ContainerStarted","Data":"ef79b225096ec8e5a924c6fe9759e018da4978ab810c84d7e81a4483a1c07218"} Jul 06 16:24:13.131893 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:13.131858 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-294bx" event={"ID":"6a8a4796-dbbc-43e2-a459-7acf11f9468d","Type":"ContainerStarted","Data":"db925100e854f945db55cb328befb74fea517a0d4c814970010807d25bb8d30e"} Jul 06 16:24:13.133153 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:13.133133 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-rrqzz" event={"ID":"3d80051d-4dd9-44d5-9be4-64a500f8c435","Type":"ContainerStarted","Data":"8a5aefbd139f73988381d5b8580608473750d63ca03311eb3378115129a02de0"} Jul 06 16:24:13.565546 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:13.564523 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d695ec8b-3624-40aa-afe2-841d320d0a66-original-pull-secret\") pod \"global-pull-secret-syncer-wvcq5\" (UID: \"d695ec8b-3624-40aa-afe2-841d320d0a66\") " pod="kube-system/global-pull-secret-syncer-wvcq5" Jul 06 16:24:13.565546 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:13.564870 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3106072a-c698-44ac-9b92-8a7fd1e08d8a-metrics-certs\") pod \"network-metrics-daemon-2zdgn\" (UID: \"3106072a-c698-44ac-9b92-8a7fd1e08d8a\") " pod="openshift-multus/network-metrics-daemon-2zdgn" Jul 06 16:24:13.565546 ip-10-0-128-22 kubenswrapper[2584]: E0706 16:24:13.564700 2584 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 06 16:24:13.565546 ip-10-0-128-22 kubenswrapper[2584]: E0706 16:24:13.565107 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d695ec8b-3624-40aa-afe2-841d320d0a66-original-pull-secret podName:d695ec8b-3624-40aa-afe2-841d320d0a66 nodeName:}" failed. No retries permitted until 2026-07-06 16:24:15.565086032 +0000 UTC m=+6.179226135 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d695ec8b-3624-40aa-afe2-841d320d0a66-original-pull-secret") pod "global-pull-secret-syncer-wvcq5" (UID: "d695ec8b-3624-40aa-afe2-841d320d0a66") : object "kube-system"/"original-pull-secret" not registered Jul 06 16:24:13.565546 ip-10-0-128-22 kubenswrapper[2584]: E0706 16:24:13.565033 2584 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 16:24:13.565546 ip-10-0-128-22 kubenswrapper[2584]: E0706 16:24:13.565517 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3106072a-c698-44ac-9b92-8a7fd1e08d8a-metrics-certs podName:3106072a-c698-44ac-9b92-8a7fd1e08d8a nodeName:}" failed. No retries permitted until 2026-07-06 16:24:15.565498164 +0000 UTC m=+6.179638270 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3106072a-c698-44ac-9b92-8a7fd1e08d8a-metrics-certs") pod "network-metrics-daemon-2zdgn" (UID: "3106072a-c698-44ac-9b92-8a7fd1e08d8a") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 16:24:13.665879 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:13.665833 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jz4gp\" (UniqueName: \"kubernetes.io/projected/6c5de78f-76c0-4c6a-9c41-787f8452e5b4-kube-api-access-jz4gp\") pod \"network-check-target-v8kjk\" (UID: \"6c5de78f-76c0-4c6a-9c41-787f8452e5b4\") " pod="openshift-network-diagnostics/network-check-target-v8kjk" Jul 06 16:24:13.666079 ip-10-0-128-22 kubenswrapper[2584]: E0706 16:24:13.666025 2584 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 06 16:24:13.666079 ip-10-0-128-22 kubenswrapper[2584]: E0706 16:24:13.666046 2584 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 06 16:24:13.666079 ip-10-0-128-22 kubenswrapper[2584]: E0706 16:24:13.666060 2584 projected.go:196] Error preparing data for projected volume kube-api-access-jz4gp for pod openshift-network-diagnostics/network-check-target-v8kjk: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 16:24:13.666253 ip-10-0-128-22 kubenswrapper[2584]: E0706 16:24:13.666123 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6c5de78f-76c0-4c6a-9c41-787f8452e5b4-kube-api-access-jz4gp podName:6c5de78f-76c0-4c6a-9c41-787f8452e5b4 nodeName:}" failed. No retries permitted until 2026-07-06 16:24:15.666104233 +0000 UTC m=+6.280244359 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-jz4gp" (UniqueName: "kubernetes.io/projected/6c5de78f-76c0-4c6a-9c41-787f8452e5b4-kube-api-access-jz4gp") pod "network-check-target-v8kjk" (UID: "6c5de78f-76c0-4c6a-9c41-787f8452e5b4") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 16:24:13.892938 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:13.892750 2584 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-05 16:19:12 +0000 UTC" deadline="2028-01-23 21:48:33.177225535 +0000 UTC" Jul 06 16:24:13.892938 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:13.892838 2584 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13589h24m19.284392452s" Jul 06 16:24:14.117973 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:14.117937 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-v8kjk" Jul 06 16:24:14.118154 ip-10-0-128-22 kubenswrapper[2584]: E0706 16:24:14.118090 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-v8kjk" podUID="6c5de78f-76c0-4c6a-9c41-787f8452e5b4" Jul 06 16:24:14.118539 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:14.118519 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2zdgn" Jul 06 16:24:14.118644 ip-10-0-128-22 kubenswrapper[2584]: E0706 16:24:14.118624 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2zdgn" podUID="3106072a-c698-44ac-9b92-8a7fd1e08d8a" Jul 06 16:24:14.119970 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:14.119919 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wvcq5" Jul 06 16:24:14.120090 ip-10-0-128-22 kubenswrapper[2584]: E0706 16:24:14.120064 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wvcq5" podUID="d695ec8b-3624-40aa-afe2-841d320d0a66" Jul 06 16:24:14.146909 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:14.146820 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-22.ec2.internal" event={"ID":"a4d219c131416ebd840bc1507523f196","Type":"ContainerStarted","Data":"7b0e022b0923437a3cdd3ad0f2504191a29160cf436cadd1ccd495b97594d0e6"} Jul 06 16:24:15.591298 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:15.591255 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d695ec8b-3624-40aa-afe2-841d320d0a66-original-pull-secret\") pod \"global-pull-secret-syncer-wvcq5\" (UID: \"d695ec8b-3624-40aa-afe2-841d320d0a66\") " pod="kube-system/global-pull-secret-syncer-wvcq5" Jul 06 16:24:15.591818 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:15.591322 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3106072a-c698-44ac-9b92-8a7fd1e08d8a-metrics-certs\") pod \"network-metrics-daemon-2zdgn\" (UID: \"3106072a-c698-44ac-9b92-8a7fd1e08d8a\") " pod="openshift-multus/network-metrics-daemon-2zdgn" Jul 06 16:24:15.591818 ip-10-0-128-22 kubenswrapper[2584]: E0706 16:24:15.591525 2584 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 16:24:15.591818 ip-10-0-128-22 kubenswrapper[2584]: E0706 16:24:15.591627 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3106072a-c698-44ac-9b92-8a7fd1e08d8a-metrics-certs podName:3106072a-c698-44ac-9b92-8a7fd1e08d8a nodeName:}" failed. No retries permitted until 2026-07-06 16:24:19.591605651 +0000 UTC m=+10.205745769 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3106072a-c698-44ac-9b92-8a7fd1e08d8a-metrics-certs") pod "network-metrics-daemon-2zdgn" (UID: "3106072a-c698-44ac-9b92-8a7fd1e08d8a") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 16:24:15.592014 ip-10-0-128-22 kubenswrapper[2584]: E0706 16:24:15.591980 2584 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 06 16:24:15.592070 ip-10-0-128-22 kubenswrapper[2584]: E0706 16:24:15.592038 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d695ec8b-3624-40aa-afe2-841d320d0a66-original-pull-secret podName:d695ec8b-3624-40aa-afe2-841d320d0a66 nodeName:}" failed. No retries permitted until 2026-07-06 16:24:19.592021753 +0000 UTC m=+10.206161869 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d695ec8b-3624-40aa-afe2-841d320d0a66-original-pull-secret") pod "global-pull-secret-syncer-wvcq5" (UID: "d695ec8b-3624-40aa-afe2-841d320d0a66") : object "kube-system"/"original-pull-secret" not registered Jul 06 16:24:15.692193 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:15.692143 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jz4gp\" (UniqueName: \"kubernetes.io/projected/6c5de78f-76c0-4c6a-9c41-787f8452e5b4-kube-api-access-jz4gp\") pod \"network-check-target-v8kjk\" (UID: \"6c5de78f-76c0-4c6a-9c41-787f8452e5b4\") " pod="openshift-network-diagnostics/network-check-target-v8kjk" Jul 06 16:24:15.692388 ip-10-0-128-22 kubenswrapper[2584]: E0706 16:24:15.692372 2584 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 06 16:24:15.692450 ip-10-0-128-22 kubenswrapper[2584]: E0706 16:24:15.692403 2584 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 06 16:24:15.692450 ip-10-0-128-22 kubenswrapper[2584]: E0706 16:24:15.692420 2584 projected.go:196] Error preparing data for projected volume kube-api-access-jz4gp for pod openshift-network-diagnostics/network-check-target-v8kjk: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 16:24:15.692556 ip-10-0-128-22 kubenswrapper[2584]: E0706 16:24:15.692485 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6c5de78f-76c0-4c6a-9c41-787f8452e5b4-kube-api-access-jz4gp podName:6c5de78f-76c0-4c6a-9c41-787f8452e5b4 nodeName:}" failed. No retries permitted until 2026-07-06 16:24:19.69246488 +0000 UTC m=+10.306605002 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-jz4gp" (UniqueName: "kubernetes.io/projected/6c5de78f-76c0-4c6a-9c41-787f8452e5b4-kube-api-access-jz4gp") pod "network-check-target-v8kjk" (UID: "6c5de78f-76c0-4c6a-9c41-787f8452e5b4") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 16:24:16.113997 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:16.113910 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2zdgn" Jul 06 16:24:16.113997 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:16.113962 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-v8kjk" Jul 06 16:24:16.114221 ip-10-0-128-22 kubenswrapper[2584]: E0706 16:24:16.114071 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2zdgn" podUID="3106072a-c698-44ac-9b92-8a7fd1e08d8a" Jul 06 16:24:16.114221 ip-10-0-128-22 kubenswrapper[2584]: E0706 16:24:16.114168 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-v8kjk" podUID="6c5de78f-76c0-4c6a-9c41-787f8452e5b4" Jul 06 16:24:16.114221 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:16.114215 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wvcq5" Jul 06 16:24:16.114376 ip-10-0-128-22 kubenswrapper[2584]: E0706 16:24:16.114287 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wvcq5" podUID="d695ec8b-3624-40aa-afe2-841d320d0a66" Jul 06 16:24:16.157382 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:16.157341 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-22.ec2.internal" event={"ID":"681ecc11fd0a8e367cd02518060abe2d","Type":"ContainerStarted","Data":"8918f5adc59e94ef2a6ea36f0814dfdd7c068bf174a94f8a5820e221f660930d"} Jul 06 16:24:16.181018 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:16.180953 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-22.ec2.internal" podStartSLOduration=4.180931627 podStartE2EDuration="4.180931627s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 16:24:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 16:24:14.170226517 +0000 UTC m=+4.784366641" watchObservedRunningTime="2026-07-06 16:24:16.180931627 +0000 UTC m=+6.795071755" Jul 06 16:24:18.113756 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:18.113065 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-v8kjk" Jul 06 16:24:18.113756 ip-10-0-128-22 kubenswrapper[2584]: E0706 16:24:18.113197 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-v8kjk" podUID="6c5de78f-76c0-4c6a-9c41-787f8452e5b4" Jul 06 16:24:18.113756 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:18.113644 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2zdgn" Jul 06 16:24:18.113756 ip-10-0-128-22 kubenswrapper[2584]: E0706 16:24:18.113757 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2zdgn" podUID="3106072a-c698-44ac-9b92-8a7fd1e08d8a" Jul 06 16:24:18.114447 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:18.113830 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wvcq5" Jul 06 16:24:18.114447 ip-10-0-128-22 kubenswrapper[2584]: E0706 16:24:18.113915 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wvcq5" podUID="d695ec8b-3624-40aa-afe2-841d320d0a66" Jul 06 16:24:18.166832 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:18.166123 2584 generic.go:356] "Generic (PLEG): container finished" podID="681ecc11fd0a8e367cd02518060abe2d" containerID="8918f5adc59e94ef2a6ea36f0814dfdd7c068bf174a94f8a5820e221f660930d" exitCode=0 Jul 06 16:24:18.166832 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:18.166181 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-22.ec2.internal" event={"ID":"681ecc11fd0a8e367cd02518060abe2d","Type":"ContainerDied","Data":"8918f5adc59e94ef2a6ea36f0814dfdd7c068bf174a94f8a5820e221f660930d"} Jul 06 16:24:19.625831 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:19.625789 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d695ec8b-3624-40aa-afe2-841d320d0a66-original-pull-secret\") pod \"global-pull-secret-syncer-wvcq5\" (UID: \"d695ec8b-3624-40aa-afe2-841d320d0a66\") " pod="kube-system/global-pull-secret-syncer-wvcq5" Jul 06 16:24:19.626315 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:19.625860 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3106072a-c698-44ac-9b92-8a7fd1e08d8a-metrics-certs\") pod \"network-metrics-daemon-2zdgn\" (UID: \"3106072a-c698-44ac-9b92-8a7fd1e08d8a\") " pod="openshift-multus/network-metrics-daemon-2zdgn" Jul 06 16:24:19.626315 ip-10-0-128-22 kubenswrapper[2584]: E0706 16:24:19.625980 2584 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 16:24:19.626315 ip-10-0-128-22 kubenswrapper[2584]: E0706 16:24:19.626025 2584 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 06 16:24:19.626315 ip-10-0-128-22 kubenswrapper[2584]: E0706 16:24:19.626052 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3106072a-c698-44ac-9b92-8a7fd1e08d8a-metrics-certs podName:3106072a-c698-44ac-9b92-8a7fd1e08d8a nodeName:}" failed. No retries permitted until 2026-07-06 16:24:27.626030824 +0000 UTC m=+18.240170945 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3106072a-c698-44ac-9b92-8a7fd1e08d8a-metrics-certs") pod "network-metrics-daemon-2zdgn" (UID: "3106072a-c698-44ac-9b92-8a7fd1e08d8a") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 16:24:19.626315 ip-10-0-128-22 kubenswrapper[2584]: E0706 16:24:19.626077 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d695ec8b-3624-40aa-afe2-841d320d0a66-original-pull-secret podName:d695ec8b-3624-40aa-afe2-841d320d0a66 nodeName:}" failed. No retries permitted until 2026-07-06 16:24:27.626066264 +0000 UTC m=+18.240206365 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d695ec8b-3624-40aa-afe2-841d320d0a66-original-pull-secret") pod "global-pull-secret-syncer-wvcq5" (UID: "d695ec8b-3624-40aa-afe2-841d320d0a66") : object "kube-system"/"original-pull-secret" not registered Jul 06 16:24:19.727350 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:19.727311 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jz4gp\" (UniqueName: \"kubernetes.io/projected/6c5de78f-76c0-4c6a-9c41-787f8452e5b4-kube-api-access-jz4gp\") pod \"network-check-target-v8kjk\" (UID: \"6c5de78f-76c0-4c6a-9c41-787f8452e5b4\") " pod="openshift-network-diagnostics/network-check-target-v8kjk" Jul 06 16:24:19.727535 ip-10-0-128-22 kubenswrapper[2584]: E0706 16:24:19.727492 2584 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 06 16:24:19.727535 ip-10-0-128-22 kubenswrapper[2584]: E0706 16:24:19.727518 2584 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 06 16:24:19.727535 ip-10-0-128-22 kubenswrapper[2584]: E0706 16:24:19.727533 2584 projected.go:196] Error preparing data for projected volume kube-api-access-jz4gp for pod openshift-network-diagnostics/network-check-target-v8kjk: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 16:24:19.727689 ip-10-0-128-22 kubenswrapper[2584]: E0706 16:24:19.727590 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6c5de78f-76c0-4c6a-9c41-787f8452e5b4-kube-api-access-jz4gp podName:6c5de78f-76c0-4c6a-9c41-787f8452e5b4 nodeName:}" failed. No retries permitted until 2026-07-06 16:24:27.727571813 +0000 UTC m=+18.341711920 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-jz4gp" (UniqueName: "kubernetes.io/projected/6c5de78f-76c0-4c6a-9c41-787f8452e5b4-kube-api-access-jz4gp") pod "network-check-target-v8kjk" (UID: "6c5de78f-76c0-4c6a-9c41-787f8452e5b4") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 16:24:20.113750 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:20.113639 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-v8kjk" Jul 06 16:24:20.113750 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:20.113663 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2zdgn" Jul 06 16:24:20.113993 ip-10-0-128-22 kubenswrapper[2584]: E0706 16:24:20.113778 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-v8kjk" podUID="6c5de78f-76c0-4c6a-9c41-787f8452e5b4" Jul 06 16:24:20.113993 ip-10-0-128-22 kubenswrapper[2584]: E0706 16:24:20.113942 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2zdgn" podUID="3106072a-c698-44ac-9b92-8a7fd1e08d8a" Jul 06 16:24:20.115063 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:20.115040 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wvcq5" Jul 06 16:24:20.115163 ip-10-0-128-22 kubenswrapper[2584]: E0706 16:24:20.115124 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wvcq5" podUID="d695ec8b-3624-40aa-afe2-841d320d0a66" Jul 06 16:24:22.113098 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:22.113063 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-v8kjk" Jul 06 16:24:22.113098 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:22.113088 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2zdgn" Jul 06 16:24:22.113623 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:22.113221 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wvcq5" Jul 06 16:24:22.113623 ip-10-0-128-22 kubenswrapper[2584]: E0706 16:24:22.113229 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-v8kjk" podUID="6c5de78f-76c0-4c6a-9c41-787f8452e5b4" Jul 06 16:24:22.113623 ip-10-0-128-22 kubenswrapper[2584]: E0706 16:24:22.113328 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2zdgn" podUID="3106072a-c698-44ac-9b92-8a7fd1e08d8a" Jul 06 16:24:22.113623 ip-10-0-128-22 kubenswrapper[2584]: E0706 16:24:22.113434 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wvcq5" podUID="d695ec8b-3624-40aa-afe2-841d320d0a66" Jul 06 16:24:24.113058 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:24.112822 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-v8kjk" Jul 06 16:24:24.113494 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:24.112828 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2zdgn" Jul 06 16:24:24.113494 ip-10-0-128-22 kubenswrapper[2584]: E0706 16:24:24.113110 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-v8kjk" podUID="6c5de78f-76c0-4c6a-9c41-787f8452e5b4" Jul 06 16:24:24.113494 ip-10-0-128-22 kubenswrapper[2584]: E0706 16:24:24.113214 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2zdgn" podUID="3106072a-c698-44ac-9b92-8a7fd1e08d8a" Jul 06 16:24:24.113494 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:24.112830 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wvcq5" Jul 06 16:24:24.113494 ip-10-0-128-22 kubenswrapper[2584]: E0706 16:24:24.113321 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wvcq5" podUID="d695ec8b-3624-40aa-afe2-841d320d0a66" Jul 06 16:24:24.179507 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:24.179468 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-k4gbb" event={"ID":"9bf128db-4ed4-483d-a886-cb334e6fa4ef","Type":"ContainerStarted","Data":"179a5a7ea6102ebb4d27116559e16e277a9ef627776f763c39792229f7350fa8"} Jul 06 16:24:24.181321 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:24.181295 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-22.ec2.internal" event={"ID":"681ecc11fd0a8e367cd02518060abe2d","Type":"ContainerStarted","Data":"2446bf5e884a55489297eaa120f18759827363ad5fce84da4820eea4cd0e5f72"} Jul 06 16:24:24.182654 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:24.182631 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8mtdq" event={"ID":"656a2b09-7d06-48e6-9ddf-c261858ab728","Type":"ContainerStarted","Data":"760e54b31ad8ff2597a2e6eba08fb1b63d0a4bc6a8db1f78abb9204c1681aec1"} Jul 06 16:24:24.184120 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:24.184094 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-gq74s" event={"ID":"28844609-46d4-4ebf-9c6a-e714ef3bba00","Type":"ContainerStarted","Data":"d274a89bd4f5b329a9f88e5047376fd02cf19c19db884a0fd761abfc8c72833f"} Jul 06 16:24:24.185523 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:24.185502 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-wg98x" event={"ID":"4c862983-24dd-471d-928b-aa352b202fd9","Type":"ContainerStarted","Data":"33dc48eb0e2a29f2863130e6cbffb7fde0dd810d71002a058f36f766eb6811d9"} Jul 06 16:24:24.186845 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:24.186819 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-27rkm" event={"ID":"e98b6fdb-3725-4c37-8a97-6a59fbc25f76","Type":"ContainerStarted","Data":"a8761ac41a53f8c4f67012247c73d6d12a840a370265caf1ac477f20f5870630"} Jul 06 16:24:24.188458 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:24.188433 2584 generic.go:356] "Generic (PLEG): container finished" podID="a40a609b-06a9-48c2-a98a-dc948e54049c" containerID="d26a750280ae07018312470b3318d0e213754dc3618a01d22926b830141ae2ef" exitCode=0 Jul 06 16:24:24.188536 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:24.188469 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-55rq5" event={"ID":"a40a609b-06a9-48c2-a98a-dc948e54049c","Type":"ContainerDied","Data":"d26a750280ae07018312470b3318d0e213754dc3618a01d22926b830141ae2ef"} Jul 06 16:24:24.245305 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:24.245245 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-k4gbb" podStartSLOduration=2.03434199 podStartE2EDuration="13.245222383s" totalImagesPullingTime="11.210880393s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 16:24:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 16:24:24.212848592 +0000 UTC m=+14.826988713" watchObservedRunningTime="2026-07-06 16:24:24.245222383 +0000 UTC m=+14.859362502" Jul 06 16:24:24.274925 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:24.274861 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-wg98x" podStartSLOduration=3.014762297 podStartE2EDuration="14.274843987s" totalImagesPullingTime="11.26008169s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 16:24:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 16:24:24.245388968 +0000 UTC m=+14.859529080" watchObservedRunningTime="2026-07-06 16:24:24.274843987 +0000 UTC m=+14.888984112" Jul 06 16:24:24.275095 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:24.274952 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-27rkm" podStartSLOduration=2.066842006 podStartE2EDuration="13.274945082s" totalImagesPullingTime="11.208103076s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 16:24:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 16:24:24.274660135 +0000 UTC m=+14.888800255" watchObservedRunningTime="2026-07-06 16:24:24.274945082 +0000 UTC m=+14.889085205" Jul 06 16:24:24.306492 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:24.306455 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-k4gbb" Jul 06 16:24:24.307109 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:24.307083 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-k4gbb" Jul 06 16:24:24.352407 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:24.352357 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-22.ec2.internal" podStartSLOduration=12.352338854 podStartE2EDuration="12.352338854s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 16:24:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 16:24:24.315563303 +0000 UTC m=+14.929703425" watchObservedRunningTime="2026-07-06 16:24:24.352338854 +0000 UTC m=+14.966478975" Jul 06 16:24:24.352573 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:24.352528 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-gq74s" podStartSLOduration=2.978704676 podStartE2EDuration="14.352521435s" totalImagesPullingTime="11.373816759s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 16:24:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 16:24:24.351615749 +0000 UTC m=+14.965755872" watchObservedRunningTime="2026-07-06 16:24:24.352521435 +0000 UTC m=+14.966661557" Jul 06 16:24:24.375092 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:24.374949 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:24:24.446746 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:24.446685 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:24:24.521022 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:24.520982 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:24:25.192903 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:25.192852 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-smm2d" event={"ID":"6d4323bb-289e-490c-938f-d4501d0fd81b","Type":"ContainerStarted","Data":"4f6e18030e1c4c432415d49db049d56f3d392b2a1af220be2c628df790c44cf3"} Jul 06 16:24:25.193702 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:25.193675 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-k4gbb" Jul 06 16:24:25.194226 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:25.194202 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-k4gbb" Jul 06 16:24:25.214061 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:25.214003 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-smm2d" podStartSLOduration=3.968740718 podStartE2EDuration="15.213981375s" totalImagesPullingTime="11.245240657s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 16:24:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 16:24:25.213833628 +0000 UTC m=+15.827973750" watchObservedRunningTime="2026-07-06 16:24:25.213981375 +0000 UTC m=+15.828121503" Jul 06 16:24:25.352951 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:25.352879 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:24:25.426935 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:25.426895 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:24:26.105076 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:26.105037 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:24:26.113086 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:26.113057 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wvcq5" Jul 06 16:24:26.113086 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:26.113072 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2zdgn" Jul 06 16:24:26.113306 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:26.113057 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-v8kjk" Jul 06 16:24:26.113306 ip-10-0-128-22 kubenswrapper[2584]: E0706 16:24:26.113196 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wvcq5" podUID="d695ec8b-3624-40aa-afe2-841d320d0a66" Jul 06 16:24:26.113306 ip-10-0-128-22 kubenswrapper[2584]: E0706 16:24:26.113258 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-v8kjk" podUID="6c5de78f-76c0-4c6a-9c41-787f8452e5b4" Jul 06 16:24:26.113431 ip-10-0-128-22 kubenswrapper[2584]: E0706 16:24:26.113351 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2zdgn" podUID="3106072a-c698-44ac-9b92-8a7fd1e08d8a" Jul 06 16:24:26.204057 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:26.204027 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:24:27.673243 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:27.673199 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:24:27.689841 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:27.689764 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d695ec8b-3624-40aa-afe2-841d320d0a66-original-pull-secret\") pod \"global-pull-secret-syncer-wvcq5\" (UID: \"d695ec8b-3624-40aa-afe2-841d320d0a66\") " pod="kube-system/global-pull-secret-syncer-wvcq5" Jul 06 16:24:27.689841 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:27.689814 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3106072a-c698-44ac-9b92-8a7fd1e08d8a-metrics-certs\") pod \"network-metrics-daemon-2zdgn\" (UID: \"3106072a-c698-44ac-9b92-8a7fd1e08d8a\") " pod="openshift-multus/network-metrics-daemon-2zdgn" Jul 06 16:24:27.690073 ip-10-0-128-22 kubenswrapper[2584]: E0706 16:24:27.689889 2584 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 06 16:24:27.690073 ip-10-0-128-22 kubenswrapper[2584]: E0706 16:24:27.689909 2584 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 16:24:27.690073 ip-10-0-128-22 kubenswrapper[2584]: E0706 16:24:27.689953 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d695ec8b-3624-40aa-afe2-841d320d0a66-original-pull-secret podName:d695ec8b-3624-40aa-afe2-841d320d0a66 nodeName:}" failed. No retries permitted until 2026-07-06 16:24:43.689935798 +0000 UTC m=+34.304075907 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d695ec8b-3624-40aa-afe2-841d320d0a66-original-pull-secret") pod "global-pull-secret-syncer-wvcq5" (UID: "d695ec8b-3624-40aa-afe2-841d320d0a66") : object "kube-system"/"original-pull-secret" not registered Jul 06 16:24:27.690073 ip-10-0-128-22 kubenswrapper[2584]: E0706 16:24:27.689969 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3106072a-c698-44ac-9b92-8a7fd1e08d8a-metrics-certs podName:3106072a-c698-44ac-9b92-8a7fd1e08d8a nodeName:}" failed. No retries permitted until 2026-07-06 16:24:43.689961401 +0000 UTC m=+34.304101511 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3106072a-c698-44ac-9b92-8a7fd1e08d8a-metrics-certs") pod "network-metrics-daemon-2zdgn" (UID: "3106072a-c698-44ac-9b92-8a7fd1e08d8a") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 16:24:27.790466 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:27.790434 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jz4gp\" (UniqueName: \"kubernetes.io/projected/6c5de78f-76c0-4c6a-9c41-787f8452e5b4-kube-api-access-jz4gp\") pod \"network-check-target-v8kjk\" (UID: \"6c5de78f-76c0-4c6a-9c41-787f8452e5b4\") " pod="openshift-network-diagnostics/network-check-target-v8kjk" Jul 06 16:24:27.790630 ip-10-0-128-22 kubenswrapper[2584]: E0706 16:24:27.790560 2584 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 06 16:24:27.790630 ip-10-0-128-22 kubenswrapper[2584]: E0706 16:24:27.790576 2584 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 06 16:24:27.790630 ip-10-0-128-22 kubenswrapper[2584]: E0706 16:24:27.790585 2584 projected.go:196] Error preparing data for projected volume kube-api-access-jz4gp for pod openshift-network-diagnostics/network-check-target-v8kjk: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 16:24:27.790787 ip-10-0-128-22 kubenswrapper[2584]: E0706 16:24:27.790632 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6c5de78f-76c0-4c6a-9c41-787f8452e5b4-kube-api-access-jz4gp podName:6c5de78f-76c0-4c6a-9c41-787f8452e5b4 nodeName:}" failed. No retries permitted until 2026-07-06 16:24:43.790616089 +0000 UTC m=+34.404756206 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-jz4gp" (UniqueName: "kubernetes.io/projected/6c5de78f-76c0-4c6a-9c41-787f8452e5b4-kube-api-access-jz4gp") pod "network-check-target-v8kjk" (UID: "6c5de78f-76c0-4c6a-9c41-787f8452e5b4") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 16:24:28.113103 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:28.113027 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2zdgn" Jul 06 16:24:28.113268 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:28.113027 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-v8kjk" Jul 06 16:24:28.113268 ip-10-0-128-22 kubenswrapper[2584]: E0706 16:24:28.113158 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2zdgn" podUID="3106072a-c698-44ac-9b92-8a7fd1e08d8a" Jul 06 16:24:28.113268 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:28.113036 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wvcq5" Jul 06 16:24:28.113268 ip-10-0-128-22 kubenswrapper[2584]: E0706 16:24:28.113223 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-v8kjk" podUID="6c5de78f-76c0-4c6a-9c41-787f8452e5b4" Jul 06 16:24:28.113473 ip-10-0-128-22 kubenswrapper[2584]: E0706 16:24:28.113296 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wvcq5" podUID="d695ec8b-3624-40aa-afe2-841d320d0a66" Jul 06 16:24:28.756100 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:28.756056 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:24:29.094320 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:29.094213 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:24:29.295504 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:29.295436 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:24:29.450424 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:29.450368 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:24:29.628897 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:29.628797 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:24:30.057291 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:30.057251 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:24:30.113750 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:30.113704 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-v8kjk" Jul 06 16:24:30.113916 ip-10-0-128-22 kubenswrapper[2584]: E0706 16:24:30.113882 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-v8kjk" podUID="6c5de78f-76c0-4c6a-9c41-787f8452e5b4" Jul 06 16:24:30.113993 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:30.113925 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2zdgn" Jul 06 16:24:30.114055 ip-10-0-128-22 kubenswrapper[2584]: E0706 16:24:30.114024 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2zdgn" podUID="3106072a-c698-44ac-9b92-8a7fd1e08d8a" Jul 06 16:24:30.114055 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:30.114050 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wvcq5" Jul 06 16:24:30.114129 ip-10-0-128-22 kubenswrapper[2584]: E0706 16:24:30.114116 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wvcq5" podUID="d695ec8b-3624-40aa-afe2-841d320d0a66" Jul 06 16:24:30.450382 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:30.450341 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:24:30.579885 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:30.579841 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:24:31.065417 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:31.065385 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:24:31.957372 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:31.957070 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:24:32.058355 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:32.058276 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:24:32.112373 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:32.112342 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-v8kjk" Jul 06 16:24:32.112911 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:32.112380 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2zdgn" Jul 06 16:24:32.112911 ip-10-0-128-22 kubenswrapper[2584]: E0706 16:24:32.112489 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-v8kjk" podUID="6c5de78f-76c0-4c6a-9c41-787f8452e5b4" Jul 06 16:24:32.112911 ip-10-0-128-22 kubenswrapper[2584]: E0706 16:24:32.112537 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2zdgn" podUID="3106072a-c698-44ac-9b92-8a7fd1e08d8a" Jul 06 16:24:32.112911 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:32.112548 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wvcq5" Jul 06 16:24:32.112911 ip-10-0-128-22 kubenswrapper[2584]: E0706 16:24:32.112651 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wvcq5" podUID="d695ec8b-3624-40aa-afe2-841d320d0a66" Jul 06 16:24:32.278624 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:32.278502 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:24:32.455038 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:32.455007 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:24:32.582918 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:32.582829 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:24:32.683657 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:32.683625 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:24:32.850371 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:32.850273 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:24:33.256938 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:33.256905 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:24:33.360979 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:33.360946 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:24:33.716279 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:33.716204 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:24:34.112870 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:34.112780 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-v8kjk" Jul 06 16:24:34.113039 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:34.112789 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wvcq5" Jul 06 16:24:34.113039 ip-10-0-128-22 kubenswrapper[2584]: E0706 16:24:34.112889 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-v8kjk" podUID="6c5de78f-76c0-4c6a-9c41-787f8452e5b4" Jul 06 16:24:34.113039 ip-10-0-128-22 kubenswrapper[2584]: E0706 16:24:34.112998 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wvcq5" podUID="d695ec8b-3624-40aa-afe2-841d320d0a66" Jul 06 16:24:34.113039 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:34.112791 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2zdgn" Jul 06 16:24:34.113191 ip-10-0-128-22 kubenswrapper[2584]: E0706 16:24:34.113109 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2zdgn" podUID="3106072a-c698-44ac-9b92-8a7fd1e08d8a" Jul 06 16:24:34.213495 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:34.213457 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-294bx" event={"ID":"6a8a4796-dbbc-43e2-a459-7acf11f9468d","Type":"ContainerStarted","Data":"970104314c10697caf0b1f2374f8a29601b314f8a3539507529eb95c8ccd958c"} Jul 06 16:24:34.213669 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:34.213546 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-294bx" event={"ID":"6a8a4796-dbbc-43e2-a459-7acf11f9468d","Type":"ContainerStarted","Data":"6db3c726ace58eb4b418920e003211286f5872c7296ed2d15c55117ac4b87a3b"} Jul 06 16:24:34.215039 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:34.215007 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-rrqzz" event={"ID":"3d80051d-4dd9-44d5-9be4-64a500f8c435","Type":"ContainerStarted","Data":"fdfffc5321880c513c789616161bfbcb10f03da7155df6a3c0f40624224223c4"} Jul 06 16:24:34.296165 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:34.296131 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:24:34.479434 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:34.479397 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:24:34.568214 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:34.568175 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:24:34.901746 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:34.901622 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:24:35.081129 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:35.081106 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:24:35.181769 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:35.181572 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:24:35.219213 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:35.219193 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-294bx_6a8a4796-dbbc-43e2-a459-7acf11f9468d/ovn-acl-logging/0.log" Jul 06 16:24:35.219604 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:35.219582 2584 generic.go:356] "Generic (PLEG): container finished" podID="6a8a4796-dbbc-43e2-a459-7acf11f9468d" containerID="970104314c10697caf0b1f2374f8a29601b314f8a3539507529eb95c8ccd958c" exitCode=1 Jul 06 16:24:35.219697 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:35.219658 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-294bx" event={"ID":"6a8a4796-dbbc-43e2-a459-7acf11f9468d","Type":"ContainerDied","Data":"970104314c10697caf0b1f2374f8a29601b314f8a3539507529eb95c8ccd958c"} Jul 06 16:24:35.219793 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:35.219766 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-294bx" event={"ID":"6a8a4796-dbbc-43e2-a459-7acf11f9468d","Type":"ContainerStarted","Data":"911b36252c50663fc4fece352e0a306b0e24aca970915e310a6e2ab6d99c5184"} Jul 06 16:24:35.227159 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:35.227131 2584 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 06 16:24:35.455067 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:35.455042 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:24:35.659461 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:35.659333 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:24:35.936755 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:35.936604 2584 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-06T16:24:35.227154777Z","UUID":"76972b38-17fd-4d7d-be93-2d96db287d60","Handler":null,"Name":"","Endpoint":""} Jul 06 16:24:35.938199 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:35.938180 2584 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 06 16:24:35.938313 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:35.938214 2584 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 06 16:24:36.105845 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:36.105815 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:24:36.113217 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:36.113198 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2zdgn" Jul 06 16:24:36.113317 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:36.113200 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-v8kjk" Jul 06 16:24:36.113317 ip-10-0-128-22 kubenswrapper[2584]: E0706 16:24:36.113298 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2zdgn" podUID="3106072a-c698-44ac-9b92-8a7fd1e08d8a" Jul 06 16:24:36.113317 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:36.113310 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wvcq5" Jul 06 16:24:36.113469 ip-10-0-128-22 kubenswrapper[2584]: E0706 16:24:36.113399 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-v8kjk" podUID="6c5de78f-76c0-4c6a-9c41-787f8452e5b4" Jul 06 16:24:36.113526 ip-10-0-128-22 kubenswrapper[2584]: E0706 16:24:36.113506 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wvcq5" podUID="d695ec8b-3624-40aa-afe2-841d320d0a66" Jul 06 16:24:36.225298 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:36.225228 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-294bx_6a8a4796-dbbc-43e2-a459-7acf11f9468d/ovn-acl-logging/0.log" Jul 06 16:24:36.225600 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:36.225577 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-294bx" event={"ID":"6a8a4796-dbbc-43e2-a459-7acf11f9468d","Type":"ContainerStarted","Data":"353f075ae387769d70aa15647bd115fcdc328ffeb443182d010ae82d806e63a3"} Jul 06 16:24:36.225671 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:36.225643 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-294bx" event={"ID":"6a8a4796-dbbc-43e2-a459-7acf11f9468d","Type":"ContainerStarted","Data":"41e825e3341fe9b7b60be2c6b946cf8716a9383128fb47ce13d916ec3b9f44b6"} Jul 06 16:24:36.225718 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:36.225678 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-294bx" event={"ID":"6a8a4796-dbbc-43e2-a459-7acf11f9468d","Type":"ContainerStarted","Data":"c9b57a60bbb8efec84f1783c9ac23ac4ba7b77e3f46d06cfd98c724a148d607d"} Jul 06 16:24:36.227146 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:36.227122 2584 generic.go:356] "Generic (PLEG): container finished" podID="a40a609b-06a9-48c2-a98a-dc948e54049c" containerID="f8fff4da65fa889b090066699041d965e973722de4bb2f96e1b06efb044bb393" exitCode=0 Jul 06 16:24:36.227278 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:36.227191 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-55rq5" event={"ID":"a40a609b-06a9-48c2-a98a-dc948e54049c","Type":"ContainerDied","Data":"f8fff4da65fa889b090066699041d965e973722de4bb2f96e1b06efb044bb393"} Jul 06 16:24:36.228931 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:36.228913 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8mtdq" event={"ID":"656a2b09-7d06-48e6-9ddf-c261858ab728","Type":"ContainerStarted","Data":"4e36db310c7509313dd4e721adbe32ee0c534384275a967705a9ec9467604fed"} Jul 06 16:24:36.259699 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:36.259649 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-rrqzz" podStartSLOduration=5.256730739 podStartE2EDuration="26.259633278s" totalImagesPullingTime="21.002902539s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 16:24:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 16:24:34.260194696 +0000 UTC m=+24.874334829" watchObservedRunningTime="2026-07-06 16:24:36.259633278 +0000 UTC m=+26.873773417" Jul 06 16:24:36.461248 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:36.461209 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:24:36.850941 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:36.850909 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:24:37.232766 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:37.232715 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8mtdq" event={"ID":"656a2b09-7d06-48e6-9ddf-c261858ab728","Type":"ContainerStarted","Data":"84f2179921e84f92556eb9dcd678cd77ec5c6b7aac742aaa40a79c159b109c81"} Jul 06 16:24:37.260677 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:37.260621 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8mtdq" podStartSLOduration=2.34307138 podStartE2EDuration="26.260602556s" totalImagesPullingTime="23.917531176s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 16:24:11 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 16:24:37.260504762 +0000 UTC m=+27.874644884" watchObservedRunningTime="2026-07-06 16:24:37.260602556 +0000 UTC m=+27.874742677" Jul 06 16:24:37.665286 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:37.665260 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:24:37.877267 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:37.877226 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:24:38.113091 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:38.113005 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-v8kjk" Jul 06 16:24:38.113302 ip-10-0-128-22 kubenswrapper[2584]: E0706 16:24:38.113106 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-v8kjk" podUID="6c5de78f-76c0-4c6a-9c41-787f8452e5b4" Jul 06 16:24:38.113302 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:38.113142 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wvcq5" Jul 06 16:24:38.113302 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:38.113005 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2zdgn" Jul 06 16:24:38.113302 ip-10-0-128-22 kubenswrapper[2584]: E0706 16:24:38.113253 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wvcq5" podUID="d695ec8b-3624-40aa-afe2-841d320d0a66" Jul 06 16:24:38.113455 ip-10-0-128-22 kubenswrapper[2584]: E0706 16:24:38.113324 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2zdgn" podUID="3106072a-c698-44ac-9b92-8a7fd1e08d8a" Jul 06 16:24:38.133783 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:38.133751 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:24:38.233502 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:38.233467 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:24:38.237270 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:38.237248 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-294bx_6a8a4796-dbbc-43e2-a459-7acf11f9468d/ovn-acl-logging/0.log" Jul 06 16:24:38.237603 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:38.237582 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-294bx" event={"ID":"6a8a4796-dbbc-43e2-a459-7acf11f9468d","Type":"ContainerStarted","Data":"3dd59a166b7b78f84f48b980037e750a24191a8e7c4b639f10054f4a0df9216e"} Jul 06 16:24:38.239519 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:38.239497 2584 generic.go:356] "Generic (PLEG): container finished" podID="a40a609b-06a9-48c2-a98a-dc948e54049c" containerID="5cb8a88cce99f5aca18e7cb2930731051a0a965ae96311295fe83966d5cff07c" exitCode=0 Jul 06 16:24:38.239625 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:38.239582 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-55rq5" event={"ID":"a40a609b-06a9-48c2-a98a-dc948e54049c","Type":"ContainerDied","Data":"5cb8a88cce99f5aca18e7cb2930731051a0a965ae96311295fe83966d5cff07c"} Jul 06 16:24:38.328866 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:38.328826 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:24:38.479634 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:38.479605 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:24:38.859947 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:38.859859 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:24:39.253041 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:39.253010 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:24:39.326342 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:39.326312 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:24:39.475603 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:39.475581 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:24:40.113760 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:40.113579 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-v8kjk" Jul 06 16:24:40.114315 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:40.113656 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2zdgn" Jul 06 16:24:40.114315 ip-10-0-128-22 kubenswrapper[2584]: E0706 16:24:40.113856 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-v8kjk" podUID="6c5de78f-76c0-4c6a-9c41-787f8452e5b4" Jul 06 16:24:40.114315 ip-10-0-128-22 kubenswrapper[2584]: E0706 16:24:40.113969 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2zdgn" podUID="3106072a-c698-44ac-9b92-8a7fd1e08d8a" Jul 06 16:24:40.114315 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:40.113692 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wvcq5" Jul 06 16:24:40.114315 ip-10-0-128-22 kubenswrapper[2584]: E0706 16:24:40.114096 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wvcq5" podUID="d695ec8b-3624-40aa-afe2-841d320d0a66" Jul 06 16:24:40.246101 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:40.246072 2584 generic.go:356] "Generic (PLEG): container finished" podID="a40a609b-06a9-48c2-a98a-dc948e54049c" containerID="ecb9e234b960cd6800c40429ff6e4c32c33e2ef1aff2bef8da8f6fdc1dc20ed1" exitCode=0 Jul 06 16:24:40.246273 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:40.246169 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-55rq5" event={"ID":"a40a609b-06a9-48c2-a98a-dc948e54049c","Type":"ContainerDied","Data":"ecb9e234b960cd6800c40429ff6e4c32c33e2ef1aff2bef8da8f6fdc1dc20ed1"} Jul 06 16:24:40.249607 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:40.249588 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-294bx_6a8a4796-dbbc-43e2-a459-7acf11f9468d/ovn-acl-logging/0.log" Jul 06 16:24:40.249917 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:40.249897 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-294bx" event={"ID":"6a8a4796-dbbc-43e2-a459-7acf11f9468d","Type":"ContainerStarted","Data":"ae814ff6151075c55fa6cfac6e025332726afa393fbf44386b01f8bba47495d5"} Jul 06 16:24:40.250181 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:40.250167 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-294bx" Jul 06 16:24:40.250252 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:40.250239 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-294bx" Jul 06 16:24:40.250317 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:40.250291 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-294bx" Jul 06 16:24:40.250363 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:40.250339 2584 scope.go:117] "RemoveContainer" containerID="970104314c10697caf0b1f2374f8a29601b314f8a3539507529eb95c8ccd958c" Jul 06 16:24:40.267051 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:40.267027 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-294bx" Jul 06 16:24:40.267195 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:40.267182 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-294bx" Jul 06 16:24:41.255633 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:41.255605 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-294bx_6a8a4796-dbbc-43e2-a459-7acf11f9468d/ovn-acl-logging/0.log" Jul 06 16:24:41.256074 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:41.255934 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-294bx" event={"ID":"6a8a4796-dbbc-43e2-a459-7acf11f9468d","Type":"ContainerStarted","Data":"af461ce6f224fe032cd6869a8ce176fe55feac5185d47c2e2da510319d8ef04b"} Jul 06 16:24:41.292157 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:41.292082 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-294bx" podStartSLOduration=10.296083418 podStartE2EDuration="31.292065877s" totalImagesPullingTime="20.995982459s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 16:24:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 16:24:41.291750589 +0000 UTC m=+31.905890709" watchObservedRunningTime="2026-07-06 16:24:41.292065877 +0000 UTC m=+31.906206011" Jul 06 16:24:42.112695 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:42.112662 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-v8kjk" Jul 06 16:24:42.112695 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:42.112670 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2zdgn" Jul 06 16:24:42.112985 ip-10-0-128-22 kubenswrapper[2584]: E0706 16:24:42.112797 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-v8kjk" podUID="6c5de78f-76c0-4c6a-9c41-787f8452e5b4" Jul 06 16:24:42.112985 ip-10-0-128-22 kubenswrapper[2584]: E0706 16:24:42.112848 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2zdgn" podUID="3106072a-c698-44ac-9b92-8a7fd1e08d8a" Jul 06 16:24:42.112985 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:42.112888 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wvcq5" Jul 06 16:24:42.112985 ip-10-0-128-22 kubenswrapper[2584]: E0706 16:24:42.112963 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wvcq5" podUID="d695ec8b-3624-40aa-afe2-841d320d0a66" Jul 06 16:24:43.709584 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:43.709544 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d695ec8b-3624-40aa-afe2-841d320d0a66-original-pull-secret\") pod \"global-pull-secret-syncer-wvcq5\" (UID: \"d695ec8b-3624-40aa-afe2-841d320d0a66\") " pod="kube-system/global-pull-secret-syncer-wvcq5" Jul 06 16:24:43.710244 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:43.709597 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3106072a-c698-44ac-9b92-8a7fd1e08d8a-metrics-certs\") pod \"network-metrics-daemon-2zdgn\" (UID: \"3106072a-c698-44ac-9b92-8a7fd1e08d8a\") " pod="openshift-multus/network-metrics-daemon-2zdgn" Jul 06 16:24:43.710244 ip-10-0-128-22 kubenswrapper[2584]: E0706 16:24:43.709704 2584 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 16:24:43.710244 ip-10-0-128-22 kubenswrapper[2584]: E0706 16:24:43.709782 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3106072a-c698-44ac-9b92-8a7fd1e08d8a-metrics-certs podName:3106072a-c698-44ac-9b92-8a7fd1e08d8a nodeName:}" failed. No retries permitted until 2026-07-06 16:25:15.709762688 +0000 UTC m=+66.323902790 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3106072a-c698-44ac-9b92-8a7fd1e08d8a-metrics-certs") pod "network-metrics-daemon-2zdgn" (UID: "3106072a-c698-44ac-9b92-8a7fd1e08d8a") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 16:24:43.710244 ip-10-0-128-22 kubenswrapper[2584]: E0706 16:24:43.709809 2584 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 06 16:24:43.710244 ip-10-0-128-22 kubenswrapper[2584]: E0706 16:24:43.709857 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d695ec8b-3624-40aa-afe2-841d320d0a66-original-pull-secret podName:d695ec8b-3624-40aa-afe2-841d320d0a66 nodeName:}" failed. No retries permitted until 2026-07-06 16:25:15.709845215 +0000 UTC m=+66.323985317 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d695ec8b-3624-40aa-afe2-841d320d0a66-original-pull-secret") pod "global-pull-secret-syncer-wvcq5" (UID: "d695ec8b-3624-40aa-afe2-841d320d0a66") : object "kube-system"/"original-pull-secret" not registered Jul 06 16:24:43.810891 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:43.810858 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jz4gp\" (UniqueName: \"kubernetes.io/projected/6c5de78f-76c0-4c6a-9c41-787f8452e5b4-kube-api-access-jz4gp\") pod \"network-check-target-v8kjk\" (UID: \"6c5de78f-76c0-4c6a-9c41-787f8452e5b4\") " pod="openshift-network-diagnostics/network-check-target-v8kjk" Jul 06 16:24:43.811044 ip-10-0-128-22 kubenswrapper[2584]: E0706 16:24:43.811012 2584 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 06 16:24:43.811044 ip-10-0-128-22 kubenswrapper[2584]: E0706 16:24:43.811030 2584 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 06 16:24:43.811044 ip-10-0-128-22 kubenswrapper[2584]: E0706 16:24:43.811040 2584 projected.go:196] Error preparing data for projected volume kube-api-access-jz4gp for pod openshift-network-diagnostics/network-check-target-v8kjk: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 16:24:43.811145 ip-10-0-128-22 kubenswrapper[2584]: E0706 16:24:43.811091 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6c5de78f-76c0-4c6a-9c41-787f8452e5b4-kube-api-access-jz4gp podName:6c5de78f-76c0-4c6a-9c41-787f8452e5b4 nodeName:}" failed. No retries permitted until 2026-07-06 16:25:15.811075374 +0000 UTC m=+66.425215494 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-jz4gp" (UniqueName: "kubernetes.io/projected/6c5de78f-76c0-4c6a-9c41-787f8452e5b4-kube-api-access-jz4gp") pod "network-check-target-v8kjk" (UID: "6c5de78f-76c0-4c6a-9c41-787f8452e5b4") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 16:24:44.112419 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:44.112340 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2zdgn" Jul 06 16:24:44.112593 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:44.112340 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wvcq5" Jul 06 16:24:44.112593 ip-10-0-128-22 kubenswrapper[2584]: E0706 16:24:44.112478 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2zdgn" podUID="3106072a-c698-44ac-9b92-8a7fd1e08d8a" Jul 06 16:24:44.112593 ip-10-0-128-22 kubenswrapper[2584]: E0706 16:24:44.112552 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wvcq5" podUID="d695ec8b-3624-40aa-afe2-841d320d0a66" Jul 06 16:24:44.112593 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:44.112339 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-v8kjk" Jul 06 16:24:44.112834 ip-10-0-128-22 kubenswrapper[2584]: E0706 16:24:44.112663 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-v8kjk" podUID="6c5de78f-76c0-4c6a-9c41-787f8452e5b4" Jul 06 16:24:46.112524 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:46.112494 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2zdgn" Jul 06 16:24:46.112993 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:46.112502 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wvcq5" Jul 06 16:24:46.112993 ip-10-0-128-22 kubenswrapper[2584]: E0706 16:24:46.112609 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2zdgn" podUID="3106072a-c698-44ac-9b92-8a7fd1e08d8a" Jul 06 16:24:46.112993 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:46.112626 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-v8kjk" Jul 06 16:24:46.112993 ip-10-0-128-22 kubenswrapper[2584]: E0706 16:24:46.112740 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wvcq5" podUID="d695ec8b-3624-40aa-afe2-841d320d0a66" Jul 06 16:24:46.112993 ip-10-0-128-22 kubenswrapper[2584]: E0706 16:24:46.112833 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-v8kjk" podUID="6c5de78f-76c0-4c6a-9c41-787f8452e5b4" Jul 06 16:24:46.417872 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:46.417836 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:24:46.612485 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:46.612453 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:24:46.734010 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:46.733912 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:24:46.850490 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:46.850459 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:24:47.403576 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:47.403536 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-wvcq5"] Jul 06 16:24:47.404017 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:47.403690 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wvcq5" Jul 06 16:24:47.404017 ip-10-0-128-22 kubenswrapper[2584]: E0706 16:24:47.403813 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wvcq5" podUID="d695ec8b-3624-40aa-afe2-841d320d0a66" Jul 06 16:24:47.408155 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:47.407770 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-v8kjk"] Jul 06 16:24:47.408155 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:47.407885 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-v8kjk" Jul 06 16:24:47.408155 ip-10-0-128-22 kubenswrapper[2584]: E0706 16:24:47.407984 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-v8kjk" podUID="6c5de78f-76c0-4c6a-9c41-787f8452e5b4" Jul 06 16:24:47.408739 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:47.408703 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-2zdgn"] Jul 06 16:24:47.408845 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:47.408827 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2zdgn" Jul 06 16:24:47.408951 ip-10-0-128-22 kubenswrapper[2584]: E0706 16:24:47.408927 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2zdgn" podUID="3106072a-c698-44ac-9b92-8a7fd1e08d8a" Jul 06 16:24:49.113219 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:49.113032 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-v8kjk" Jul 06 16:24:49.113681 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:49.113060 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2zdgn" Jul 06 16:24:49.113681 ip-10-0-128-22 kubenswrapper[2584]: E0706 16:24:49.113311 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-v8kjk" podUID="6c5de78f-76c0-4c6a-9c41-787f8452e5b4" Jul 06 16:24:49.113681 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:49.113095 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wvcq5" Jul 06 16:24:49.113681 ip-10-0-128-22 kubenswrapper[2584]: E0706 16:24:49.113373 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2zdgn" podUID="3106072a-c698-44ac-9b92-8a7fd1e08d8a" Jul 06 16:24:49.113681 ip-10-0-128-22 kubenswrapper[2584]: E0706 16:24:49.113428 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wvcq5" podUID="d695ec8b-3624-40aa-afe2-841d320d0a66" Jul 06 16:24:49.278035 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:49.278000 2584 generic.go:356] "Generic (PLEG): container finished" podID="a40a609b-06a9-48c2-a98a-dc948e54049c" containerID="ef69e95e3f6edb93fd0ec26c3ac17760cb4f96dbf83c81ba28b21accaafe32d3" exitCode=0 Jul 06 16:24:49.278199 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:49.278069 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-55rq5" event={"ID":"a40a609b-06a9-48c2-a98a-dc948e54049c","Type":"ContainerDied","Data":"ef69e95e3f6edb93fd0ec26c3ac17760cb4f96dbf83c81ba28b21accaafe32d3"} Jul 06 16:24:50.282583 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:50.282549 2584 generic.go:356] "Generic (PLEG): container finished" podID="a40a609b-06a9-48c2-a98a-dc948e54049c" containerID="4206e9f5d2e02d384a64ddc8abc968946b2ca7b8b13c9867a8a0043ed6a4f95f" exitCode=0 Jul 06 16:24:50.283017 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:50.282625 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-55rq5" event={"ID":"a40a609b-06a9-48c2-a98a-dc948e54049c","Type":"ContainerDied","Data":"4206e9f5d2e02d384a64ddc8abc968946b2ca7b8b13c9867a8a0043ed6a4f95f"} Jul 06 16:24:50.865098 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:50.865068 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:24:50.969359 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:50.969325 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:24:51.031091 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:51.031056 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:24:51.112812 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:51.112779 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-v8kjk" Jul 06 16:24:51.112970 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:51.112779 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2zdgn" Jul 06 16:24:51.112970 ip-10-0-128-22 kubenswrapper[2584]: E0706 16:24:51.112893 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-v8kjk" podUID="6c5de78f-76c0-4c6a-9c41-787f8452e5b4" Jul 06 16:24:51.112970 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:51.112779 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wvcq5" Jul 06 16:24:51.113093 ip-10-0-128-22 kubenswrapper[2584]: E0706 16:24:51.112973 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2zdgn" podUID="3106072a-c698-44ac-9b92-8a7fd1e08d8a" Jul 06 16:24:51.113093 ip-10-0-128-22 kubenswrapper[2584]: E0706 16:24:51.113027 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wvcq5" podUID="d695ec8b-3624-40aa-afe2-841d320d0a66" Jul 06 16:24:51.288112 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:51.288071 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-55rq5" event={"ID":"a40a609b-06a9-48c2-a98a-dc948e54049c","Type":"ContainerStarted","Data":"0b25ca9e973d799296445a26f41d2ea0506a9d6fb61f5c03efd25545f942eb26"} Jul 06 16:24:51.320679 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:51.320626 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-55rq5" podStartSLOduration=7.122185862 podStartE2EDuration="40.320612264s" totalImagesPullingTime="33.198426402s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 16:24:11 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 16:24:51.319213155 +0000 UTC m=+41.933353276" watchObservedRunningTime="2026-07-06 16:24:51.320612264 +0000 UTC m=+41.934752386" Jul 06 16:24:51.448327 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:51.448293 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:24:51.529531 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:51.529498 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:24:51.657042 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:51.657017 2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-22.ec2.internal" event="NodeReady" Jul 06 16:24:51.657165 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:51.657153 2584 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 06 16:24:51.717905 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:51.717874 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-t9nd7"] Jul 06 16:24:51.737710 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:51.737648 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:24:51.741939 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:51.741914 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-wj5s2"] Jul 06 16:24:51.742067 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:51.742035 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-t9nd7" Jul 06 16:24:51.744916 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:51.744894 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 06 16:24:51.744916 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:51.744909 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 06 16:24:51.745083 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:51.744899 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-wmlq9\"" Jul 06 16:24:51.766087 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:51.766062 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-t9nd7"] Jul 06 16:24:51.766193 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:51.766098 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-wj5s2"] Jul 06 16:24:51.766193 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:51.766174 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-wj5s2" Jul 06 16:24:51.769860 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:51.769840 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 06 16:24:51.769991 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:51.769932 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-25tl8\"" Jul 06 16:24:51.769991 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:51.769942 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 06 16:24:51.770163 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:51.770150 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 06 16:24:51.834572 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:51.834494 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-pdz7v"] Jul 06 16:24:51.850670 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:51.850648 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-pdz7v" Jul 06 16:24:51.854825 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:51.854790 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-gb7wf\"" Jul 06 16:24:51.855128 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:51.855110 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 06 16:24:51.855234 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:51.855165 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 06 16:24:51.855298 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:51.855230 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 06 16:24:51.855298 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:51.855251 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 06 16:24:51.856048 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:51.856026 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-pdz7v"] Jul 06 16:24:51.869949 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:51.869926 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dvs77\" (UniqueName: \"kubernetes.io/projected/61652b90-d462-4c62-8c64-6e0843e2ebd5-kube-api-access-dvs77\") pod \"dns-default-t9nd7\" (UID: \"61652b90-d462-4c62-8c64-6e0843e2ebd5\") " pod="openshift-dns/dns-default-t9nd7" Jul 06 16:24:51.870047 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:51.869981 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/61652b90-d462-4c62-8c64-6e0843e2ebd5-config-volume\") pod \"dns-default-t9nd7\" (UID: \"61652b90-d462-4c62-8c64-6e0843e2ebd5\") " pod="openshift-dns/dns-default-t9nd7" Jul 06 16:24:51.870047 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:51.870007 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/61652b90-d462-4c62-8c64-6e0843e2ebd5-tmp-dir\") pod \"dns-default-t9nd7\" (UID: \"61652b90-d462-4c62-8c64-6e0843e2ebd5\") " pod="openshift-dns/dns-default-t9nd7" Jul 06 16:24:51.870047 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:51.870025 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0ab9e759-73b4-4f75-b1ab-276f94c7ec1e-cert\") pod \"ingress-canary-wj5s2\" (UID: \"0ab9e759-73b4-4f75-b1ab-276f94c7ec1e\") " pod="openshift-ingress-canary/ingress-canary-wj5s2" Jul 06 16:24:51.870047 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:51.870045 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4zcnp\" (UniqueName: \"kubernetes.io/projected/0ab9e759-73b4-4f75-b1ab-276f94c7ec1e-kube-api-access-4zcnp\") pod \"ingress-canary-wj5s2\" (UID: \"0ab9e759-73b4-4f75-b1ab-276f94c7ec1e\") " pod="openshift-ingress-canary/ingress-canary-wj5s2" Jul 06 16:24:51.870170 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:51.870066 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/61652b90-d462-4c62-8c64-6e0843e2ebd5-metrics-tls\") pod \"dns-default-t9nd7\" (UID: \"61652b90-d462-4c62-8c64-6e0843e2ebd5\") " pod="openshift-dns/dns-default-t9nd7" Jul 06 16:24:51.957312 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:51.957283 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:24:51.971350 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:51.971329 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dvs77\" (UniqueName: \"kubernetes.io/projected/61652b90-d462-4c62-8c64-6e0843e2ebd5-kube-api-access-dvs77\") pod \"dns-default-t9nd7\" (UID: \"61652b90-d462-4c62-8c64-6e0843e2ebd5\") " pod="openshift-dns/dns-default-t9nd7" Jul 06 16:24:51.971440 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:51.971357 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/ffbbbede-1d14-4088-853c-02b22a61e00a-crio-socket\") pod \"insights-runtime-extractor-pdz7v\" (UID: \"ffbbbede-1d14-4088-853c-02b22a61e00a\") " pod="openshift-insights/insights-runtime-extractor-pdz7v" Jul 06 16:24:51.971440 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:51.971378 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-975tr\" (UniqueName: \"kubernetes.io/projected/ffbbbede-1d14-4088-853c-02b22a61e00a-kube-api-access-975tr\") pod \"insights-runtime-extractor-pdz7v\" (UID: \"ffbbbede-1d14-4088-853c-02b22a61e00a\") " pod="openshift-insights/insights-runtime-extractor-pdz7v" Jul 06 16:24:51.971440 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:51.971412 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ffbbbede-1d14-4088-853c-02b22a61e00a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-pdz7v\" (UID: \"ffbbbede-1d14-4088-853c-02b22a61e00a\") " pod="openshift-insights/insights-runtime-extractor-pdz7v" Jul 06 16:24:51.971548 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:51.971498 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/61652b90-d462-4c62-8c64-6e0843e2ebd5-config-volume\") pod \"dns-default-t9nd7\" (UID: \"61652b90-d462-4c62-8c64-6e0843e2ebd5\") " pod="openshift-dns/dns-default-t9nd7" Jul 06 16:24:51.971548 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:51.971525 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/ffbbbede-1d14-4088-853c-02b22a61e00a-data-volume\") pod \"insights-runtime-extractor-pdz7v\" (UID: \"ffbbbede-1d14-4088-853c-02b22a61e00a\") " pod="openshift-insights/insights-runtime-extractor-pdz7v" Jul 06 16:24:51.971548 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:51.971544 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/61652b90-d462-4c62-8c64-6e0843e2ebd5-tmp-dir\") pod \"dns-default-t9nd7\" (UID: \"61652b90-d462-4c62-8c64-6e0843e2ebd5\") " pod="openshift-dns/dns-default-t9nd7" Jul 06 16:24:51.971632 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:51.971561 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0ab9e759-73b4-4f75-b1ab-276f94c7ec1e-cert\") pod \"ingress-canary-wj5s2\" (UID: \"0ab9e759-73b4-4f75-b1ab-276f94c7ec1e\") " pod="openshift-ingress-canary/ingress-canary-wj5s2" Jul 06 16:24:51.971693 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:51.971679 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4zcnp\" (UniqueName: \"kubernetes.io/projected/0ab9e759-73b4-4f75-b1ab-276f94c7ec1e-kube-api-access-4zcnp\") pod \"ingress-canary-wj5s2\" (UID: \"0ab9e759-73b4-4f75-b1ab-276f94c7ec1e\") " pod="openshift-ingress-canary/ingress-canary-wj5s2" Jul 06 16:24:51.971751 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:51.971703 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/61652b90-d462-4c62-8c64-6e0843e2ebd5-metrics-tls\") pod \"dns-default-t9nd7\" (UID: \"61652b90-d462-4c62-8c64-6e0843e2ebd5\") " pod="openshift-dns/dns-default-t9nd7" Jul 06 16:24:51.971791 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:51.971769 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ffbbbede-1d14-4088-853c-02b22a61e00a-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-pdz7v\" (UID: \"ffbbbede-1d14-4088-853c-02b22a61e00a\") " pod="openshift-insights/insights-runtime-extractor-pdz7v" Jul 06 16:24:51.972026 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:51.972008 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/61652b90-d462-4c62-8c64-6e0843e2ebd5-tmp-dir\") pod \"dns-default-t9nd7\" (UID: \"61652b90-d462-4c62-8c64-6e0843e2ebd5\") " pod="openshift-dns/dns-default-t9nd7" Jul 06 16:24:51.972204 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:51.972174 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/61652b90-d462-4c62-8c64-6e0843e2ebd5-config-volume\") pod \"dns-default-t9nd7\" (UID: \"61652b90-d462-4c62-8c64-6e0843e2ebd5\") " pod="openshift-dns/dns-default-t9nd7" Jul 06 16:24:51.975392 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:51.975364 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/61652b90-d462-4c62-8c64-6e0843e2ebd5-metrics-tls\") pod \"dns-default-t9nd7\" (UID: \"61652b90-d462-4c62-8c64-6e0843e2ebd5\") " pod="openshift-dns/dns-default-t9nd7" Jul 06 16:24:51.975501 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:51.975481 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0ab9e759-73b4-4f75-b1ab-276f94c7ec1e-cert\") pod \"ingress-canary-wj5s2\" (UID: \"0ab9e759-73b4-4f75-b1ab-276f94c7ec1e\") " pod="openshift-ingress-canary/ingress-canary-wj5s2" Jul 06 16:24:51.983227 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:51.983206 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4zcnp\" (UniqueName: \"kubernetes.io/projected/0ab9e759-73b4-4f75-b1ab-276f94c7ec1e-kube-api-access-4zcnp\") pod \"ingress-canary-wj5s2\" (UID: \"0ab9e759-73b4-4f75-b1ab-276f94c7ec1e\") " pod="openshift-ingress-canary/ingress-canary-wj5s2" Jul 06 16:24:51.983801 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:51.983783 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dvs77\" (UniqueName: \"kubernetes.io/projected/61652b90-d462-4c62-8c64-6e0843e2ebd5-kube-api-access-dvs77\") pod \"dns-default-t9nd7\" (UID: \"61652b90-d462-4c62-8c64-6e0843e2ebd5\") " pod="openshift-dns/dns-default-t9nd7" Jul 06 16:24:52.050967 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:52.050934 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-t9nd7" Jul 06 16:24:52.056638 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:52.056615 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:24:52.072910 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:52.072882 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ffbbbede-1d14-4088-853c-02b22a61e00a-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-pdz7v\" (UID: \"ffbbbede-1d14-4088-853c-02b22a61e00a\") " pod="openshift-insights/insights-runtime-extractor-pdz7v" Jul 06 16:24:52.073032 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:52.072916 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/ffbbbede-1d14-4088-853c-02b22a61e00a-crio-socket\") pod \"insights-runtime-extractor-pdz7v\" (UID: \"ffbbbede-1d14-4088-853c-02b22a61e00a\") " pod="openshift-insights/insights-runtime-extractor-pdz7v" Jul 06 16:24:52.073032 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:52.072932 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-975tr\" (UniqueName: \"kubernetes.io/projected/ffbbbede-1d14-4088-853c-02b22a61e00a-kube-api-access-975tr\") pod \"insights-runtime-extractor-pdz7v\" (UID: \"ffbbbede-1d14-4088-853c-02b22a61e00a\") " pod="openshift-insights/insights-runtime-extractor-pdz7v" Jul 06 16:24:52.073032 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:52.072959 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ffbbbede-1d14-4088-853c-02b22a61e00a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-pdz7v\" (UID: \"ffbbbede-1d14-4088-853c-02b22a61e00a\") " pod="openshift-insights/insights-runtime-extractor-pdz7v" Jul 06 16:24:52.073032 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:52.073006 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/ffbbbede-1d14-4088-853c-02b22a61e00a-data-volume\") pod \"insights-runtime-extractor-pdz7v\" (UID: \"ffbbbede-1d14-4088-853c-02b22a61e00a\") " pod="openshift-insights/insights-runtime-extractor-pdz7v" Jul 06 16:24:52.073233 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:52.073128 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/ffbbbede-1d14-4088-853c-02b22a61e00a-crio-socket\") pod \"insights-runtime-extractor-pdz7v\" (UID: \"ffbbbede-1d14-4088-853c-02b22a61e00a\") " pod="openshift-insights/insights-runtime-extractor-pdz7v" Jul 06 16:24:52.073288 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:52.073277 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/ffbbbede-1d14-4088-853c-02b22a61e00a-data-volume\") pod \"insights-runtime-extractor-pdz7v\" (UID: \"ffbbbede-1d14-4088-853c-02b22a61e00a\") " pod="openshift-insights/insights-runtime-extractor-pdz7v" Jul 06 16:24:52.073480 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:52.073463 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ffbbbede-1d14-4088-853c-02b22a61e00a-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-pdz7v\" (UID: \"ffbbbede-1d14-4088-853c-02b22a61e00a\") " pod="openshift-insights/insights-runtime-extractor-pdz7v" Jul 06 16:24:52.073540 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:52.073533 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-wj5s2" Jul 06 16:24:52.075294 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:52.075276 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ffbbbede-1d14-4088-853c-02b22a61e00a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-pdz7v\" (UID: \"ffbbbede-1d14-4088-853c-02b22a61e00a\") " pod="openshift-insights/insights-runtime-extractor-pdz7v" Jul 06 16:24:52.084107 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:52.084090 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-975tr\" (UniqueName: \"kubernetes.io/projected/ffbbbede-1d14-4088-853c-02b22a61e00a-kube-api-access-975tr\") pod \"insights-runtime-extractor-pdz7v\" (UID: \"ffbbbede-1d14-4088-853c-02b22a61e00a\") " pod="openshift-insights/insights-runtime-extractor-pdz7v" Jul 06 16:24:52.160264 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:52.160064 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-pdz7v" Jul 06 16:24:52.283581 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:52.283554 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-wj5s2"] Jul 06 16:24:52.293872 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:52.289441 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-t9nd7"] Jul 06 16:24:52.335879 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:52.335849 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-pdz7v"] Jul 06 16:24:52.338894 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:52.338871 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podffbbbede_1d14_4088_853c_02b22a61e00a.slice/crio-e40f68f3eea3beb62f65194e10aa119f214d77312a8646ff5ac884c7b626f3b2 WatchSource:0}: Error finding container e40f68f3eea3beb62f65194e10aa119f214d77312a8646ff5ac884c7b626f3b2: Status 404 returned error can't find the container with id e40f68f3eea3beb62f65194e10aa119f214d77312a8646ff5ac884c7b626f3b2 Jul 06 16:24:53.113999 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:53.113286 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wvcq5" Jul 06 16:24:53.113999 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:53.113712 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2zdgn" Jul 06 16:24:53.115153 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:53.113792 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-v8kjk" Jul 06 16:24:53.116804 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:53.116786 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 06 16:24:53.119712 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:53.119252 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 06 16:24:53.119712 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:53.119294 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 06 16:24:53.119712 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:53.119502 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 06 16:24:53.119712 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:53.119544 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-hn2l7\"" Jul 06 16:24:53.119712 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:53.119695 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-6db6n\"" Jul 06 16:24:53.301750 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:53.300986 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-wj5s2" event={"ID":"0ab9e759-73b4-4f75-b1ab-276f94c7ec1e","Type":"ContainerStarted","Data":"084aa5a620c3dd1b745022f9a681f738ed4adb41573358381dbd233f95a7c0dd"} Jul 06 16:24:53.303447 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:53.303377 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-t9nd7" event={"ID":"61652b90-d462-4c62-8c64-6e0843e2ebd5","Type":"ContainerStarted","Data":"a564ff5cd8e9498b0da8097b984e91aaa8bd56cb86351de7584206b16d3fda61"} Jul 06 16:24:53.305983 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:53.305949 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-pdz7v" event={"ID":"ffbbbede-1d14-4088-853c-02b22a61e00a","Type":"ContainerStarted","Data":"e40f68f3eea3beb62f65194e10aa119f214d77312a8646ff5ac884c7b626f3b2"} Jul 06 16:24:53.349187 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:53.349117 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:24:53.415563 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:53.415514 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:24:58.029692 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:58.029660 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:24:58.101035 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:58.101012 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-567f7b8b7c-hpfd9"] Jul 06 16:24:58.114278 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:58.114251 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-hpfd9" Jul 06 16:24:58.120469 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:58.120444 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\"" Jul 06 16:24:58.122144 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:58.121912 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\"" Jul 06 16:24:58.122144 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:58.121982 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 06 16:24:58.122144 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:58.122047 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 06 16:24:58.122144 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:58.121985 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-c8vd6\"" Jul 06 16:24:58.122144 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:58.121985 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 06 16:24:58.122993 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:58.122976 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-567f7b8b7c-hpfd9"] Jul 06 16:24:58.126981 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:58.126960 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:24:58.221395 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:58.221349 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/d96c7a27-3a8e-4050-affe-c9be14fcbfa4-prometheus-operator-tls\") pod \"prometheus-operator-567f7b8b7c-hpfd9\" (UID: \"d96c7a27-3a8e-4050-affe-c9be14fcbfa4\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-hpfd9" Jul 06 16:24:58.221556 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:58.221425 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d96c7a27-3a8e-4050-affe-c9be14fcbfa4-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-567f7b8b7c-hpfd9\" (UID: \"d96c7a27-3a8e-4050-affe-c9be14fcbfa4\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-hpfd9" Jul 06 16:24:58.221556 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:58.221482 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d96c7a27-3a8e-4050-affe-c9be14fcbfa4-metrics-client-ca\") pod \"prometheus-operator-567f7b8b7c-hpfd9\" (UID: \"d96c7a27-3a8e-4050-affe-c9be14fcbfa4\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-hpfd9" Jul 06 16:24:58.221625 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:58.221561 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vmv64\" (UniqueName: \"kubernetes.io/projected/d96c7a27-3a8e-4050-affe-c9be14fcbfa4-kube-api-access-vmv64\") pod \"prometheus-operator-567f7b8b7c-hpfd9\" (UID: \"d96c7a27-3a8e-4050-affe-c9be14fcbfa4\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-hpfd9" Jul 06 16:24:58.321341 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:58.321295 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-wj5s2" event={"ID":"0ab9e759-73b4-4f75-b1ab-276f94c7ec1e","Type":"ContainerStarted","Data":"30fb8250bf83c7f27017ebd6c9df6ef55c1de3697b11b4beb70288bb5a7722bc"} Jul 06 16:24:58.321982 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:58.321955 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/d96c7a27-3a8e-4050-affe-c9be14fcbfa4-prometheus-operator-tls\") pod \"prometheus-operator-567f7b8b7c-hpfd9\" (UID: \"d96c7a27-3a8e-4050-affe-c9be14fcbfa4\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-hpfd9" Jul 06 16:24:58.322095 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:58.321999 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d96c7a27-3a8e-4050-affe-c9be14fcbfa4-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-567f7b8b7c-hpfd9\" (UID: \"d96c7a27-3a8e-4050-affe-c9be14fcbfa4\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-hpfd9" Jul 06 16:24:58.322095 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:58.322060 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d96c7a27-3a8e-4050-affe-c9be14fcbfa4-metrics-client-ca\") pod \"prometheus-operator-567f7b8b7c-hpfd9\" (UID: \"d96c7a27-3a8e-4050-affe-c9be14fcbfa4\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-hpfd9" Jul 06 16:24:58.322206 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:58.322114 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vmv64\" (UniqueName: \"kubernetes.io/projected/d96c7a27-3a8e-4050-affe-c9be14fcbfa4-kube-api-access-vmv64\") pod \"prometheus-operator-567f7b8b7c-hpfd9\" (UID: \"d96c7a27-3a8e-4050-affe-c9be14fcbfa4\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-hpfd9" Jul 06 16:24:58.322206 ip-10-0-128-22 kubenswrapper[2584]: E0706 16:24:58.322164 2584 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-tls: secret "prometheus-operator-tls" not found Jul 06 16:24:58.322302 ip-10-0-128-22 kubenswrapper[2584]: E0706 16:24:58.322233 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d96c7a27-3a8e-4050-affe-c9be14fcbfa4-prometheus-operator-tls podName:d96c7a27-3a8e-4050-affe-c9be14fcbfa4 nodeName:}" failed. No retries permitted until 2026-07-06 16:24:58.822210895 +0000 UTC m=+49.436351011 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "prometheus-operator-tls" (UniqueName: "kubernetes.io/secret/d96c7a27-3a8e-4050-affe-c9be14fcbfa4-prometheus-operator-tls") pod "prometheus-operator-567f7b8b7c-hpfd9" (UID: "d96c7a27-3a8e-4050-affe-c9be14fcbfa4") : secret "prometheus-operator-tls" not found Jul 06 16:24:58.323078 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:58.323046 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d96c7a27-3a8e-4050-affe-c9be14fcbfa4-metrics-client-ca\") pod \"prometheus-operator-567f7b8b7c-hpfd9\" (UID: \"d96c7a27-3a8e-4050-affe-c9be14fcbfa4\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-hpfd9" Jul 06 16:24:58.323278 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:58.323247 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-t9nd7" event={"ID":"61652b90-d462-4c62-8c64-6e0843e2ebd5","Type":"ContainerStarted","Data":"068871f0b66f2d7303dfb86b406f6f66c3da6ac4f9d2326a19f24e9e907261e8"} Jul 06 16:24:58.327250 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:58.327224 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d96c7a27-3a8e-4050-affe-c9be14fcbfa4-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-567f7b8b7c-hpfd9\" (UID: \"d96c7a27-3a8e-4050-affe-c9be14fcbfa4\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-hpfd9" Jul 06 16:24:58.336294 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:58.336241 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vmv64\" (UniqueName: \"kubernetes.io/projected/d96c7a27-3a8e-4050-affe-c9be14fcbfa4-kube-api-access-vmv64\") pod \"prometheus-operator-567f7b8b7c-hpfd9\" (UID: \"d96c7a27-3a8e-4050-affe-c9be14fcbfa4\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-hpfd9" Jul 06 16:24:58.484592 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:58.484563 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:24:58.827130 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:58.827088 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/d96c7a27-3a8e-4050-affe-c9be14fcbfa4-prometheus-operator-tls\") pod \"prometheus-operator-567f7b8b7c-hpfd9\" (UID: \"d96c7a27-3a8e-4050-affe-c9be14fcbfa4\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-hpfd9" Jul 06 16:24:58.829692 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:58.829668 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/d96c7a27-3a8e-4050-affe-c9be14fcbfa4-prometheus-operator-tls\") pod \"prometheus-operator-567f7b8b7c-hpfd9\" (UID: \"d96c7a27-3a8e-4050-affe-c9be14fcbfa4\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-hpfd9" Jul 06 16:24:59.027983 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:59.027945 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-hpfd9" Jul 06 16:24:59.173398 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:59.173338 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-wj5s2" podStartSLOduration=2.439053719 podStartE2EDuration="8.173313363s" totalImagesPullingTime="5.734259644s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 16:24:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 16:24:58.356211869 +0000 UTC m=+48.970351990" watchObservedRunningTime="2026-07-06 16:24:59.173313363 +0000 UTC m=+49.787453486" Jul 06 16:24:59.173943 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:59.173867 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-567f7b8b7c-hpfd9"] Jul 06 16:24:59.176310 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:24:59.176274 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd96c7a27_3a8e_4050_affe_c9be14fcbfa4.slice/crio-9f7538343d46dbe56f340a17b03f72bee2e6938bfd3d002132c46e3699bfa41b WatchSource:0}: Error finding container 9f7538343d46dbe56f340a17b03f72bee2e6938bfd3d002132c46e3699bfa41b: Status 404 returned error can't find the container with id 9f7538343d46dbe56f340a17b03f72bee2e6938bfd3d002132c46e3699bfa41b Jul 06 16:24:59.327105 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:59.327070 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-t9nd7" event={"ID":"61652b90-d462-4c62-8c64-6e0843e2ebd5","Type":"ContainerStarted","Data":"a57956c146df17aec91dc391638dc7474763dff1b84972a9060160366e2436fb"} Jul 06 16:24:59.327294 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:59.327173 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-t9nd7" Jul 06 16:24:59.328591 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:59.328568 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-pdz7v" event={"ID":"ffbbbede-1d14-4088-853c-02b22a61e00a","Type":"ContainerStarted","Data":"c351ca44b7e616f92228626b0004c82eb1280e0106816eef445fe4e22e0160fc"} Jul 06 16:24:59.329706 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:59.329687 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-hpfd9" event={"ID":"d96c7a27-3a8e-4050-affe-c9be14fcbfa4","Type":"ContainerStarted","Data":"9f7538343d46dbe56f340a17b03f72bee2e6938bfd3d002132c46e3699bfa41b"} Jul 06 16:24:59.349890 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:24:59.349815 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-t9nd7" podStartSLOduration=2.61505969 podStartE2EDuration="8.349798618s" totalImagesPullingTime="5.734738928s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 16:24:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 16:24:59.348953604 +0000 UTC m=+49.963093724" watchObservedRunningTime="2026-07-06 16:24:59.349798618 +0000 UTC m=+49.963938739" Jul 06 16:25:00.338709 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:00.338627 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-pdz7v" event={"ID":"ffbbbede-1d14-4088-853c-02b22a61e00a","Type":"ContainerStarted","Data":"01de208d25fce5cd81623a337c0bbc66c2de7e6a74dc68b166659c68960254ba"} Jul 06 16:25:02.063413 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:02.063372 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:25:02.347040 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:02.346940 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-hpfd9" event={"ID":"d96c7a27-3a8e-4050-affe-c9be14fcbfa4","Type":"ContainerStarted","Data":"b2dc5263218679e43740a904b13833bb1085e630571175ec08c9d01d427e406b"} Jul 06 16:25:03.352059 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:03.351972 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-pdz7v" event={"ID":"ffbbbede-1d14-4088-853c-02b22a61e00a","Type":"ContainerStarted","Data":"99fcbd3e40b19c850a6466f7f3ab22f0049e812ddbe6b9316510aff63be2db9a"} Jul 06 16:25:03.353612 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:03.353590 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-hpfd9" event={"ID":"d96c7a27-3a8e-4050-affe-c9be14fcbfa4","Type":"ContainerStarted","Data":"c0dc9f26bba457039ad36cf07d0a87b4790daa03c249572179ac98b8e0b4ef93"} Jul 06 16:25:03.383086 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:03.383033 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-pdz7v" podStartSLOduration=1.93093508 podStartE2EDuration="12.383016194s" totalImagesPullingTime="10.452081114s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 16:24:51 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 16:25:03.380752822 +0000 UTC m=+53.994892945" watchObservedRunningTime="2026-07-06 16:25:03.383016194 +0000 UTC m=+53.997156315" Jul 06 16:25:03.405246 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:03.405188 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-hpfd9" podStartSLOduration=3.262616725 podStartE2EDuration="5.405169308s" totalImagesPullingTime="2.142552583s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 16:24:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 16:25:03.404271193 +0000 UTC m=+54.018411315" watchObservedRunningTime="2026-07-06 16:25:03.405169308 +0000 UTC m=+54.019309429" Jul 06 16:25:05.411517 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:05.411472 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:25:05.758531 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:05.758425 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:25:05.873648 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:05.873602 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:25:05.877889 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:05.877864 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-54cf96dfcf-vk9j9"] Jul 06 16:25:05.908128 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:05.908100 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-54cf96dfcf-vk9j9"] Jul 06 16:25:05.908276 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:05.908220 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-vk9j9" Jul 06 16:25:05.915423 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:05.915398 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\"" Jul 06 16:25:05.915747 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:05.915706 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-lsrvn\"" Jul 06 16:25:05.916648 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:05.916632 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\"" Jul 06 16:25:05.917490 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:05.917472 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-7c9dbb8b7d-z5c75"] Jul 06 16:25:05.938626 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:05.938595 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-z5c75" Jul 06 16:25:05.946382 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:05.946360 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\"" Jul 06 16:25:05.946495 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:05.946360 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\"" Jul 06 16:25:05.946795 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:05.946782 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\"" Jul 06 16:25:05.946844 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:05.946787 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-k9b6p\"" Jul 06 16:25:05.955883 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:05.955859 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-7c9dbb8b7d-z5c75"] Jul 06 16:25:05.969474 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:05.969449 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:25:05.979244 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:05.979220 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ed031c36-5db2-40a8-87e1-2d21d54e5c18-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-7c9dbb8b7d-z5c75\" (UID: \"ed031c36-5db2-40a8-87e1-2d21d54e5c18\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-z5c75" Jul 06 16:25:05.979376 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:05.979251 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/0fbab4dc-9098-4407-924a-20c35bbff2ee-openshift-state-metrics-tls\") pod \"openshift-state-metrics-54cf96dfcf-vk9j9\" (UID: \"0fbab4dc-9098-4407-924a-20c35bbff2ee\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-vk9j9" Jul 06 16:25:05.979376 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:05.979270 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ed031c36-5db2-40a8-87e1-2d21d54e5c18-metrics-client-ca\") pod \"kube-state-metrics-7c9dbb8b7d-z5c75\" (UID: \"ed031c36-5db2-40a8-87e1-2d21d54e5c18\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-z5c75" Jul 06 16:25:05.979376 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:05.979298 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0fbab4dc-9098-4407-924a-20c35bbff2ee-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-54cf96dfcf-vk9j9\" (UID: \"0fbab4dc-9098-4407-924a-20c35bbff2ee\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-vk9j9" Jul 06 16:25:05.979376 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:05.979316 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5rchs\" (UniqueName: \"kubernetes.io/projected/0fbab4dc-9098-4407-924a-20c35bbff2ee-kube-api-access-5rchs\") pod \"openshift-state-metrics-54cf96dfcf-vk9j9\" (UID: \"0fbab4dc-9098-4407-924a-20c35bbff2ee\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-vk9j9" Jul 06 16:25:05.979376 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:05.979332 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bvcm4\" (UniqueName: \"kubernetes.io/projected/ed031c36-5db2-40a8-87e1-2d21d54e5c18-kube-api-access-bvcm4\") pod \"kube-state-metrics-7c9dbb8b7d-z5c75\" (UID: \"ed031c36-5db2-40a8-87e1-2d21d54e5c18\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-z5c75" Jul 06 16:25:05.979536 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:05.979386 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/ed031c36-5db2-40a8-87e1-2d21d54e5c18-volume-directive-shadow\") pod \"kube-state-metrics-7c9dbb8b7d-z5c75\" (UID: \"ed031c36-5db2-40a8-87e1-2d21d54e5c18\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-z5c75" Jul 06 16:25:05.979536 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:05.979453 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/ed031c36-5db2-40a8-87e1-2d21d54e5c18-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-7c9dbb8b7d-z5c75\" (UID: \"ed031c36-5db2-40a8-87e1-2d21d54e5c18\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-z5c75" Jul 06 16:25:05.979536 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:05.979481 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0fbab4dc-9098-4407-924a-20c35bbff2ee-metrics-client-ca\") pod \"openshift-state-metrics-54cf96dfcf-vk9j9\" (UID: \"0fbab4dc-9098-4407-924a-20c35bbff2ee\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-vk9j9" Jul 06 16:25:05.979536 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:05.979507 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/ed031c36-5db2-40a8-87e1-2d21d54e5c18-kube-state-metrics-tls\") pod \"kube-state-metrics-7c9dbb8b7d-z5c75\" (UID: \"ed031c36-5db2-40a8-87e1-2d21d54e5c18\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-z5c75" Jul 06 16:25:06.045967 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:06.045881 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-7pwll"] Jul 06 16:25:06.069024 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:06.068992 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-7pwll" Jul 06 16:25:06.073745 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:06.073706 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 06 16:25:06.074021 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:06.074007 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 06 16:25:06.075048 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:06.075027 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-hnwqs\"" Jul 06 16:25:06.077500 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:06.077483 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 06 16:25:06.079793 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:06.079774 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/facad3ae-3b4c-4812-93b3-b398a5811e12-node-exporter-textfile\") pod \"node-exporter-7pwll\" (UID: \"facad3ae-3b4c-4812-93b3-b398a5811e12\") " pod="openshift-monitoring/node-exporter-7pwll" Jul 06 16:25:06.079889 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:06.079807 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wchzx\" (UniqueName: \"kubernetes.io/projected/facad3ae-3b4c-4812-93b3-b398a5811e12-kube-api-access-wchzx\") pod \"node-exporter-7pwll\" (UID: \"facad3ae-3b4c-4812-93b3-b398a5811e12\") " pod="openshift-monitoring/node-exporter-7pwll" Jul 06 16:25:06.079889 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:06.079832 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/facad3ae-3b4c-4812-93b3-b398a5811e12-metrics-client-ca\") pod \"node-exporter-7pwll\" (UID: \"facad3ae-3b4c-4812-93b3-b398a5811e12\") " pod="openshift-monitoring/node-exporter-7pwll" Jul 06 16:25:06.079889 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:06.079874 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/facad3ae-3b4c-4812-93b3-b398a5811e12-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-7pwll\" (UID: \"facad3ae-3b4c-4812-93b3-b398a5811e12\") " pod="openshift-monitoring/node-exporter-7pwll" Jul 06 16:25:06.080040 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:06.079910 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/facad3ae-3b4c-4812-93b3-b398a5811e12-node-exporter-tls\") pod \"node-exporter-7pwll\" (UID: \"facad3ae-3b4c-4812-93b3-b398a5811e12\") " pod="openshift-monitoring/node-exporter-7pwll" Jul 06 16:25:06.080040 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:06.079946 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/facad3ae-3b4c-4812-93b3-b398a5811e12-root\") pod \"node-exporter-7pwll\" (UID: \"facad3ae-3b4c-4812-93b3-b398a5811e12\") " pod="openshift-monitoring/node-exporter-7pwll" Jul 06 16:25:06.080040 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:06.079994 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/ed031c36-5db2-40a8-87e1-2d21d54e5c18-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-7c9dbb8b7d-z5c75\" (UID: \"ed031c36-5db2-40a8-87e1-2d21d54e5c18\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-z5c75" Jul 06 16:25:06.080040 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:06.080029 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0fbab4dc-9098-4407-924a-20c35bbff2ee-metrics-client-ca\") pod \"openshift-state-metrics-54cf96dfcf-vk9j9\" (UID: \"0fbab4dc-9098-4407-924a-20c35bbff2ee\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-vk9j9" Jul 06 16:25:06.080188 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:06.080054 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/ed031c36-5db2-40a8-87e1-2d21d54e5c18-kube-state-metrics-tls\") pod \"kube-state-metrics-7c9dbb8b7d-z5c75\" (UID: \"ed031c36-5db2-40a8-87e1-2d21d54e5c18\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-z5c75" Jul 06 16:25:06.080188 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:06.080100 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ed031c36-5db2-40a8-87e1-2d21d54e5c18-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-7c9dbb8b7d-z5c75\" (UID: \"ed031c36-5db2-40a8-87e1-2d21d54e5c18\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-z5c75" Jul 06 16:25:06.080188 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:06.080122 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/0fbab4dc-9098-4407-924a-20c35bbff2ee-openshift-state-metrics-tls\") pod \"openshift-state-metrics-54cf96dfcf-vk9j9\" (UID: \"0fbab4dc-9098-4407-924a-20c35bbff2ee\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-vk9j9" Jul 06 16:25:06.080188 ip-10-0-128-22 kubenswrapper[2584]: E0706 16:25:06.080131 2584 secret.go:189] Couldn't get secret openshift-monitoring/kube-state-metrics-tls: secret "kube-state-metrics-tls" not found Jul 06 16:25:06.080372 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:06.080183 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/facad3ae-3b4c-4812-93b3-b398a5811e12-node-exporter-wtmp\") pod \"node-exporter-7pwll\" (UID: \"facad3ae-3b4c-4812-93b3-b398a5811e12\") " pod="openshift-monitoring/node-exporter-7pwll" Jul 06 16:25:06.080372 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:06.080220 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0fbab4dc-9098-4407-924a-20c35bbff2ee-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-54cf96dfcf-vk9j9\" (UID: \"0fbab4dc-9098-4407-924a-20c35bbff2ee\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-vk9j9" Jul 06 16:25:06.080372 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:06.080255 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5rchs\" (UniqueName: \"kubernetes.io/projected/0fbab4dc-9098-4407-924a-20c35bbff2ee-kube-api-access-5rchs\") pod \"openshift-state-metrics-54cf96dfcf-vk9j9\" (UID: \"0fbab4dc-9098-4407-924a-20c35bbff2ee\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-vk9j9" Jul 06 16:25:06.080372 ip-10-0-128-22 kubenswrapper[2584]: E0706 16:25:06.080286 2584 secret.go:189] Couldn't get secret openshift-monitoring/openshift-state-metrics-tls: secret "openshift-state-metrics-tls" not found Jul 06 16:25:06.080372 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:06.080322 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/ed031c36-5db2-40a8-87e1-2d21d54e5c18-volume-directive-shadow\") pod \"kube-state-metrics-7c9dbb8b7d-z5c75\" (UID: \"ed031c36-5db2-40a8-87e1-2d21d54e5c18\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-z5c75" Jul 06 16:25:06.080372 ip-10-0-128-22 kubenswrapper[2584]: E0706 16:25:06.080356 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0fbab4dc-9098-4407-924a-20c35bbff2ee-openshift-state-metrics-tls podName:0fbab4dc-9098-4407-924a-20c35bbff2ee nodeName:}" failed. No retries permitted until 2026-07-06 16:25:06.580336666 +0000 UTC m=+57.194476782 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "openshift-state-metrics-tls" (UniqueName: "kubernetes.io/secret/0fbab4dc-9098-4407-924a-20c35bbff2ee-openshift-state-metrics-tls") pod "openshift-state-metrics-54cf96dfcf-vk9j9" (UID: "0fbab4dc-9098-4407-924a-20c35bbff2ee") : secret "openshift-state-metrics-tls" not found Jul 06 16:25:06.080664 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:06.080427 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/facad3ae-3b4c-4812-93b3-b398a5811e12-node-exporter-accelerators-collector-config\") pod \"node-exporter-7pwll\" (UID: \"facad3ae-3b4c-4812-93b3-b398a5811e12\") " pod="openshift-monitoring/node-exporter-7pwll" Jul 06 16:25:06.080664 ip-10-0-128-22 kubenswrapper[2584]: E0706 16:25:06.080439 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ed031c36-5db2-40a8-87e1-2d21d54e5c18-kube-state-metrics-tls podName:ed031c36-5db2-40a8-87e1-2d21d54e5c18 nodeName:}" failed. No retries permitted until 2026-07-06 16:25:06.580417921 +0000 UTC m=+57.194558025 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-state-metrics-tls" (UniqueName: "kubernetes.io/secret/ed031c36-5db2-40a8-87e1-2d21d54e5c18-kube-state-metrics-tls") pod "kube-state-metrics-7c9dbb8b7d-z5c75" (UID: "ed031c36-5db2-40a8-87e1-2d21d54e5c18") : secret "kube-state-metrics-tls" not found Jul 06 16:25:06.080664 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:06.080490 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ed031c36-5db2-40a8-87e1-2d21d54e5c18-metrics-client-ca\") pod \"kube-state-metrics-7c9dbb8b7d-z5c75\" (UID: \"ed031c36-5db2-40a8-87e1-2d21d54e5c18\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-z5c75" Jul 06 16:25:06.080664 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:06.080520 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bvcm4\" (UniqueName: \"kubernetes.io/projected/ed031c36-5db2-40a8-87e1-2d21d54e5c18-kube-api-access-bvcm4\") pod \"kube-state-metrics-7c9dbb8b7d-z5c75\" (UID: \"ed031c36-5db2-40a8-87e1-2d21d54e5c18\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-z5c75" Jul 06 16:25:06.080664 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:06.080553 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/facad3ae-3b4c-4812-93b3-b398a5811e12-sys\") pod \"node-exporter-7pwll\" (UID: \"facad3ae-3b4c-4812-93b3-b398a5811e12\") " pod="openshift-monitoring/node-exporter-7pwll" Jul 06 16:25:06.080932 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:06.080666 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/ed031c36-5db2-40a8-87e1-2d21d54e5c18-volume-directive-shadow\") pod \"kube-state-metrics-7c9dbb8b7d-z5c75\" (UID: \"ed031c36-5db2-40a8-87e1-2d21d54e5c18\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-z5c75" Jul 06 16:25:06.080932 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:06.080787 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/ed031c36-5db2-40a8-87e1-2d21d54e5c18-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-7c9dbb8b7d-z5c75\" (UID: \"ed031c36-5db2-40a8-87e1-2d21d54e5c18\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-z5c75" Jul 06 16:25:06.081004 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:06.080933 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0fbab4dc-9098-4407-924a-20c35bbff2ee-metrics-client-ca\") pod \"openshift-state-metrics-54cf96dfcf-vk9j9\" (UID: \"0fbab4dc-9098-4407-924a-20c35bbff2ee\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-vk9j9" Jul 06 16:25:06.081157 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:06.081141 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ed031c36-5db2-40a8-87e1-2d21d54e5c18-metrics-client-ca\") pod \"kube-state-metrics-7c9dbb8b7d-z5c75\" (UID: \"ed031c36-5db2-40a8-87e1-2d21d54e5c18\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-z5c75" Jul 06 16:25:06.083021 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:06.083001 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0fbab4dc-9098-4407-924a-20c35bbff2ee-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-54cf96dfcf-vk9j9\" (UID: \"0fbab4dc-9098-4407-924a-20c35bbff2ee\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-vk9j9" Jul 06 16:25:06.083103 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:06.083089 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ed031c36-5db2-40a8-87e1-2d21d54e5c18-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-7c9dbb8b7d-z5c75\" (UID: \"ed031c36-5db2-40a8-87e1-2d21d54e5c18\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-z5c75" Jul 06 16:25:06.140526 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:06.140494 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bvcm4\" (UniqueName: \"kubernetes.io/projected/ed031c36-5db2-40a8-87e1-2d21d54e5c18-kube-api-access-bvcm4\") pod \"kube-state-metrics-7c9dbb8b7d-z5c75\" (UID: \"ed031c36-5db2-40a8-87e1-2d21d54e5c18\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-z5c75" Jul 06 16:25:06.160847 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:06.160806 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:25:06.168586 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:06.168556 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5rchs\" (UniqueName: \"kubernetes.io/projected/0fbab4dc-9098-4407-924a-20c35bbff2ee-kube-api-access-5rchs\") pod \"openshift-state-metrics-54cf96dfcf-vk9j9\" (UID: \"0fbab4dc-9098-4407-924a-20c35bbff2ee\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-vk9j9" Jul 06 16:25:06.181657 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:06.181622 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/facad3ae-3b4c-4812-93b3-b398a5811e12-node-exporter-accelerators-collector-config\") pod \"node-exporter-7pwll\" (UID: \"facad3ae-3b4c-4812-93b3-b398a5811e12\") " pod="openshift-monitoring/node-exporter-7pwll" Jul 06 16:25:06.181657 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:06.181659 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/facad3ae-3b4c-4812-93b3-b398a5811e12-sys\") pod \"node-exporter-7pwll\" (UID: \"facad3ae-3b4c-4812-93b3-b398a5811e12\") " pod="openshift-monitoring/node-exporter-7pwll" Jul 06 16:25:06.181912 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:06.181683 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/facad3ae-3b4c-4812-93b3-b398a5811e12-node-exporter-textfile\") pod \"node-exporter-7pwll\" (UID: \"facad3ae-3b4c-4812-93b3-b398a5811e12\") " pod="openshift-monitoring/node-exporter-7pwll" Jul 06 16:25:06.181912 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:06.181701 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wchzx\" (UniqueName: \"kubernetes.io/projected/facad3ae-3b4c-4812-93b3-b398a5811e12-kube-api-access-wchzx\") pod \"node-exporter-7pwll\" (UID: \"facad3ae-3b4c-4812-93b3-b398a5811e12\") " pod="openshift-monitoring/node-exporter-7pwll" Jul 06 16:25:06.181912 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:06.181740 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/facad3ae-3b4c-4812-93b3-b398a5811e12-metrics-client-ca\") pod \"node-exporter-7pwll\" (UID: \"facad3ae-3b4c-4812-93b3-b398a5811e12\") " pod="openshift-monitoring/node-exporter-7pwll" Jul 06 16:25:06.181912 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:06.181762 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/facad3ae-3b4c-4812-93b3-b398a5811e12-sys\") pod \"node-exporter-7pwll\" (UID: \"facad3ae-3b4c-4812-93b3-b398a5811e12\") " pod="openshift-monitoring/node-exporter-7pwll" Jul 06 16:25:06.181912 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:06.181867 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/facad3ae-3b4c-4812-93b3-b398a5811e12-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-7pwll\" (UID: \"facad3ae-3b4c-4812-93b3-b398a5811e12\") " pod="openshift-monitoring/node-exporter-7pwll" Jul 06 16:25:06.181912 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:06.181899 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/facad3ae-3b4c-4812-93b3-b398a5811e12-node-exporter-tls\") pod \"node-exporter-7pwll\" (UID: \"facad3ae-3b4c-4812-93b3-b398a5811e12\") " pod="openshift-monitoring/node-exporter-7pwll" Jul 06 16:25:06.181912 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:06.181918 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/facad3ae-3b4c-4812-93b3-b398a5811e12-root\") pod \"node-exporter-7pwll\" (UID: \"facad3ae-3b4c-4812-93b3-b398a5811e12\") " pod="openshift-monitoring/node-exporter-7pwll" Jul 06 16:25:06.182296 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:06.181983 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/facad3ae-3b4c-4812-93b3-b398a5811e12-root\") pod \"node-exporter-7pwll\" (UID: \"facad3ae-3b4c-4812-93b3-b398a5811e12\") " pod="openshift-monitoring/node-exporter-7pwll" Jul 06 16:25:06.182296 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:06.182025 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/facad3ae-3b4c-4812-93b3-b398a5811e12-node-exporter-textfile\") pod \"node-exporter-7pwll\" (UID: \"facad3ae-3b4c-4812-93b3-b398a5811e12\") " pod="openshift-monitoring/node-exporter-7pwll" Jul 06 16:25:06.182296 ip-10-0-128-22 kubenswrapper[2584]: E0706 16:25:06.182036 2584 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Jul 06 16:25:06.182296 ip-10-0-128-22 kubenswrapper[2584]: E0706 16:25:06.182110 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/facad3ae-3b4c-4812-93b3-b398a5811e12-node-exporter-tls podName:facad3ae-3b4c-4812-93b3-b398a5811e12 nodeName:}" failed. No retries permitted until 2026-07-06 16:25:06.68208791 +0000 UTC m=+57.296228015 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/facad3ae-3b4c-4812-93b3-b398a5811e12-node-exporter-tls") pod "node-exporter-7pwll" (UID: "facad3ae-3b4c-4812-93b3-b398a5811e12") : secret "node-exporter-tls" not found Jul 06 16:25:06.182296 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:06.182163 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/facad3ae-3b4c-4812-93b3-b398a5811e12-node-exporter-wtmp\") pod \"node-exporter-7pwll\" (UID: \"facad3ae-3b4c-4812-93b3-b398a5811e12\") " pod="openshift-monitoring/node-exporter-7pwll" Jul 06 16:25:06.182494 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:06.182313 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/facad3ae-3b4c-4812-93b3-b398a5811e12-metrics-client-ca\") pod \"node-exporter-7pwll\" (UID: \"facad3ae-3b4c-4812-93b3-b398a5811e12\") " pod="openshift-monitoring/node-exporter-7pwll" Jul 06 16:25:06.182494 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:06.182318 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/facad3ae-3b4c-4812-93b3-b398a5811e12-node-exporter-wtmp\") pod \"node-exporter-7pwll\" (UID: \"facad3ae-3b4c-4812-93b3-b398a5811e12\") " pod="openshift-monitoring/node-exporter-7pwll" Jul 06 16:25:06.182494 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:06.182352 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/facad3ae-3b4c-4812-93b3-b398a5811e12-node-exporter-accelerators-collector-config\") pod \"node-exporter-7pwll\" (UID: \"facad3ae-3b4c-4812-93b3-b398a5811e12\") " pod="openshift-monitoring/node-exporter-7pwll" Jul 06 16:25:06.184089 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:06.184072 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/facad3ae-3b4c-4812-93b3-b398a5811e12-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-7pwll\" (UID: \"facad3ae-3b4c-4812-93b3-b398a5811e12\") " pod="openshift-monitoring/node-exporter-7pwll" Jul 06 16:25:06.275768 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:06.275713 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wchzx\" (UniqueName: \"kubernetes.io/projected/facad3ae-3b4c-4812-93b3-b398a5811e12-kube-api-access-wchzx\") pod \"node-exporter-7pwll\" (UID: \"facad3ae-3b4c-4812-93b3-b398a5811e12\") " pod="openshift-monitoring/node-exporter-7pwll" Jul 06 16:25:06.453054 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:06.453016 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:25:06.544551 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:06.544510 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:25:06.585108 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:06.585074 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/ed031c36-5db2-40a8-87e1-2d21d54e5c18-kube-state-metrics-tls\") pod \"kube-state-metrics-7c9dbb8b7d-z5c75\" (UID: \"ed031c36-5db2-40a8-87e1-2d21d54e5c18\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-z5c75" Jul 06 16:25:06.585108 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:06.585113 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/0fbab4dc-9098-4407-924a-20c35bbff2ee-openshift-state-metrics-tls\") pod \"openshift-state-metrics-54cf96dfcf-vk9j9\" (UID: \"0fbab4dc-9098-4407-924a-20c35bbff2ee\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-vk9j9" Jul 06 16:25:06.587493 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:06.587468 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/ed031c36-5db2-40a8-87e1-2d21d54e5c18-kube-state-metrics-tls\") pod \"kube-state-metrics-7c9dbb8b7d-z5c75\" (UID: \"ed031c36-5db2-40a8-87e1-2d21d54e5c18\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-z5c75" Jul 06 16:25:06.587493 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:06.587488 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/0fbab4dc-9098-4407-924a-20c35bbff2ee-openshift-state-metrics-tls\") pod \"openshift-state-metrics-54cf96dfcf-vk9j9\" (UID: \"0fbab4dc-9098-4407-924a-20c35bbff2ee\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-vk9j9" Jul 06 16:25:06.686088 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:06.686051 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/facad3ae-3b4c-4812-93b3-b398a5811e12-node-exporter-tls\") pod \"node-exporter-7pwll\" (UID: \"facad3ae-3b4c-4812-93b3-b398a5811e12\") " pod="openshift-monitoring/node-exporter-7pwll" Jul 06 16:25:06.688316 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:06.688284 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/facad3ae-3b4c-4812-93b3-b398a5811e12-node-exporter-tls\") pod \"node-exporter-7pwll\" (UID: \"facad3ae-3b4c-4812-93b3-b398a5811e12\") " pod="openshift-monitoring/node-exporter-7pwll" Jul 06 16:25:06.808978 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:06.808873 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:25:06.816764 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:06.816734 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-vk9j9" Jul 06 16:25:06.847572 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:06.847537 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-z5c75" Jul 06 16:25:06.960756 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:06.960629 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-54cf96dfcf-vk9j9"] Jul 06 16:25:06.965272 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:06.964390 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:25:06.968013 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:25:06.967989 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0fbab4dc_9098_4407_924a_20c35bbff2ee.slice/crio-246a79fc660371891ced34071ae3d8dae51a58befe543f7b5563ad33a4c0cab5 WatchSource:0}: Error finding container 246a79fc660371891ced34071ae3d8dae51a58befe543f7b5563ad33a4c0cab5: Status 404 returned error can't find the container with id 246a79fc660371891ced34071ae3d8dae51a58befe543f7b5563ad33a4c0cab5 Jul 06 16:25:06.977928 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:06.977901 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-7pwll" Jul 06 16:25:07.005921 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:25:07.005894 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfacad3ae_3b4c_4812_93b3_b398a5811e12.slice/crio-d559084a23eb1df8bdf988ba19e0de0d6644169e6652ee446ac78218afd1a621 WatchSource:0}: Error finding container d559084a23eb1df8bdf988ba19e0de0d6644169e6652ee446ac78218afd1a621: Status 404 returned error can't find the container with id d559084a23eb1df8bdf988ba19e0de0d6644169e6652ee446ac78218afd1a621 Jul 06 16:25:07.014343 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:07.014321 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-7c9dbb8b7d-z5c75"] Jul 06 16:25:07.017236 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:25:07.017218 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poded031c36_5db2_40a8_87e1_2d21d54e5c18.slice/crio-60292e50d5af157aeabd93e5867e86bafa70a39a2040ae20d9c963b03866c38d WatchSource:0}: Error finding container 60292e50d5af157aeabd93e5867e86bafa70a39a2040ae20d9c963b03866c38d: Status 404 returned error can't find the container with id 60292e50d5af157aeabd93e5867e86bafa70a39a2040ae20d9c963b03866c38d Jul 06 16:25:07.049040 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:07.049013 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 06 16:25:07.051993 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:07.051971 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 06 16:25:07.055596 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:07.055575 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jul 06 16:25:07.056149 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:07.056126 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jul 06 16:25:07.056248 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:07.056230 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jul 06 16:25:07.056294 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:07.056234 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-s56wg\"" Jul 06 16:25:07.056294 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:07.056281 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jul 06 16:25:07.056388 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:07.056357 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jul 06 16:25:07.057139 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:07.056601 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jul 06 16:25:07.057139 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:07.056716 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jul 06 16:25:07.057139 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:07.056980 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jul 06 16:25:07.057430 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:07.057326 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jul 06 16:25:07.087822 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:07.087796 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 06 16:25:07.088988 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:07.088957 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qjv9x\" (UniqueName: \"kubernetes.io/projected/1e2e6141-69a0-4861-a82c-fd3377cf7752-kube-api-access-qjv9x\") pod \"alertmanager-main-0\" (UID: \"1e2e6141-69a0-4861-a82c-fd3377cf7752\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 16:25:07.089119 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:07.088998 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/1e2e6141-69a0-4861-a82c-fd3377cf7752-config-out\") pod \"alertmanager-main-0\" (UID: \"1e2e6141-69a0-4861-a82c-fd3377cf7752\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 16:25:07.089119 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:07.089082 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1e2e6141-69a0-4861-a82c-fd3377cf7752-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"1e2e6141-69a0-4861-a82c-fd3377cf7752\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 16:25:07.089221 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:07.089128 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/1e2e6141-69a0-4861-a82c-fd3377cf7752-web-config\") pod \"alertmanager-main-0\" (UID: \"1e2e6141-69a0-4861-a82c-fd3377cf7752\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 16:25:07.089221 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:07.089176 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/1e2e6141-69a0-4861-a82c-fd3377cf7752-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"1e2e6141-69a0-4861-a82c-fd3377cf7752\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 16:25:07.089221 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:07.089204 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/1e2e6141-69a0-4861-a82c-fd3377cf7752-config-volume\") pod \"alertmanager-main-0\" (UID: \"1e2e6141-69a0-4861-a82c-fd3377cf7752\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 16:25:07.089359 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:07.089236 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/1e2e6141-69a0-4861-a82c-fd3377cf7752-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"1e2e6141-69a0-4861-a82c-fd3377cf7752\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 16:25:07.089359 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:07.089271 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/1e2e6141-69a0-4861-a82c-fd3377cf7752-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"1e2e6141-69a0-4861-a82c-fd3377cf7752\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 16:25:07.089359 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:07.089303 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/1e2e6141-69a0-4861-a82c-fd3377cf7752-tls-assets\") pod \"alertmanager-main-0\" (UID: \"1e2e6141-69a0-4861-a82c-fd3377cf7752\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 16:25:07.089359 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:07.089355 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1e2e6141-69a0-4861-a82c-fd3377cf7752-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"1e2e6141-69a0-4861-a82c-fd3377cf7752\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 16:25:07.089522 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:07.089384 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/1e2e6141-69a0-4861-a82c-fd3377cf7752-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"1e2e6141-69a0-4861-a82c-fd3377cf7752\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 16:25:07.089522 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:07.089416 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/1e2e6141-69a0-4861-a82c-fd3377cf7752-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"1e2e6141-69a0-4861-a82c-fd3377cf7752\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 16:25:07.089522 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:07.089443 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/1e2e6141-69a0-4861-a82c-fd3377cf7752-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"1e2e6141-69a0-4861-a82c-fd3377cf7752\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 16:25:07.105879 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:07.105855 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:25:07.190662 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:07.190629 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/1e2e6141-69a0-4861-a82c-fd3377cf7752-tls-assets\") pod \"alertmanager-main-0\" (UID: \"1e2e6141-69a0-4861-a82c-fd3377cf7752\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 16:25:07.190662 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:07.190667 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1e2e6141-69a0-4861-a82c-fd3377cf7752-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"1e2e6141-69a0-4861-a82c-fd3377cf7752\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 16:25:07.190950 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:07.190686 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/1e2e6141-69a0-4861-a82c-fd3377cf7752-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"1e2e6141-69a0-4861-a82c-fd3377cf7752\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 16:25:07.190950 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:07.190719 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/1e2e6141-69a0-4861-a82c-fd3377cf7752-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"1e2e6141-69a0-4861-a82c-fd3377cf7752\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 16:25:07.190950 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:07.190765 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/1e2e6141-69a0-4861-a82c-fd3377cf7752-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"1e2e6141-69a0-4861-a82c-fd3377cf7752\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 16:25:07.190950 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:07.190859 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qjv9x\" (UniqueName: \"kubernetes.io/projected/1e2e6141-69a0-4861-a82c-fd3377cf7752-kube-api-access-qjv9x\") pod \"alertmanager-main-0\" (UID: \"1e2e6141-69a0-4861-a82c-fd3377cf7752\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 16:25:07.190950 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:07.190900 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/1e2e6141-69a0-4861-a82c-fd3377cf7752-config-out\") pod \"alertmanager-main-0\" (UID: \"1e2e6141-69a0-4861-a82c-fd3377cf7752\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 16:25:07.191219 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:07.190970 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1e2e6141-69a0-4861-a82c-fd3377cf7752-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"1e2e6141-69a0-4861-a82c-fd3377cf7752\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 16:25:07.191219 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:07.191019 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/1e2e6141-69a0-4861-a82c-fd3377cf7752-web-config\") pod \"alertmanager-main-0\" (UID: \"1e2e6141-69a0-4861-a82c-fd3377cf7752\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 16:25:07.191219 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:07.191074 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/1e2e6141-69a0-4861-a82c-fd3377cf7752-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"1e2e6141-69a0-4861-a82c-fd3377cf7752\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 16:25:07.191219 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:07.191106 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/1e2e6141-69a0-4861-a82c-fd3377cf7752-config-volume\") pod \"alertmanager-main-0\" (UID: \"1e2e6141-69a0-4861-a82c-fd3377cf7752\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 16:25:07.191219 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:07.191142 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/1e2e6141-69a0-4861-a82c-fd3377cf7752-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"1e2e6141-69a0-4861-a82c-fd3377cf7752\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 16:25:07.191219 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:07.191179 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/1e2e6141-69a0-4861-a82c-fd3377cf7752-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"1e2e6141-69a0-4861-a82c-fd3377cf7752\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 16:25:07.191529 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:07.191179 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/1e2e6141-69a0-4861-a82c-fd3377cf7752-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"1e2e6141-69a0-4861-a82c-fd3377cf7752\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 16:25:07.191811 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:07.191786 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1e2e6141-69a0-4861-a82c-fd3377cf7752-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"1e2e6141-69a0-4861-a82c-fd3377cf7752\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 16:25:07.192410 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:07.192355 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1e2e6141-69a0-4861-a82c-fd3377cf7752-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"1e2e6141-69a0-4861-a82c-fd3377cf7752\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 16:25:07.194047 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:07.194023 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/1e2e6141-69a0-4861-a82c-fd3377cf7752-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"1e2e6141-69a0-4861-a82c-fd3377cf7752\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 16:25:07.194344 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:07.194304 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/1e2e6141-69a0-4861-a82c-fd3377cf7752-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"1e2e6141-69a0-4861-a82c-fd3377cf7752\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 16:25:07.194796 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:07.194753 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/1e2e6141-69a0-4861-a82c-fd3377cf7752-config-volume\") pod \"alertmanager-main-0\" (UID: \"1e2e6141-69a0-4861-a82c-fd3377cf7752\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 16:25:07.195054 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:07.195036 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/1e2e6141-69a0-4861-a82c-fd3377cf7752-web-config\") pod \"alertmanager-main-0\" (UID: \"1e2e6141-69a0-4861-a82c-fd3377cf7752\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 16:25:07.195251 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:07.195233 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/1e2e6141-69a0-4861-a82c-fd3377cf7752-config-out\") pod \"alertmanager-main-0\" (UID: \"1e2e6141-69a0-4861-a82c-fd3377cf7752\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 16:25:07.195540 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:07.195517 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/1e2e6141-69a0-4861-a82c-fd3377cf7752-tls-assets\") pod \"alertmanager-main-0\" (UID: \"1e2e6141-69a0-4861-a82c-fd3377cf7752\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 16:25:07.195600 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:07.195521 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/1e2e6141-69a0-4861-a82c-fd3377cf7752-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"1e2e6141-69a0-4861-a82c-fd3377cf7752\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 16:25:07.195600 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:07.195584 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/1e2e6141-69a0-4861-a82c-fd3377cf7752-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"1e2e6141-69a0-4861-a82c-fd3377cf7752\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 16:25:07.195600 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:07.195587 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/1e2e6141-69a0-4861-a82c-fd3377cf7752-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"1e2e6141-69a0-4861-a82c-fd3377cf7752\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 16:25:07.213161 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:07.213135 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qjv9x\" (UniqueName: \"kubernetes.io/projected/1e2e6141-69a0-4861-a82c-fd3377cf7752-kube-api-access-qjv9x\") pod \"alertmanager-main-0\" (UID: \"1e2e6141-69a0-4861-a82c-fd3377cf7752\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 16:25:07.231910 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:07.231877 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:25:07.366473 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:07.366385 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-7pwll" event={"ID":"facad3ae-3b4c-4812-93b3-b398a5811e12","Type":"ContainerStarted","Data":"d559084a23eb1df8bdf988ba19e0de0d6644169e6652ee446ac78218afd1a621"} Jul 06 16:25:07.367409 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:07.367381 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-z5c75" event={"ID":"ed031c36-5db2-40a8-87e1-2d21d54e5c18","Type":"ContainerStarted","Data":"60292e50d5af157aeabd93e5867e86bafa70a39a2040ae20d9c963b03866c38d"} Jul 06 16:25:07.369032 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:07.369010 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-vk9j9" event={"ID":"0fbab4dc-9098-4407-924a-20c35bbff2ee","Type":"ContainerStarted","Data":"d786d251b21bf0b2d09714cc7537aea9122221e3ea515fbc76805c2f0d75a886"} Jul 06 16:25:07.369106 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:07.369053 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-vk9j9" event={"ID":"0fbab4dc-9098-4407-924a-20c35bbff2ee","Type":"ContainerStarted","Data":"eaf38cf6ee1407bb0e65e565ede5847b7e8131279bc74ded05feb871e07da3f1"} Jul 06 16:25:07.369106 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:07.369073 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-vk9j9" event={"ID":"0fbab4dc-9098-4407-924a-20c35bbff2ee","Type":"ContainerStarted","Data":"246a79fc660371891ced34071ae3d8dae51a58befe543f7b5563ad33a4c0cab5"} Jul 06 16:25:07.370138 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:07.370122 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 06 16:25:07.517930 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:07.517897 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 06 16:25:07.521872 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:25:07.521846 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1e2e6141_69a0_4861_a82c_fd3377cf7752.slice/crio-bc666cc65954cc1b9ec1f94bb4122f0ce735e386f132ae8eeaf121685c87b545 WatchSource:0}: Error finding container bc666cc65954cc1b9ec1f94bb4122f0ce735e386f132ae8eeaf121685c87b545: Status 404 returned error can't find the container with id bc666cc65954cc1b9ec1f94bb4122f0ce735e386f132ae8eeaf121685c87b545 Jul 06 16:25:08.378661 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:08.377986 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-7pwll" event={"ID":"facad3ae-3b4c-4812-93b3-b398a5811e12","Type":"ContainerStarted","Data":"9fd42974f6e5af5c23d8fbaf14b829cac2a065568d5c6979209e7e7858d57b2a"} Jul 06 16:25:08.381108 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:08.381068 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1e2e6141-69a0-4861-a82c-fd3377cf7752","Type":"ContainerStarted","Data":"bc666cc65954cc1b9ec1f94bb4122f0ce735e386f132ae8eeaf121685c87b545"} Jul 06 16:25:08.839536 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:08.839491 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-599bcfcb5b-8wd62"] Jul 06 16:25:08.842252 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:08.842224 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-599bcfcb5b-8wd62" Jul 06 16:25:08.846242 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:08.846060 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\"" Jul 06 16:25:08.846744 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:08.846698 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\"" Jul 06 16:25:08.846744 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:08.846715 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\"" Jul 06 16:25:08.847218 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:08.847001 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\"" Jul 06 16:25:08.847218 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:08.847162 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-66tgcisst9rte\"" Jul 06 16:25:08.847795 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:08.847600 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-7x2kj\"" Jul 06 16:25:08.847795 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:08.847677 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\"" Jul 06 16:25:08.855433 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:08.855328 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:25:08.866783 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:08.866665 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-599bcfcb5b-8wd62"] Jul 06 16:25:08.906900 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:08.906866 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/db061724-64a7-4ee4-9021-d01e787b6c9a-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-599bcfcb5b-8wd62\" (UID: \"db061724-64a7-4ee4-9021-d01e787b6c9a\") " pod="openshift-monitoring/thanos-querier-599bcfcb5b-8wd62" Jul 06 16:25:08.907046 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:08.906921 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/db061724-64a7-4ee4-9021-d01e787b6c9a-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-599bcfcb5b-8wd62\" (UID: \"db061724-64a7-4ee4-9021-d01e787b6c9a\") " pod="openshift-monitoring/thanos-querier-599bcfcb5b-8wd62" Jul 06 16:25:08.907046 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:08.906952 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7jt45\" (UniqueName: \"kubernetes.io/projected/db061724-64a7-4ee4-9021-d01e787b6c9a-kube-api-access-7jt45\") pod \"thanos-querier-599bcfcb5b-8wd62\" (UID: \"db061724-64a7-4ee4-9021-d01e787b6c9a\") " pod="openshift-monitoring/thanos-querier-599bcfcb5b-8wd62" Jul 06 16:25:08.907046 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:08.907033 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/db061724-64a7-4ee4-9021-d01e787b6c9a-secret-thanos-querier-tls\") pod \"thanos-querier-599bcfcb5b-8wd62\" (UID: \"db061724-64a7-4ee4-9021-d01e787b6c9a\") " pod="openshift-monitoring/thanos-querier-599bcfcb5b-8wd62" Jul 06 16:25:08.907212 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:08.907071 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/db061724-64a7-4ee4-9021-d01e787b6c9a-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-599bcfcb5b-8wd62\" (UID: \"db061724-64a7-4ee4-9021-d01e787b6c9a\") " pod="openshift-monitoring/thanos-querier-599bcfcb5b-8wd62" Jul 06 16:25:08.907212 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:08.907112 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/db061724-64a7-4ee4-9021-d01e787b6c9a-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-599bcfcb5b-8wd62\" (UID: \"db061724-64a7-4ee4-9021-d01e787b6c9a\") " pod="openshift-monitoring/thanos-querier-599bcfcb5b-8wd62" Jul 06 16:25:08.907212 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:08.907145 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/db061724-64a7-4ee4-9021-d01e787b6c9a-metrics-client-ca\") pod \"thanos-querier-599bcfcb5b-8wd62\" (UID: \"db061724-64a7-4ee4-9021-d01e787b6c9a\") " pod="openshift-monitoring/thanos-querier-599bcfcb5b-8wd62" Jul 06 16:25:08.907212 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:08.907175 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/db061724-64a7-4ee4-9021-d01e787b6c9a-secret-grpc-tls\") pod \"thanos-querier-599bcfcb5b-8wd62\" (UID: \"db061724-64a7-4ee4-9021-d01e787b6c9a\") " pod="openshift-monitoring/thanos-querier-599bcfcb5b-8wd62" Jul 06 16:25:08.953023 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:08.952829 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:25:09.008529 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:09.008488 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/db061724-64a7-4ee4-9021-d01e787b6c9a-secret-thanos-querier-tls\") pod \"thanos-querier-599bcfcb5b-8wd62\" (UID: \"db061724-64a7-4ee4-9021-d01e787b6c9a\") " pod="openshift-monitoring/thanos-querier-599bcfcb5b-8wd62" Jul 06 16:25:09.008710 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:09.008544 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/db061724-64a7-4ee4-9021-d01e787b6c9a-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-599bcfcb5b-8wd62\" (UID: \"db061724-64a7-4ee4-9021-d01e787b6c9a\") " pod="openshift-monitoring/thanos-querier-599bcfcb5b-8wd62" Jul 06 16:25:09.008806 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:09.008755 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/db061724-64a7-4ee4-9021-d01e787b6c9a-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-599bcfcb5b-8wd62\" (UID: \"db061724-64a7-4ee4-9021-d01e787b6c9a\") " pod="openshift-monitoring/thanos-querier-599bcfcb5b-8wd62" Jul 06 16:25:09.008874 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:09.008827 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/db061724-64a7-4ee4-9021-d01e787b6c9a-metrics-client-ca\") pod \"thanos-querier-599bcfcb5b-8wd62\" (UID: \"db061724-64a7-4ee4-9021-d01e787b6c9a\") " pod="openshift-monitoring/thanos-querier-599bcfcb5b-8wd62" Jul 06 16:25:09.008874 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:09.008862 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/db061724-64a7-4ee4-9021-d01e787b6c9a-secret-grpc-tls\") pod \"thanos-querier-599bcfcb5b-8wd62\" (UID: \"db061724-64a7-4ee4-9021-d01e787b6c9a\") " pod="openshift-monitoring/thanos-querier-599bcfcb5b-8wd62" Jul 06 16:25:09.009058 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:09.008973 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/db061724-64a7-4ee4-9021-d01e787b6c9a-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-599bcfcb5b-8wd62\" (UID: \"db061724-64a7-4ee4-9021-d01e787b6c9a\") " pod="openshift-monitoring/thanos-querier-599bcfcb5b-8wd62" Jul 06 16:25:09.009058 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:09.009014 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/db061724-64a7-4ee4-9021-d01e787b6c9a-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-599bcfcb5b-8wd62\" (UID: \"db061724-64a7-4ee4-9021-d01e787b6c9a\") " pod="openshift-monitoring/thanos-querier-599bcfcb5b-8wd62" Jul 06 16:25:09.009058 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:09.009038 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7jt45\" (UniqueName: \"kubernetes.io/projected/db061724-64a7-4ee4-9021-d01e787b6c9a-kube-api-access-7jt45\") pod \"thanos-querier-599bcfcb5b-8wd62\" (UID: \"db061724-64a7-4ee4-9021-d01e787b6c9a\") " pod="openshift-monitoring/thanos-querier-599bcfcb5b-8wd62" Jul 06 16:25:09.009793 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:09.009707 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/db061724-64a7-4ee4-9021-d01e787b6c9a-metrics-client-ca\") pod \"thanos-querier-599bcfcb5b-8wd62\" (UID: \"db061724-64a7-4ee4-9021-d01e787b6c9a\") " pod="openshift-monitoring/thanos-querier-599bcfcb5b-8wd62" Jul 06 16:25:09.012461 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:09.011893 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/db061724-64a7-4ee4-9021-d01e787b6c9a-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-599bcfcb5b-8wd62\" (UID: \"db061724-64a7-4ee4-9021-d01e787b6c9a\") " pod="openshift-monitoring/thanos-querier-599bcfcb5b-8wd62" Jul 06 16:25:09.012461 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:09.011893 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/db061724-64a7-4ee4-9021-d01e787b6c9a-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-599bcfcb5b-8wd62\" (UID: \"db061724-64a7-4ee4-9021-d01e787b6c9a\") " pod="openshift-monitoring/thanos-querier-599bcfcb5b-8wd62" Jul 06 16:25:09.012461 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:09.012395 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/db061724-64a7-4ee4-9021-d01e787b6c9a-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-599bcfcb5b-8wd62\" (UID: \"db061724-64a7-4ee4-9021-d01e787b6c9a\") " pod="openshift-monitoring/thanos-querier-599bcfcb5b-8wd62" Jul 06 16:25:09.012461 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:09.012427 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/db061724-64a7-4ee4-9021-d01e787b6c9a-secret-grpc-tls\") pod \"thanos-querier-599bcfcb5b-8wd62\" (UID: \"db061724-64a7-4ee4-9021-d01e787b6c9a\") " pod="openshift-monitoring/thanos-querier-599bcfcb5b-8wd62" Jul 06 16:25:09.012461 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:09.012444 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/db061724-64a7-4ee4-9021-d01e787b6c9a-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-599bcfcb5b-8wd62\" (UID: \"db061724-64a7-4ee4-9021-d01e787b6c9a\") " pod="openshift-monitoring/thanos-querier-599bcfcb5b-8wd62" Jul 06 16:25:09.012803 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:09.012688 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/db061724-64a7-4ee4-9021-d01e787b6c9a-secret-thanos-querier-tls\") pod \"thanos-querier-599bcfcb5b-8wd62\" (UID: \"db061724-64a7-4ee4-9021-d01e787b6c9a\") " pod="openshift-monitoring/thanos-querier-599bcfcb5b-8wd62" Jul 06 16:25:09.023553 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:09.023527 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7jt45\" (UniqueName: \"kubernetes.io/projected/db061724-64a7-4ee4-9021-d01e787b6c9a-kube-api-access-7jt45\") pod \"thanos-querier-599bcfcb5b-8wd62\" (UID: \"db061724-64a7-4ee4-9021-d01e787b6c9a\") " pod="openshift-monitoring/thanos-querier-599bcfcb5b-8wd62" Jul 06 16:25:09.063206 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:09.063172 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:25:09.156794 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:09.156666 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-599bcfcb5b-8wd62" Jul 06 16:25:09.171053 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:09.171022 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:25:09.341958 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:09.341927 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-t9nd7" Jul 06 16:25:09.394790 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:09.394227 2584 generic.go:356] "Generic (PLEG): container finished" podID="facad3ae-3b4c-4812-93b3-b398a5811e12" containerID="9fd42974f6e5af5c23d8fbaf14b829cac2a065568d5c6979209e7e7858d57b2a" exitCode=0 Jul 06 16:25:09.394790 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:09.394322 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-7pwll" event={"ID":"facad3ae-3b4c-4812-93b3-b398a5811e12","Type":"ContainerDied","Data":"9fd42974f6e5af5c23d8fbaf14b829cac2a065568d5c6979209e7e7858d57b2a"} Jul 06 16:25:09.398138 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:09.397898 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-z5c75" event={"ID":"ed031c36-5db2-40a8-87e1-2d21d54e5c18","Type":"ContainerStarted","Data":"4df840429e2c72a63372eb97edae26dd20c7ec2f3c651ea9ac74198680a1f4d0"} Jul 06 16:25:09.398138 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:09.397982 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-z5c75" event={"ID":"ed031c36-5db2-40a8-87e1-2d21d54e5c18","Type":"ContainerStarted","Data":"4106022aaac7fe7c10afe25b257d11c5db1f1859642ab1ff46fde8c2b0a87d17"} Jul 06 16:25:09.398138 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:09.398012 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-z5c75" event={"ID":"ed031c36-5db2-40a8-87e1-2d21d54e5c18","Type":"ContainerStarted","Data":"38fcfd92a9320c6fe6cabcc21083cc1d5e3cc917bc512d3b55304c2baff5671a"} Jul 06 16:25:09.399994 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:09.399922 2584 generic.go:356] "Generic (PLEG): container finished" podID="1e2e6141-69a0-4861-a82c-fd3377cf7752" containerID="7cd6748e7ed2b1f839ce1ac7bfa64d688508336ba52c5b4b4164f9ff78019955" exitCode=0 Jul 06 16:25:09.400068 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:09.399997 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1e2e6141-69a0-4861-a82c-fd3377cf7752","Type":"ContainerDied","Data":"7cd6748e7ed2b1f839ce1ac7bfa64d688508336ba52c5b4b4164f9ff78019955"} Jul 06 16:25:09.504668 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:09.504636 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-599bcfcb5b-8wd62"] Jul 06 16:25:09.507545 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:25:09.507519 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddb061724_64a7_4ee4_9021_d01e787b6c9a.slice/crio-942656d98c04ac0c861451d4630eacbc308afd4743db2f5d96fbb1cdb7049e6f WatchSource:0}: Error finding container 942656d98c04ac0c861451d4630eacbc308afd4743db2f5d96fbb1cdb7049e6f: Status 404 returned error can't find the container with id 942656d98c04ac0c861451d4630eacbc308afd4743db2f5d96fbb1cdb7049e6f Jul 06 16:25:10.394018 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:10.393977 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:25:10.409149 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:10.409108 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-7pwll" event={"ID":"facad3ae-3b4c-4812-93b3-b398a5811e12","Type":"ContainerStarted","Data":"0b6414affc808a4ebced0b2ea447e06150d0d99060b8dc5730d49061cb6c005f"} Jul 06 16:25:10.409344 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:10.409181 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-7pwll" event={"ID":"facad3ae-3b4c-4812-93b3-b398a5811e12","Type":"ContainerStarted","Data":"21e1fc02128e0536262fbd99898b8c7d6a2445bee978f17fa221331d23a568d4"} Jul 06 16:25:10.412542 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:10.412452 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-vk9j9" event={"ID":"0fbab4dc-9098-4407-924a-20c35bbff2ee","Type":"ContainerStarted","Data":"44ae1bc835059e87b4d6e45732a7743aee104a68d4cf7df1328a8bde244dbabc"} Jul 06 16:25:10.414318 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:10.414291 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-599bcfcb5b-8wd62" event={"ID":"db061724-64a7-4ee4-9021-d01e787b6c9a","Type":"ContainerStarted","Data":"942656d98c04ac0c861451d4630eacbc308afd4743db2f5d96fbb1cdb7049e6f"} Jul 06 16:25:10.459949 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:10.459890 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-z5c75" podStartSLOduration=3.998555611 podStartE2EDuration="5.459866734s" totalImagesPullingTime="1.461311123s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 16:25:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 16:25:09.54338909 +0000 UTC m=+60.157529212" watchObservedRunningTime="2026-07-06 16:25:10.459866734 +0000 UTC m=+61.074006855" Jul 06 16:25:10.467700 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:10.467671 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-d5dcff7cf-gg9zd"] Jul 06 16:25:10.469900 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:10.469871 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-gg9zd" Jul 06 16:25:10.476348 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:10.475836 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\"" Jul 06 16:25:10.476348 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:10.476123 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-b7xbz\"" Jul 06 16:25:10.494534 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:10.494480 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-7pwll" podStartSLOduration=3.367488277 podStartE2EDuration="4.494461381s" totalImagesPullingTime="1.054840337s" totalInitContainerRuntime="72.132767ms" isStatefulPod=true podCreationTimestamp="2026-07-06 16:25:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 16:25:10.469272884 +0000 UTC m=+61.083413008" watchObservedRunningTime="2026-07-06 16:25:10.494461381 +0000 UTC m=+61.108601502" Jul 06 16:25:10.494899 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:10.494873 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-d5dcff7cf-gg9zd"] Jul 06 16:25:10.510757 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:10.510677 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-vk9j9" podStartSLOduration=3.245648426 podStartE2EDuration="5.510657869s" totalImagesPullingTime="2.265009443s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 16:25:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 16:25:10.50974978 +0000 UTC m=+61.123889899" watchObservedRunningTime="2026-07-06 16:25:10.510657869 +0000 UTC m=+61.124797992" Jul 06 16:25:10.520495 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:10.520462 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/bfc4049e-f24b-403c-8fad-70c53a6eb323-monitoring-plugin-cert\") pod \"monitoring-plugin-d5dcff7cf-gg9zd\" (UID: \"bfc4049e-f24b-403c-8fad-70c53a6eb323\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-gg9zd" Jul 06 16:25:10.520657 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:10.520564 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w45sp\" (UniqueName: \"kubernetes.io/projected/bfc4049e-f24b-403c-8fad-70c53a6eb323-kube-api-access-w45sp\") pod \"monitoring-plugin-d5dcff7cf-gg9zd\" (UID: \"bfc4049e-f24b-403c-8fad-70c53a6eb323\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-gg9zd" Jul 06 16:25:10.562515 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:10.562482 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:25:10.572155 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:10.572119 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-5b7f5d65f6-wj8m9"] Jul 06 16:25:10.574491 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:10.574470 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-5b7f5d65f6-wj8m9" Jul 06 16:25:10.578619 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:10.578596 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-k9jfr\"" Jul 06 16:25:10.579160 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:10.579137 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jul 06 16:25:10.579296 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:10.579234 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\"" Jul 06 16:25:10.579822 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:10.579688 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\"" Jul 06 16:25:10.579822 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:10.579694 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\"" Jul 06 16:25:10.579822 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:10.579697 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-2rfjc5qqqakhg\"" Jul 06 16:25:10.609975 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:10.609901 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-5b7f5d65f6-wj8m9"] Jul 06 16:25:10.621641 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:10.621602 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4ef11f66-522d-4dc8-99cc-3b883daa2f6a-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-5b7f5d65f6-wj8m9\" (UID: \"4ef11f66-522d-4dc8-99cc-3b883daa2f6a\") " pod="openshift-monitoring/metrics-server-5b7f5d65f6-wj8m9" Jul 06 16:25:10.621845 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:10.621654 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/4ef11f66-522d-4dc8-99cc-3b883daa2f6a-metrics-server-audit-profiles\") pod \"metrics-server-5b7f5d65f6-wj8m9\" (UID: \"4ef11f66-522d-4dc8-99cc-3b883daa2f6a\") " pod="openshift-monitoring/metrics-server-5b7f5d65f6-wj8m9" Jul 06 16:25:10.621845 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:10.621687 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w45sp\" (UniqueName: \"kubernetes.io/projected/bfc4049e-f24b-403c-8fad-70c53a6eb323-kube-api-access-w45sp\") pod \"monitoring-plugin-d5dcff7cf-gg9zd\" (UID: \"bfc4049e-f24b-403c-8fad-70c53a6eb323\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-gg9zd" Jul 06 16:25:10.621845 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:10.621710 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/4ef11f66-522d-4dc8-99cc-3b883daa2f6a-secret-metrics-server-tls\") pod \"metrics-server-5b7f5d65f6-wj8m9\" (UID: \"4ef11f66-522d-4dc8-99cc-3b883daa2f6a\") " pod="openshift-monitoring/metrics-server-5b7f5d65f6-wj8m9" Jul 06 16:25:10.621845 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:10.621773 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/4ef11f66-522d-4dc8-99cc-3b883daa2f6a-audit-log\") pod \"metrics-server-5b7f5d65f6-wj8m9\" (UID: \"4ef11f66-522d-4dc8-99cc-3b883daa2f6a\") " pod="openshift-monitoring/metrics-server-5b7f5d65f6-wj8m9" Jul 06 16:25:10.621845 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:10.621799 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/4ef11f66-522d-4dc8-99cc-3b883daa2f6a-secret-metrics-server-client-certs\") pod \"metrics-server-5b7f5d65f6-wj8m9\" (UID: \"4ef11f66-522d-4dc8-99cc-3b883daa2f6a\") " pod="openshift-monitoring/metrics-server-5b7f5d65f6-wj8m9" Jul 06 16:25:10.622101 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:10.621849 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rb9fb\" (UniqueName: \"kubernetes.io/projected/4ef11f66-522d-4dc8-99cc-3b883daa2f6a-kube-api-access-rb9fb\") pod \"metrics-server-5b7f5d65f6-wj8m9\" (UID: \"4ef11f66-522d-4dc8-99cc-3b883daa2f6a\") " pod="openshift-monitoring/metrics-server-5b7f5d65f6-wj8m9" Jul 06 16:25:10.622101 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:10.621927 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/4ef11f66-522d-4dc8-99cc-3b883daa2f6a-client-ca-bundle\") pod \"metrics-server-5b7f5d65f6-wj8m9\" (UID: \"4ef11f66-522d-4dc8-99cc-3b883daa2f6a\") " pod="openshift-monitoring/metrics-server-5b7f5d65f6-wj8m9" Jul 06 16:25:10.622101 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:10.621976 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/bfc4049e-f24b-403c-8fad-70c53a6eb323-monitoring-plugin-cert\") pod \"monitoring-plugin-d5dcff7cf-gg9zd\" (UID: \"bfc4049e-f24b-403c-8fad-70c53a6eb323\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-gg9zd" Jul 06 16:25:10.624992 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:10.624956 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/bfc4049e-f24b-403c-8fad-70c53a6eb323-monitoring-plugin-cert\") pod \"monitoring-plugin-d5dcff7cf-gg9zd\" (UID: \"bfc4049e-f24b-403c-8fad-70c53a6eb323\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-gg9zd" Jul 06 16:25:10.654347 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:10.654266 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w45sp\" (UniqueName: \"kubernetes.io/projected/bfc4049e-f24b-403c-8fad-70c53a6eb323-kube-api-access-w45sp\") pod \"monitoring-plugin-d5dcff7cf-gg9zd\" (UID: \"bfc4049e-f24b-403c-8fad-70c53a6eb323\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-gg9zd" Jul 06 16:25:10.685755 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:10.685705 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:25:10.723348 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:10.723304 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/4ef11f66-522d-4dc8-99cc-3b883daa2f6a-client-ca-bundle\") pod \"metrics-server-5b7f5d65f6-wj8m9\" (UID: \"4ef11f66-522d-4dc8-99cc-3b883daa2f6a\") " pod="openshift-monitoring/metrics-server-5b7f5d65f6-wj8m9" Jul 06 16:25:10.723531 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:10.723376 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4ef11f66-522d-4dc8-99cc-3b883daa2f6a-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-5b7f5d65f6-wj8m9\" (UID: \"4ef11f66-522d-4dc8-99cc-3b883daa2f6a\") " pod="openshift-monitoring/metrics-server-5b7f5d65f6-wj8m9" Jul 06 16:25:10.723531 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:10.723411 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/4ef11f66-522d-4dc8-99cc-3b883daa2f6a-metrics-server-audit-profiles\") pod \"metrics-server-5b7f5d65f6-wj8m9\" (UID: \"4ef11f66-522d-4dc8-99cc-3b883daa2f6a\") " pod="openshift-monitoring/metrics-server-5b7f5d65f6-wj8m9" Jul 06 16:25:10.723531 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:10.723438 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/4ef11f66-522d-4dc8-99cc-3b883daa2f6a-secret-metrics-server-tls\") pod \"metrics-server-5b7f5d65f6-wj8m9\" (UID: \"4ef11f66-522d-4dc8-99cc-3b883daa2f6a\") " pod="openshift-monitoring/metrics-server-5b7f5d65f6-wj8m9" Jul 06 16:25:10.723531 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:10.723473 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/4ef11f66-522d-4dc8-99cc-3b883daa2f6a-audit-log\") pod \"metrics-server-5b7f5d65f6-wj8m9\" (UID: \"4ef11f66-522d-4dc8-99cc-3b883daa2f6a\") " pod="openshift-monitoring/metrics-server-5b7f5d65f6-wj8m9" Jul 06 16:25:10.723531 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:10.723502 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/4ef11f66-522d-4dc8-99cc-3b883daa2f6a-secret-metrics-server-client-certs\") pod \"metrics-server-5b7f5d65f6-wj8m9\" (UID: \"4ef11f66-522d-4dc8-99cc-3b883daa2f6a\") " pod="openshift-monitoring/metrics-server-5b7f5d65f6-wj8m9" Jul 06 16:25:10.723760 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:10.723545 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rb9fb\" (UniqueName: \"kubernetes.io/projected/4ef11f66-522d-4dc8-99cc-3b883daa2f6a-kube-api-access-rb9fb\") pod \"metrics-server-5b7f5d65f6-wj8m9\" (UID: \"4ef11f66-522d-4dc8-99cc-3b883daa2f6a\") " pod="openshift-monitoring/metrics-server-5b7f5d65f6-wj8m9" Jul 06 16:25:10.724354 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:10.724334 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/4ef11f66-522d-4dc8-99cc-3b883daa2f6a-audit-log\") pod \"metrics-server-5b7f5d65f6-wj8m9\" (UID: \"4ef11f66-522d-4dc8-99cc-3b883daa2f6a\") " pod="openshift-monitoring/metrics-server-5b7f5d65f6-wj8m9" Jul 06 16:25:10.724599 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:10.724547 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4ef11f66-522d-4dc8-99cc-3b883daa2f6a-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-5b7f5d65f6-wj8m9\" (UID: \"4ef11f66-522d-4dc8-99cc-3b883daa2f6a\") " pod="openshift-monitoring/metrics-server-5b7f5d65f6-wj8m9" Jul 06 16:25:10.724955 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:10.724932 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/4ef11f66-522d-4dc8-99cc-3b883daa2f6a-metrics-server-audit-profiles\") pod \"metrics-server-5b7f5d65f6-wj8m9\" (UID: \"4ef11f66-522d-4dc8-99cc-3b883daa2f6a\") " pod="openshift-monitoring/metrics-server-5b7f5d65f6-wj8m9" Jul 06 16:25:10.726779 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:10.726733 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/4ef11f66-522d-4dc8-99cc-3b883daa2f6a-secret-metrics-server-tls\") pod \"metrics-server-5b7f5d65f6-wj8m9\" (UID: \"4ef11f66-522d-4dc8-99cc-3b883daa2f6a\") " pod="openshift-monitoring/metrics-server-5b7f5d65f6-wj8m9" Jul 06 16:25:10.726865 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:10.726818 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/4ef11f66-522d-4dc8-99cc-3b883daa2f6a-secret-metrics-server-client-certs\") pod \"metrics-server-5b7f5d65f6-wj8m9\" (UID: \"4ef11f66-522d-4dc8-99cc-3b883daa2f6a\") " pod="openshift-monitoring/metrics-server-5b7f5d65f6-wj8m9" Jul 06 16:25:10.727062 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:10.727042 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/4ef11f66-522d-4dc8-99cc-3b883daa2f6a-client-ca-bundle\") pod \"metrics-server-5b7f5d65f6-wj8m9\" (UID: \"4ef11f66-522d-4dc8-99cc-3b883daa2f6a\") " pod="openshift-monitoring/metrics-server-5b7f5d65f6-wj8m9" Jul 06 16:25:10.739865 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:10.739838 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rb9fb\" (UniqueName: \"kubernetes.io/projected/4ef11f66-522d-4dc8-99cc-3b883daa2f6a-kube-api-access-rb9fb\") pod \"metrics-server-5b7f5d65f6-wj8m9\" (UID: \"4ef11f66-522d-4dc8-99cc-3b883daa2f6a\") " pod="openshift-monitoring/metrics-server-5b7f5d65f6-wj8m9" Jul 06 16:25:10.783603 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:10.783568 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-gg9zd" Jul 06 16:25:10.886143 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:10.886113 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-5b7f5d65f6-wj8m9" Jul 06 16:25:10.949341 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:10.949262 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-d5dcff7cf-gg9zd"] Jul 06 16:25:10.952122 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:25:10.952092 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbfc4049e_f24b_403c_8fad_70c53a6eb323.slice/crio-a5eb4f6775919cecdae7d52f66fd4f83921f43c5cdfcc27a2898f92520a4aaeb WatchSource:0}: Error finding container a5eb4f6775919cecdae7d52f66fd4f83921f43c5cdfcc27a2898f92520a4aaeb: Status 404 returned error can't find the container with id a5eb4f6775919cecdae7d52f66fd4f83921f43c5cdfcc27a2898f92520a4aaeb Jul 06 16:25:11.054997 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:11.054967 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-5b7f5d65f6-wj8m9"] Jul 06 16:25:11.055231 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:25:11.055199 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4ef11f66_522d_4dc8_99cc_3b883daa2f6a.slice/crio-062102e22df5e023a805b8e8709ffd057d408e47a48fccd28c160e68f78d6962 WatchSource:0}: Error finding container 062102e22df5e023a805b8e8709ffd057d408e47a48fccd28c160e68f78d6962: Status 404 returned error can't find the container with id 062102e22df5e023a805b8e8709ffd057d408e47a48fccd28c160e68f78d6962 Jul 06 16:25:11.223021 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:11.222943 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-55dc9fbd88-bkmtv"] Jul 06 16:25:11.225614 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:11.225592 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-55dc9fbd88-bkmtv" Jul 06 16:25:11.231471 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:11.231447 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\"" Jul 06 16:25:11.231602 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:11.231513 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\"" Jul 06 16:25:11.231602 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:11.231449 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\"" Jul 06 16:25:11.231901 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:11.231880 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\"" Jul 06 16:25:11.231901 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:11.231888 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\"" Jul 06 16:25:11.232460 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:11.232444 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-jxnss\"" Jul 06 16:25:11.239666 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:11.239645 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\"" Jul 06 16:25:11.250705 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:11.250684 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-55dc9fbd88-bkmtv"] Jul 06 16:25:11.328351 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:11.328311 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/b223d48a-945b-4d16-ab8e-ffab4a19c40a-telemeter-client-tls\") pod \"telemeter-client-55dc9fbd88-bkmtv\" (UID: \"b223d48a-945b-4d16-ab8e-ffab4a19c40a\") " pod="openshift-monitoring/telemeter-client-55dc9fbd88-bkmtv" Jul 06 16:25:11.328532 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:11.328374 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b223d48a-945b-4d16-ab8e-ffab4a19c40a-telemeter-trusted-ca-bundle\") pod \"telemeter-client-55dc9fbd88-bkmtv\" (UID: \"b223d48a-945b-4d16-ab8e-ffab4a19c40a\") " pod="openshift-monitoring/telemeter-client-55dc9fbd88-bkmtv" Jul 06 16:25:11.328532 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:11.328458 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b223d48a-945b-4d16-ab8e-ffab4a19c40a-serving-certs-ca-bundle\") pod \"telemeter-client-55dc9fbd88-bkmtv\" (UID: \"b223d48a-945b-4d16-ab8e-ffab4a19c40a\") " pod="openshift-monitoring/telemeter-client-55dc9fbd88-bkmtv" Jul 06 16:25:11.328626 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:11.328542 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b223d48a-945b-4d16-ab8e-ffab4a19c40a-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-55dc9fbd88-bkmtv\" (UID: \"b223d48a-945b-4d16-ab8e-ffab4a19c40a\") " pod="openshift-monitoring/telemeter-client-55dc9fbd88-bkmtv" Jul 06 16:25:11.328626 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:11.328591 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/b223d48a-945b-4d16-ab8e-ffab4a19c40a-secret-telemeter-client\") pod \"telemeter-client-55dc9fbd88-bkmtv\" (UID: \"b223d48a-945b-4d16-ab8e-ffab4a19c40a\") " pod="openshift-monitoring/telemeter-client-55dc9fbd88-bkmtv" Jul 06 16:25:11.328753 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:11.328669 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/b223d48a-945b-4d16-ab8e-ffab4a19c40a-federate-client-tls\") pod \"telemeter-client-55dc9fbd88-bkmtv\" (UID: \"b223d48a-945b-4d16-ab8e-ffab4a19c40a\") " pod="openshift-monitoring/telemeter-client-55dc9fbd88-bkmtv" Jul 06 16:25:11.328753 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:11.328711 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nbpmb\" (UniqueName: \"kubernetes.io/projected/b223d48a-945b-4d16-ab8e-ffab4a19c40a-kube-api-access-nbpmb\") pod \"telemeter-client-55dc9fbd88-bkmtv\" (UID: \"b223d48a-945b-4d16-ab8e-ffab4a19c40a\") " pod="openshift-monitoring/telemeter-client-55dc9fbd88-bkmtv" Jul 06 16:25:11.328860 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:11.328792 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b223d48a-945b-4d16-ab8e-ffab4a19c40a-metrics-client-ca\") pod \"telemeter-client-55dc9fbd88-bkmtv\" (UID: \"b223d48a-945b-4d16-ab8e-ffab4a19c40a\") " pod="openshift-monitoring/telemeter-client-55dc9fbd88-bkmtv" Jul 06 16:25:11.419483 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:11.419446 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-gg9zd" event={"ID":"bfc4049e-f24b-403c-8fad-70c53a6eb323","Type":"ContainerStarted","Data":"a5eb4f6775919cecdae7d52f66fd4f83921f43c5cdfcc27a2898f92520a4aaeb"} Jul 06 16:25:11.420867 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:11.420841 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-5b7f5d65f6-wj8m9" event={"ID":"4ef11f66-522d-4dc8-99cc-3b883daa2f6a","Type":"ContainerStarted","Data":"062102e22df5e023a805b8e8709ffd057d408e47a48fccd28c160e68f78d6962"} Jul 06 16:25:11.429778 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:11.429750 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b223d48a-945b-4d16-ab8e-ffab4a19c40a-metrics-client-ca\") pod \"telemeter-client-55dc9fbd88-bkmtv\" (UID: \"b223d48a-945b-4d16-ab8e-ffab4a19c40a\") " pod="openshift-monitoring/telemeter-client-55dc9fbd88-bkmtv" Jul 06 16:25:11.429996 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:11.429796 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/b223d48a-945b-4d16-ab8e-ffab4a19c40a-telemeter-client-tls\") pod \"telemeter-client-55dc9fbd88-bkmtv\" (UID: \"b223d48a-945b-4d16-ab8e-ffab4a19c40a\") " pod="openshift-monitoring/telemeter-client-55dc9fbd88-bkmtv" Jul 06 16:25:11.429996 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:11.429840 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b223d48a-945b-4d16-ab8e-ffab4a19c40a-telemeter-trusted-ca-bundle\") pod \"telemeter-client-55dc9fbd88-bkmtv\" (UID: \"b223d48a-945b-4d16-ab8e-ffab4a19c40a\") " pod="openshift-monitoring/telemeter-client-55dc9fbd88-bkmtv" Jul 06 16:25:11.429996 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:11.429923 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b223d48a-945b-4d16-ab8e-ffab4a19c40a-serving-certs-ca-bundle\") pod \"telemeter-client-55dc9fbd88-bkmtv\" (UID: \"b223d48a-945b-4d16-ab8e-ffab4a19c40a\") " pod="openshift-monitoring/telemeter-client-55dc9fbd88-bkmtv" Jul 06 16:25:11.429996 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:11.429962 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b223d48a-945b-4d16-ab8e-ffab4a19c40a-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-55dc9fbd88-bkmtv\" (UID: \"b223d48a-945b-4d16-ab8e-ffab4a19c40a\") " pod="openshift-monitoring/telemeter-client-55dc9fbd88-bkmtv" Jul 06 16:25:11.430206 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:11.430009 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/b223d48a-945b-4d16-ab8e-ffab4a19c40a-secret-telemeter-client\") pod \"telemeter-client-55dc9fbd88-bkmtv\" (UID: \"b223d48a-945b-4d16-ab8e-ffab4a19c40a\") " pod="openshift-monitoring/telemeter-client-55dc9fbd88-bkmtv" Jul 06 16:25:11.430206 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:11.430039 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/b223d48a-945b-4d16-ab8e-ffab4a19c40a-federate-client-tls\") pod \"telemeter-client-55dc9fbd88-bkmtv\" (UID: \"b223d48a-945b-4d16-ab8e-ffab4a19c40a\") " pod="openshift-monitoring/telemeter-client-55dc9fbd88-bkmtv" Jul 06 16:25:11.430206 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:11.430083 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nbpmb\" (UniqueName: \"kubernetes.io/projected/b223d48a-945b-4d16-ab8e-ffab4a19c40a-kube-api-access-nbpmb\") pod \"telemeter-client-55dc9fbd88-bkmtv\" (UID: \"b223d48a-945b-4d16-ab8e-ffab4a19c40a\") " pod="openshift-monitoring/telemeter-client-55dc9fbd88-bkmtv" Jul 06 16:25:11.431285 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:11.430620 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b223d48a-945b-4d16-ab8e-ffab4a19c40a-metrics-client-ca\") pod \"telemeter-client-55dc9fbd88-bkmtv\" (UID: \"b223d48a-945b-4d16-ab8e-ffab4a19c40a\") " pod="openshift-monitoring/telemeter-client-55dc9fbd88-bkmtv" Jul 06 16:25:11.431285 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:11.430683 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b223d48a-945b-4d16-ab8e-ffab4a19c40a-telemeter-trusted-ca-bundle\") pod \"telemeter-client-55dc9fbd88-bkmtv\" (UID: \"b223d48a-945b-4d16-ab8e-ffab4a19c40a\") " pod="openshift-monitoring/telemeter-client-55dc9fbd88-bkmtv" Jul 06 16:25:11.431285 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:11.431243 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b223d48a-945b-4d16-ab8e-ffab4a19c40a-serving-certs-ca-bundle\") pod \"telemeter-client-55dc9fbd88-bkmtv\" (UID: \"b223d48a-945b-4d16-ab8e-ffab4a19c40a\") " pod="openshift-monitoring/telemeter-client-55dc9fbd88-bkmtv" Jul 06 16:25:11.432944 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:11.432920 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b223d48a-945b-4d16-ab8e-ffab4a19c40a-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-55dc9fbd88-bkmtv\" (UID: \"b223d48a-945b-4d16-ab8e-ffab4a19c40a\") " pod="openshift-monitoring/telemeter-client-55dc9fbd88-bkmtv" Jul 06 16:25:11.433094 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:11.433077 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/b223d48a-945b-4d16-ab8e-ffab4a19c40a-telemeter-client-tls\") pod \"telemeter-client-55dc9fbd88-bkmtv\" (UID: \"b223d48a-945b-4d16-ab8e-ffab4a19c40a\") " pod="openshift-monitoring/telemeter-client-55dc9fbd88-bkmtv" Jul 06 16:25:11.433230 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:11.433204 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/b223d48a-945b-4d16-ab8e-ffab4a19c40a-secret-telemeter-client\") pod \"telemeter-client-55dc9fbd88-bkmtv\" (UID: \"b223d48a-945b-4d16-ab8e-ffab4a19c40a\") " pod="openshift-monitoring/telemeter-client-55dc9fbd88-bkmtv" Jul 06 16:25:11.433346 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:11.433202 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/b223d48a-945b-4d16-ab8e-ffab4a19c40a-federate-client-tls\") pod \"telemeter-client-55dc9fbd88-bkmtv\" (UID: \"b223d48a-945b-4d16-ab8e-ffab4a19c40a\") " pod="openshift-monitoring/telemeter-client-55dc9fbd88-bkmtv" Jul 06 16:25:11.445279 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:11.445242 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nbpmb\" (UniqueName: \"kubernetes.io/projected/b223d48a-945b-4d16-ab8e-ffab4a19c40a-kube-api-access-nbpmb\") pod \"telemeter-client-55dc9fbd88-bkmtv\" (UID: \"b223d48a-945b-4d16-ab8e-ffab4a19c40a\") " pod="openshift-monitoring/telemeter-client-55dc9fbd88-bkmtv" Jul 06 16:25:11.536828 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:11.536717 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-55dc9fbd88-bkmtv" Jul 06 16:25:12.275972 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:12.275594 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-294bx" Jul 06 16:25:12.276892 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:12.276313 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 06 16:25:12.280825 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:12.280778 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 06 16:25:12.286080 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:12.285263 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Jul 06 16:25:12.286080 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:12.285511 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Jul 06 16:25:12.286080 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:12.285751 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Jul 06 16:25:12.289339 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:12.286963 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Jul 06 16:25:12.289339 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:12.287251 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Jul 06 16:25:12.289339 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:12.287599 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Jul 06 16:25:12.289339 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:12.287807 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Jul 06 16:25:12.289339 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:12.288057 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Jul 06 16:25:12.289339 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:12.288657 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-6isj5j0bn659p\"" Jul 06 16:25:12.289339 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:12.288928 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Jul 06 16:25:12.297377 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:12.291838 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Jul 06 16:25:12.297377 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:12.291970 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Jul 06 16:25:12.297377 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:12.292074 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-v4grf\"" Jul 06 16:25:12.297377 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:12.292133 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Jul 06 16:25:12.297377 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:12.293821 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:25:12.297377 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:12.295828 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Jul 06 16:25:12.301578 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:12.300389 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Jul 06 16:25:12.306399 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:12.306316 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-55dc9fbd88-bkmtv"] Jul 06 16:25:12.321055 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:12.321026 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 06 16:25:12.344888 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:12.343767 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/387d0b57-b4b5-4acb-822e-32b3d3d5fa63-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"387d0b57-b4b5-4acb-822e-32b3d3d5fa63\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 16:25:12.344888 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:12.343809 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/387d0b57-b4b5-4acb-822e-32b3d3d5fa63-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"387d0b57-b4b5-4acb-822e-32b3d3d5fa63\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 16:25:12.344888 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:12.343849 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/387d0b57-b4b5-4acb-822e-32b3d3d5fa63-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"387d0b57-b4b5-4acb-822e-32b3d3d5fa63\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 16:25:12.344888 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:12.343873 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/387d0b57-b4b5-4acb-822e-32b3d3d5fa63-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"387d0b57-b4b5-4acb-822e-32b3d3d5fa63\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 16:25:12.344888 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:12.343929 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/387d0b57-b4b5-4acb-822e-32b3d3d5fa63-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"387d0b57-b4b5-4acb-822e-32b3d3d5fa63\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 16:25:12.344888 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:12.343963 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/387d0b57-b4b5-4acb-822e-32b3d3d5fa63-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"387d0b57-b4b5-4acb-822e-32b3d3d5fa63\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 16:25:12.344888 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:12.343996 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/387d0b57-b4b5-4acb-822e-32b3d3d5fa63-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"387d0b57-b4b5-4acb-822e-32b3d3d5fa63\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 16:25:12.344888 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:12.344043 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/387d0b57-b4b5-4acb-822e-32b3d3d5fa63-web-config\") pod \"prometheus-k8s-0\" (UID: \"387d0b57-b4b5-4acb-822e-32b3d3d5fa63\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 16:25:12.344888 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:12.344118 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9sn27\" (UniqueName: \"kubernetes.io/projected/387d0b57-b4b5-4acb-822e-32b3d3d5fa63-kube-api-access-9sn27\") pod \"prometheus-k8s-0\" (UID: \"387d0b57-b4b5-4acb-822e-32b3d3d5fa63\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 16:25:12.344888 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:12.344152 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/387d0b57-b4b5-4acb-822e-32b3d3d5fa63-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"387d0b57-b4b5-4acb-822e-32b3d3d5fa63\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 16:25:12.344888 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:12.344184 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/387d0b57-b4b5-4acb-822e-32b3d3d5fa63-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"387d0b57-b4b5-4acb-822e-32b3d3d5fa63\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 16:25:12.344888 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:12.344222 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/387d0b57-b4b5-4acb-822e-32b3d3d5fa63-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"387d0b57-b4b5-4acb-822e-32b3d3d5fa63\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 16:25:12.344888 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:12.344245 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/387d0b57-b4b5-4acb-822e-32b3d3d5fa63-config-out\") pod \"prometheus-k8s-0\" (UID: \"387d0b57-b4b5-4acb-822e-32b3d3d5fa63\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 16:25:12.345920 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:12.344286 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/387d0b57-b4b5-4acb-822e-32b3d3d5fa63-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"387d0b57-b4b5-4acb-822e-32b3d3d5fa63\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 16:25:12.345920 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:12.344356 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/387d0b57-b4b5-4acb-822e-32b3d3d5fa63-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"387d0b57-b4b5-4acb-822e-32b3d3d5fa63\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 16:25:12.345920 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:12.344667 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/387d0b57-b4b5-4acb-822e-32b3d3d5fa63-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"387d0b57-b4b5-4acb-822e-32b3d3d5fa63\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 16:25:12.345920 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:12.344706 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/387d0b57-b4b5-4acb-822e-32b3d3d5fa63-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"387d0b57-b4b5-4acb-822e-32b3d3d5fa63\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 16:25:12.345920 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:12.344760 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/387d0b57-b4b5-4acb-822e-32b3d3d5fa63-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"387d0b57-b4b5-4acb-822e-32b3d3d5fa63\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 16:25:12.345920 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:12.345087 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/387d0b57-b4b5-4acb-822e-32b3d3d5fa63-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"387d0b57-b4b5-4acb-822e-32b3d3d5fa63\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 16:25:12.345920 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:12.345186 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/387d0b57-b4b5-4acb-822e-32b3d3d5fa63-config\") pod \"prometheus-k8s-0\" (UID: \"387d0b57-b4b5-4acb-822e-32b3d3d5fa63\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 16:25:12.431897 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:12.430583 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-599bcfcb5b-8wd62" event={"ID":"db061724-64a7-4ee4-9021-d01e787b6c9a","Type":"ContainerStarted","Data":"8c51d610de2fe9297ab0cdbe3bea25cdef42a91f639d2162b5b568f994a76aa6"} Jul 06 16:25:12.431897 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:12.430668 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-599bcfcb5b-8wd62" event={"ID":"db061724-64a7-4ee4-9021-d01e787b6c9a","Type":"ContainerStarted","Data":"c475da8bd4b06f8c09198c5cb962a64a819eb50daa99ed404a834442779329c3"} Jul 06 16:25:12.431897 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:12.430706 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-599bcfcb5b-8wd62" event={"ID":"db061724-64a7-4ee4-9021-d01e787b6c9a","Type":"ContainerStarted","Data":"b6d929108e159021c405744de0f8263ba52bdbdd4a160ebb438af83264a75f14"} Jul 06 16:25:12.432423 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:12.432388 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-55dc9fbd88-bkmtv" event={"ID":"b223d48a-945b-4d16-ab8e-ffab4a19c40a","Type":"ContainerStarted","Data":"6cc978fbb7fda305f3ed84674cbfd681cf375b763ddc2904b052ba97efbe0465"} Jul 06 16:25:12.437039 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:12.437001 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1e2e6141-69a0-4861-a82c-fd3377cf7752","Type":"ContainerStarted","Data":"da4c252a9c03fd1e5933fefce261d05aa38f0aea53c495963990fe4c2868e340"} Jul 06 16:25:12.437191 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:12.437080 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1e2e6141-69a0-4861-a82c-fd3377cf7752","Type":"ContainerStarted","Data":"54679ab2ec9c48a786e1c16c0c65200061b635139aecff858ec696f7e70d781c"} Jul 06 16:25:12.437191 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:12.437115 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1e2e6141-69a0-4861-a82c-fd3377cf7752","Type":"ContainerStarted","Data":"38acc4bb61093377bae51a27d4598de5793cccfc427c42d85d7ccd54c4ca8ea2"} Jul 06 16:25:12.446749 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:12.446344 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/387d0b57-b4b5-4acb-822e-32b3d3d5fa63-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"387d0b57-b4b5-4acb-822e-32b3d3d5fa63\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 16:25:12.446749 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:12.446394 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/387d0b57-b4b5-4acb-822e-32b3d3d5fa63-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"387d0b57-b4b5-4acb-822e-32b3d3d5fa63\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 16:25:12.446749 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:12.446426 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/387d0b57-b4b5-4acb-822e-32b3d3d5fa63-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"387d0b57-b4b5-4acb-822e-32b3d3d5fa63\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 16:25:12.446749 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:12.446460 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/387d0b57-b4b5-4acb-822e-32b3d3d5fa63-web-config\") pod \"prometheus-k8s-0\" (UID: \"387d0b57-b4b5-4acb-822e-32b3d3d5fa63\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 16:25:12.446749 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:12.446492 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9sn27\" (UniqueName: \"kubernetes.io/projected/387d0b57-b4b5-4acb-822e-32b3d3d5fa63-kube-api-access-9sn27\") pod \"prometheus-k8s-0\" (UID: \"387d0b57-b4b5-4acb-822e-32b3d3d5fa63\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 16:25:12.446749 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:12.446523 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/387d0b57-b4b5-4acb-822e-32b3d3d5fa63-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"387d0b57-b4b5-4acb-822e-32b3d3d5fa63\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 16:25:12.446749 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:12.446551 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/387d0b57-b4b5-4acb-822e-32b3d3d5fa63-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"387d0b57-b4b5-4acb-822e-32b3d3d5fa63\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 16:25:12.446749 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:12.446586 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/387d0b57-b4b5-4acb-822e-32b3d3d5fa63-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"387d0b57-b4b5-4acb-822e-32b3d3d5fa63\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 16:25:12.446749 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:12.446610 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/387d0b57-b4b5-4acb-822e-32b3d3d5fa63-config-out\") pod \"prometheus-k8s-0\" (UID: \"387d0b57-b4b5-4acb-822e-32b3d3d5fa63\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 16:25:12.446749 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:12.446645 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/387d0b57-b4b5-4acb-822e-32b3d3d5fa63-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"387d0b57-b4b5-4acb-822e-32b3d3d5fa63\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 16:25:12.446749 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:12.446675 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/387d0b57-b4b5-4acb-822e-32b3d3d5fa63-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"387d0b57-b4b5-4acb-822e-32b3d3d5fa63\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 16:25:12.446749 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:12.446703 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/387d0b57-b4b5-4acb-822e-32b3d3d5fa63-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"387d0b57-b4b5-4acb-822e-32b3d3d5fa63\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 16:25:12.447562 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:12.446823 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/387d0b57-b4b5-4acb-822e-32b3d3d5fa63-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"387d0b57-b4b5-4acb-822e-32b3d3d5fa63\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 16:25:12.447562 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:12.446858 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/387d0b57-b4b5-4acb-822e-32b3d3d5fa63-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"387d0b57-b4b5-4acb-822e-32b3d3d5fa63\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 16:25:12.447562 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:12.446890 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/387d0b57-b4b5-4acb-822e-32b3d3d5fa63-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"387d0b57-b4b5-4acb-822e-32b3d3d5fa63\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 16:25:12.447562 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:12.446926 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/387d0b57-b4b5-4acb-822e-32b3d3d5fa63-config\") pod \"prometheus-k8s-0\" (UID: \"387d0b57-b4b5-4acb-822e-32b3d3d5fa63\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 16:25:12.447562 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:12.446984 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/387d0b57-b4b5-4acb-822e-32b3d3d5fa63-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"387d0b57-b4b5-4acb-822e-32b3d3d5fa63\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 16:25:12.447562 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:12.447010 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/387d0b57-b4b5-4acb-822e-32b3d3d5fa63-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"387d0b57-b4b5-4acb-822e-32b3d3d5fa63\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 16:25:12.447562 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:12.447051 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/387d0b57-b4b5-4acb-822e-32b3d3d5fa63-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"387d0b57-b4b5-4acb-822e-32b3d3d5fa63\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 16:25:12.447562 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:12.447079 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/387d0b57-b4b5-4acb-822e-32b3d3d5fa63-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"387d0b57-b4b5-4acb-822e-32b3d3d5fa63\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 16:25:12.447967 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:12.447756 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/387d0b57-b4b5-4acb-822e-32b3d3d5fa63-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"387d0b57-b4b5-4acb-822e-32b3d3d5fa63\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 16:25:12.447967 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:12.447927 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/387d0b57-b4b5-4acb-822e-32b3d3d5fa63-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"387d0b57-b4b5-4acb-822e-32b3d3d5fa63\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 16:25:12.449410 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:12.448430 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/387d0b57-b4b5-4acb-822e-32b3d3d5fa63-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"387d0b57-b4b5-4acb-822e-32b3d3d5fa63\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 16:25:12.449410 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:12.448545 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/387d0b57-b4b5-4acb-822e-32b3d3d5fa63-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"387d0b57-b4b5-4acb-822e-32b3d3d5fa63\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 16:25:12.449410 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:12.449087 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/387d0b57-b4b5-4acb-822e-32b3d3d5fa63-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"387d0b57-b4b5-4acb-822e-32b3d3d5fa63\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 16:25:12.449756 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:12.449555 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/387d0b57-b4b5-4acb-822e-32b3d3d5fa63-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"387d0b57-b4b5-4acb-822e-32b3d3d5fa63\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 16:25:12.456029 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:12.455402 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/387d0b57-b4b5-4acb-822e-32b3d3d5fa63-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"387d0b57-b4b5-4acb-822e-32b3d3d5fa63\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 16:25:12.457707 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:12.457677 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/387d0b57-b4b5-4acb-822e-32b3d3d5fa63-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"387d0b57-b4b5-4acb-822e-32b3d3d5fa63\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 16:25:12.458352 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:12.458245 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/387d0b57-b4b5-4acb-822e-32b3d3d5fa63-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"387d0b57-b4b5-4acb-822e-32b3d3d5fa63\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 16:25:12.459080 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:12.459045 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/387d0b57-b4b5-4acb-822e-32b3d3d5fa63-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"387d0b57-b4b5-4acb-822e-32b3d3d5fa63\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 16:25:12.465161 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:12.459695 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/387d0b57-b4b5-4acb-822e-32b3d3d5fa63-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"387d0b57-b4b5-4acb-822e-32b3d3d5fa63\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 16:25:12.465161 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:12.464509 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/387d0b57-b4b5-4acb-822e-32b3d3d5fa63-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"387d0b57-b4b5-4acb-822e-32b3d3d5fa63\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 16:25:12.465161 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:12.464641 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/387d0b57-b4b5-4acb-822e-32b3d3d5fa63-web-config\") pod \"prometheus-k8s-0\" (UID: \"387d0b57-b4b5-4acb-822e-32b3d3d5fa63\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 16:25:12.465161 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:12.464654 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/387d0b57-b4b5-4acb-822e-32b3d3d5fa63-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"387d0b57-b4b5-4acb-822e-32b3d3d5fa63\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 16:25:12.465161 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:12.464890 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/387d0b57-b4b5-4acb-822e-32b3d3d5fa63-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"387d0b57-b4b5-4acb-822e-32b3d3d5fa63\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 16:25:12.465161 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:12.465007 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/387d0b57-b4b5-4acb-822e-32b3d3d5fa63-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"387d0b57-b4b5-4acb-822e-32b3d3d5fa63\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 16:25:12.465505 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:12.465197 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/387d0b57-b4b5-4acb-822e-32b3d3d5fa63-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"387d0b57-b4b5-4acb-822e-32b3d3d5fa63\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 16:25:12.466077 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:12.465909 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/387d0b57-b4b5-4acb-822e-32b3d3d5fa63-config\") pod \"prometheus-k8s-0\" (UID: \"387d0b57-b4b5-4acb-822e-32b3d3d5fa63\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 16:25:12.469176 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:12.469145 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/387d0b57-b4b5-4acb-822e-32b3d3d5fa63-config-out\") pod \"prometheus-k8s-0\" (UID: \"387d0b57-b4b5-4acb-822e-32b3d3d5fa63\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 16:25:12.490924 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:12.490861 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9sn27\" (UniqueName: \"kubernetes.io/projected/387d0b57-b4b5-4acb-822e-32b3d3d5fa63-kube-api-access-9sn27\") pod \"prometheus-k8s-0\" (UID: \"387d0b57-b4b5-4acb-822e-32b3d3d5fa63\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 16:25:12.622651 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:12.622569 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 06 16:25:12.664829 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:12.664705 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:25:12.835191 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:12.835158 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 06 16:25:12.865659 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:12.865625 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:25:13.039493 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:13.039462 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:25:13.279064 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:13.279021 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:25:13.444438 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:13.443974 2584 generic.go:356] "Generic (PLEG): container finished" podID="387d0b57-b4b5-4acb-822e-32b3d3d5fa63" containerID="5db1d4dc73ea0864daa2c9c02f7fa6f2c1d97d81a49818a4dc57ad6a62c9ee01" exitCode=0 Jul 06 16:25:13.444438 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:13.444264 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"387d0b57-b4b5-4acb-822e-32b3d3d5fa63","Type":"ContainerDied","Data":"5db1d4dc73ea0864daa2c9c02f7fa6f2c1d97d81a49818a4dc57ad6a62c9ee01"} Jul 06 16:25:13.444438 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:13.444348 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"387d0b57-b4b5-4acb-822e-32b3d3d5fa63","Type":"ContainerStarted","Data":"9e3ff11885189848dcc73252b75dccf3ced4c12a23860c984c90164159b64294"} Jul 06 16:25:13.450561 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:13.450535 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1e2e6141-69a0-4861-a82c-fd3377cf7752","Type":"ContainerStarted","Data":"96f939e145c93dd8c7e48178051261d2411cacb427857b0f56512f7e631ea669"} Jul 06 16:25:13.450658 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:13.450616 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1e2e6141-69a0-4861-a82c-fd3377cf7752","Type":"ContainerStarted","Data":"e7697f0c7eb16a900aebfa73ccc5dc2a9ecd6c63f4646a27b40adf3c9c429bd8"} Jul 06 16:25:14.460831 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:14.460792 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-599bcfcb5b-8wd62" event={"ID":"db061724-64a7-4ee4-9021-d01e787b6c9a","Type":"ContainerStarted","Data":"c07abb1c190f8c0f779596cfd27ffb105afa54a6bf77cf71c35d9ec8f81de189"} Jul 06 16:25:14.461279 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:14.460870 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-599bcfcb5b-8wd62" event={"ID":"db061724-64a7-4ee4-9021-d01e787b6c9a","Type":"ContainerStarted","Data":"d0a98b7dac7d524407113f56503354cf6c5be0538f7ff8b7d5e6681b25cb37b1"} Jul 06 16:25:14.461279 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:14.460914 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-599bcfcb5b-8wd62" event={"ID":"db061724-64a7-4ee4-9021-d01e787b6c9a","Type":"ContainerStarted","Data":"f6454fb0dbd38f0781f2ed254de77f56a6230e3af6737a2022c002ae2ced762c"} Jul 06 16:25:14.461279 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:14.460953 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-599bcfcb5b-8wd62" Jul 06 16:25:14.463521 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:14.463486 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-5b7f5d65f6-wj8m9" event={"ID":"4ef11f66-522d-4dc8-99cc-3b883daa2f6a","Type":"ContainerStarted","Data":"90071cb628d0d44ff5d2433d55dc93b2286d882a1f70419bc404520370b9b057"} Jul 06 16:25:14.465844 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:14.465778 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-gg9zd" event={"ID":"bfc4049e-f24b-403c-8fad-70c53a6eb323","Type":"ContainerStarted","Data":"2a3abba933b55341d2479f12e495a67c52206c300d0d931971d134002262a31c"} Jul 06 16:25:14.466492 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:14.466318 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-gg9zd" Jul 06 16:25:14.470820 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:14.470790 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1e2e6141-69a0-4861-a82c-fd3377cf7752","Type":"ContainerStarted","Data":"8349133de0c4221eaa2784ee3efc11a27b5b7fc35e43bb3289c01a5fdb627030"} Jul 06 16:25:14.472196 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:14.472173 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-gg9zd" Jul 06 16:25:14.515568 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:14.515506 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-599bcfcb5b-8wd62" podStartSLOduration=2.398522228 podStartE2EDuration="6.515481082s" totalImagesPullingTime="4.116958854s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 16:25:08 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 16:25:14.507467193 +0000 UTC m=+65.121607307" watchObservedRunningTime="2026-07-06 16:25:14.515481082 +0000 UTC m=+65.129621204" Jul 06 16:25:14.548303 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:14.548223 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=3.077936831 podStartE2EDuration="8.548202195s" totalImagesPullingTime="5.452603905s" totalInitContainerRuntime="17.661459ms" isStatefulPod=false podCreationTimestamp="2026-07-06 16:25:06 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 16:25:14.547103143 +0000 UTC m=+65.161243270" watchObservedRunningTime="2026-07-06 16:25:14.548202195 +0000 UTC m=+65.162342316" Jul 06 16:25:14.621948 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:14.621893 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-5b7f5d65f6-wj8m9" podStartSLOduration=1.762915774 podStartE2EDuration="4.621868148s" totalImagesPullingTime="2.858952374s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 16:25:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 16:25:14.585741464 +0000 UTC m=+65.199881579" watchObservedRunningTime="2026-07-06 16:25:14.621868148 +0000 UTC m=+65.236008272" Jul 06 16:25:15.476500 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:15.476456 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-55dc9fbd88-bkmtv" event={"ID":"b223d48a-945b-4d16-ab8e-ffab4a19c40a","Type":"ContainerStarted","Data":"f6d9b51a94babe8fae61633efce1598cf0db7022d19af852140d3a405ba03c11"} Jul 06 16:25:15.476921 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:15.476567 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-55dc9fbd88-bkmtv" event={"ID":"b223d48a-945b-4d16-ab8e-ffab4a19c40a","Type":"ContainerStarted","Data":"43f0e6d99b1251b8ba678e977084f99c92937238cf99c91366da90b66e0b7f02"} Jul 06 16:25:15.476921 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:15.476599 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-55dc9fbd88-bkmtv" event={"ID":"b223d48a-945b-4d16-ab8e-ffab4a19c40a","Type":"ContainerStarted","Data":"0db7ec5a094b9d6956c174a4b1935af2f5d00dd0d445d91e5edb98a15e5cacad"} Jul 06 16:25:15.490970 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:15.490937 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:25:15.532703 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:15.532643 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-gg9zd" podStartSLOduration=2.5755915529999998 podStartE2EDuration="5.532625293s" totalImagesPullingTime="2.95703374s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 16:25:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 16:25:14.621858079 +0000 UTC m=+65.235998199" watchObservedRunningTime="2026-07-06 16:25:15.532625293 +0000 UTC m=+66.146765413" Jul 06 16:25:15.533214 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:15.533184 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-55dc9fbd88-bkmtv" podStartSLOduration=2.059648506 podStartE2EDuration="4.533172579s" totalImagesPullingTime="2.473524073s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 16:25:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 16:25:15.531790805 +0000 UTC m=+66.145930926" watchObservedRunningTime="2026-07-06 16:25:15.533172579 +0000 UTC m=+66.147312701" Jul 06 16:25:15.593693 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:15.593654 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:25:15.783766 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:15.783669 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d695ec8b-3624-40aa-afe2-841d320d0a66-original-pull-secret\") pod \"global-pull-secret-syncer-wvcq5\" (UID: \"d695ec8b-3624-40aa-afe2-841d320d0a66\") " pod="kube-system/global-pull-secret-syncer-wvcq5" Jul 06 16:25:15.783766 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:15.783712 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3106072a-c698-44ac-9b92-8a7fd1e08d8a-metrics-certs\") pod \"network-metrics-daemon-2zdgn\" (UID: \"3106072a-c698-44ac-9b92-8a7fd1e08d8a\") " pod="openshift-multus/network-metrics-daemon-2zdgn" Jul 06 16:25:15.787261 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:15.787242 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 06 16:25:15.787510 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:15.787490 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 06 16:25:15.796652 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:15.796620 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3106072a-c698-44ac-9b92-8a7fd1e08d8a-metrics-certs\") pod \"network-metrics-daemon-2zdgn\" (UID: \"3106072a-c698-44ac-9b92-8a7fd1e08d8a\") " pod="openshift-multus/network-metrics-daemon-2zdgn" Jul 06 16:25:15.796770 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:15.796736 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d695ec8b-3624-40aa-afe2-841d320d0a66-original-pull-secret\") pod \"global-pull-secret-syncer-wvcq5\" (UID: \"d695ec8b-3624-40aa-afe2-841d320d0a66\") " pod="kube-system/global-pull-secret-syncer-wvcq5" Jul 06 16:25:15.884380 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:15.884339 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jz4gp\" (UniqueName: \"kubernetes.io/projected/6c5de78f-76c0-4c6a-9c41-787f8452e5b4-kube-api-access-jz4gp\") pod \"network-check-target-v8kjk\" (UID: \"6c5de78f-76c0-4c6a-9c41-787f8452e5b4\") " pod="openshift-network-diagnostics/network-check-target-v8kjk" Jul 06 16:25:15.902545 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:15.902515 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 06 16:25:15.914462 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:15.914430 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 06 16:25:15.917905 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:15.917874 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jz4gp\" (UniqueName: \"kubernetes.io/projected/6c5de78f-76c0-4c6a-9c41-787f8452e5b4-kube-api-access-jz4gp\") pod \"network-check-target-v8kjk\" (UID: \"6c5de78f-76c0-4c6a-9c41-787f8452e5b4\") " pod="openshift-network-diagnostics/network-check-target-v8kjk" Jul 06 16:25:15.931652 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:15.931617 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wvcq5" Jul 06 16:25:15.948236 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:15.948209 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-hn2l7\"" Jul 06 16:25:15.954154 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:15.954119 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2zdgn" Jul 06 16:25:15.960255 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:15.960047 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-6db6n\"" Jul 06 16:25:15.962448 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:15.962424 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:25:15.962560 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:15.962545 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-v8kjk" Jul 06 16:25:16.119601 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:16.119579 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-wvcq5"] Jul 06 16:25:16.121626 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:25:16.121575 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd695ec8b_3624_40aa_afe2_841d320d0a66.slice/crio-5ffd4d9f5209ac5324e7118d305c9930a400983f0641c489cd02bcfa021f459c WatchSource:0}: Error finding container 5ffd4d9f5209ac5324e7118d305c9930a400983f0641c489cd02bcfa021f459c: Status 404 returned error can't find the container with id 5ffd4d9f5209ac5324e7118d305c9930a400983f0641c489cd02bcfa021f459c Jul 06 16:25:16.138969 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:16.138937 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:25:16.145747 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:16.145696 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-2zdgn"] Jul 06 16:25:16.148442 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:25:16.148413 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3106072a_c698_44ac_9b92_8a7fd1e08d8a.slice/crio-55e472b9fc2f05713bbbc8b3549dcc229ec0190868eab79f46b1cf5b8922fecf WatchSource:0}: Error finding container 55e472b9fc2f05713bbbc8b3549dcc229ec0190868eab79f46b1cf5b8922fecf: Status 404 returned error can't find the container with id 55e472b9fc2f05713bbbc8b3549dcc229ec0190868eab79f46b1cf5b8922fecf Jul 06 16:25:16.173341 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:16.173308 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-v8kjk"] Jul 06 16:25:16.177072 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:25:16.177045 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6c5de78f_76c0_4c6a_9c41_787f8452e5b4.slice/crio-09004102df0f431fba4f3a9a416828a4f529dc8d6c5c7b4b68d42abd913cf644 WatchSource:0}: Error finding container 09004102df0f431fba4f3a9a416828a4f529dc8d6c5c7b4b68d42abd913cf644: Status 404 returned error can't find the container with id 09004102df0f431fba4f3a9a416828a4f529dc8d6c5c7b4b68d42abd913cf644 Jul 06 16:25:16.257146 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:16.257107 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:25:16.390854 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:16.390774 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:25:16.482949 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:16.482913 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-wvcq5" event={"ID":"d695ec8b-3624-40aa-afe2-841d320d0a66","Type":"ContainerStarted","Data":"5ffd4d9f5209ac5324e7118d305c9930a400983f0641c489cd02bcfa021f459c"} Jul 06 16:25:16.485001 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:16.484962 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-v8kjk" event={"ID":"6c5de78f-76c0-4c6a-9c41-787f8452e5b4","Type":"ContainerStarted","Data":"09004102df0f431fba4f3a9a416828a4f529dc8d6c5c7b4b68d42abd913cf644"} Jul 06 16:25:16.487446 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:16.487403 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-2zdgn" event={"ID":"3106072a-c698-44ac-9b92-8a7fd1e08d8a","Type":"ContainerStarted","Data":"55e472b9fc2f05713bbbc8b3549dcc229ec0190868eab79f46b1cf5b8922fecf"} Jul 06 16:25:17.063198 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:17.063110 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:25:17.267275 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:17.267208 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:25:17.460514 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:17.460477 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:25:17.677500 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:17.677445 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:25:18.310372 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:18.310340 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:25:19.508053 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:19.507963 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"387d0b57-b4b5-4acb-822e-32b3d3d5fa63","Type":"ContainerStarted","Data":"4bc77931f9c26ade34e18fe206defb68e523052dbf84703999c77a2596bd8f35"} Jul 06 16:25:19.508053 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:19.508039 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"387d0b57-b4b5-4acb-822e-32b3d3d5fa63","Type":"ContainerStarted","Data":"f382f0aa8afeb3f46a0491928d22547f49ffae8e9e008acc03d8e87396753007"} Jul 06 16:25:19.508554 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:19.508068 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"387d0b57-b4b5-4acb-822e-32b3d3d5fa63","Type":"ContainerStarted","Data":"517ce8e7eb4151ac00876501d1782eefbc1ab7a3f3286461d80b3d4d9e7e0509"} Jul 06 16:25:19.510290 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:19.510260 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-2zdgn" event={"ID":"3106072a-c698-44ac-9b92-8a7fd1e08d8a","Type":"ContainerStarted","Data":"7d884a5e856f9ad6df8a706857866d457156d640a4470c8ab4deecdda2ad1e89"} Jul 06 16:25:19.510435 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:19.510316 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-2zdgn" event={"ID":"3106072a-c698-44ac-9b92-8a7fd1e08d8a","Type":"ContainerStarted","Data":"c3174123fe1777822c7d92e75ce3e75b9b6e25a7390b571af212a6cd91297d7b"} Jul 06 16:25:19.549877 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:19.549804 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-2zdgn" podStartSLOduration=65.776672503 podStartE2EDuration="1m8.54978539s" totalImagesPullingTime="2.773112887s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 16:24:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 16:25:19.54721754 +0000 UTC m=+70.161357663" watchObservedRunningTime="2026-07-06 16:25:19.54978539 +0000 UTC m=+70.163925511" Jul 06 16:25:19.730988 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:19.730951 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:25:19.831242 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:19.831161 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:25:19.926307 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:19.926276 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:25:20.484201 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:20.484172 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-599bcfcb5b-8wd62" Jul 06 16:25:22.525956 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:22.525919 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-v8kjk" event={"ID":"6c5de78f-76c0-4c6a-9c41-787f8452e5b4","Type":"ContainerStarted","Data":"c9f225d21d71f14c8ddf8e847082cd4f38f6828542291792f6aac83ba4f8830c"} Jul 06 16:25:22.526425 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:22.526078 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-v8kjk" Jul 06 16:25:22.527419 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:22.527396 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-wvcq5" event={"ID":"d695ec8b-3624-40aa-afe2-841d320d0a66","Type":"ContainerStarted","Data":"7a30f318690367a5bd76ab46fa37de211aeedf8c645332f99e919bbc40d0c68a"} Jul 06 16:25:22.530365 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:22.530344 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"387d0b57-b4b5-4acb-822e-32b3d3d5fa63","Type":"ContainerStarted","Data":"57c8fa983b9d033a5af1c7a0dde1a2ae9ce2ceaf5536ccc1c3f23c8b3873d529"} Jul 06 16:25:22.530491 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:22.530405 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"387d0b57-b4b5-4acb-822e-32b3d3d5fa63","Type":"ContainerStarted","Data":"43baa1bcfdc1b60520a09c3140c28661ec60c28abbbe4df3fce9fd0995303c6f"} Jul 06 16:25:22.530491 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:22.530444 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"387d0b57-b4b5-4acb-822e-32b3d3d5fa63","Type":"ContainerStarted","Data":"a8df07321c9f43b846bcf3aa019f35d5af28fd9be2226ec9a8b50cea219f3b9c"} Jul 06 16:25:22.549271 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:22.549221 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-v8kjk" podStartSLOduration=65.901946654 podStartE2EDuration="1m11.549203604s" totalImagesPullingTime="5.64725695s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 16:24:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 16:25:22.54760407 +0000 UTC m=+73.161744203" watchObservedRunningTime="2026-07-06 16:25:22.549203604 +0000 UTC m=+73.163343724" Jul 06 16:25:22.591746 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:22.590149 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=5.083318938 podStartE2EDuration="10.590127288s" totalImagesPullingTime="5.480298009s" totalInitContainerRuntime="26.510341ms" isStatefulPod=false podCreationTimestamp="2026-07-06 16:25:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 16:25:22.588789917 +0000 UTC m=+73.202930059" watchObservedRunningTime="2026-07-06 16:25:22.590127288 +0000 UTC m=+73.204267411" Jul 06 16:25:22.614236 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:22.614181 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-wvcq5" podStartSLOduration=65.903824148 podStartE2EDuration="1m11.614164984s" totalImagesPullingTime="5.710340836s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 16:24:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 16:25:22.612181197 +0000 UTC m=+73.226321318" watchObservedRunningTime="2026-07-06 16:25:22.614164984 +0000 UTC m=+73.228305105" Jul 06 16:25:22.623625 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:22.623562 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 06 16:25:23.066217 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:23.066183 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:25:23.166181 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:23.166152 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:25:25.400119 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:25.400086 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:25:25.534577 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:25.534531 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:25:27.830790 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:27.830751 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:25:27.924474 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:27.924440 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:25:28.931038 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:28.930998 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:25:29.024739 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:29.024702 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:25:30.887271 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:30.887232 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-5b7f5d65f6-wj8m9" Jul 06 16:25:30.887678 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:30.887325 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-5b7f5d65f6-wj8m9" Jul 06 16:25:31.643584 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:31.643472 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:25:31.727092 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:31.727057 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:25:35.433557 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:35.433525 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:25:35.537604 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:35.537571 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:25:37.242380 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:37.242334 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:25:37.328944 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:37.328909 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:25:38.078232 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:38.078193 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:25:38.971458 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:38.971425 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:25:39.786393 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:39.786361 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:25:47.124908 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:47.124874 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:25:47.221513 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:47.221479 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:25:47.784632 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:47.784591 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:25:48.332781 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:48.332744 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:25:50.891766 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:50.891711 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-5b7f5d65f6-wj8m9" Jul 06 16:25:50.895676 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:50.895653 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-5b7f5d65f6-wj8m9" Jul 06 16:25:53.054259 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:53.054225 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:25:53.155059 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:53.155026 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:25:53.537452 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:25:53.537423 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-v8kjk" Jul 06 16:26:08.063872 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:08.063833 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:26:08.130522 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:08.130490 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:26:12.623716 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:12.623672 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Jul 06 16:26:12.644174 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:12.644142 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Jul 06 16:26:12.753086 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:12.753061 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 06 16:26:13.819120 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:13.819082 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:26:22.143332 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:22.143296 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:26:22.216610 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:22.216578 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:26:22.555539 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:22.555502 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:26:22.631698 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:22.631665 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:26:26.548542 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:26.548507 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 06 16:26:26.549002 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:26.548850 2584 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="1e2e6141-69a0-4861-a82c-fd3377cf7752" containerName="alertmanager" containerID="cri-o://38acc4bb61093377bae51a27d4598de5793cccfc427c42d85d7ccd54c4ca8ea2" gracePeriod=120 Jul 06 16:26:26.549002 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:26.548908 2584 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="1e2e6141-69a0-4861-a82c-fd3377cf7752" containerName="kube-rbac-proxy-metric" containerID="cri-o://96f939e145c93dd8c7e48178051261d2411cacb427857b0f56512f7e631ea669" gracePeriod=120 Jul 06 16:26:26.549002 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:26.548968 2584 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="1e2e6141-69a0-4861-a82c-fd3377cf7752" containerName="prom-label-proxy" containerID="cri-o://8349133de0c4221eaa2784ee3efc11a27b5b7fc35e43bb3289c01a5fdb627030" gracePeriod=120 Jul 06 16:26:26.549002 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:26.548955 2584 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="1e2e6141-69a0-4861-a82c-fd3377cf7752" containerName="kube-rbac-proxy" containerID="cri-o://e7697f0c7eb16a900aebfa73ccc5dc2a9ecd6c63f4646a27b40adf3c9c429bd8" gracePeriod=120 Jul 06 16:26:26.549239 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:26.548922 2584 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="1e2e6141-69a0-4861-a82c-fd3377cf7752" containerName="config-reloader" containerID="cri-o://54679ab2ec9c48a786e1c16c0c65200061b635139aecff858ec696f7e70d781c" gracePeriod=120 Jul 06 16:26:26.549239 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:26.548908 2584 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="1e2e6141-69a0-4861-a82c-fd3377cf7752" containerName="kube-rbac-proxy-web" containerID="cri-o://da4c252a9c03fd1e5933fefce261d05aa38f0aea53c495963990fe4c2868e340" gracePeriod=120 Jul 06 16:26:26.798699 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:26.798613 2584 generic.go:356] "Generic (PLEG): container finished" podID="1e2e6141-69a0-4861-a82c-fd3377cf7752" containerID="8349133de0c4221eaa2784ee3efc11a27b5b7fc35e43bb3289c01a5fdb627030" exitCode=0 Jul 06 16:26:26.798699 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:26.798642 2584 generic.go:356] "Generic (PLEG): container finished" podID="1e2e6141-69a0-4861-a82c-fd3377cf7752" containerID="e7697f0c7eb16a900aebfa73ccc5dc2a9ecd6c63f4646a27b40adf3c9c429bd8" exitCode=0 Jul 06 16:26:26.798699 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:26.798648 2584 generic.go:356] "Generic (PLEG): container finished" podID="1e2e6141-69a0-4861-a82c-fd3377cf7752" containerID="54679ab2ec9c48a786e1c16c0c65200061b635139aecff858ec696f7e70d781c" exitCode=0 Jul 06 16:26:26.798699 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:26.798654 2584 generic.go:356] "Generic (PLEG): container finished" podID="1e2e6141-69a0-4861-a82c-fd3377cf7752" containerID="38acc4bb61093377bae51a27d4598de5793cccfc427c42d85d7ccd54c4ca8ea2" exitCode=0 Jul 06 16:26:26.798699 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:26.798684 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1e2e6141-69a0-4861-a82c-fd3377cf7752","Type":"ContainerDied","Data":"8349133de0c4221eaa2784ee3efc11a27b5b7fc35e43bb3289c01a5fdb627030"} Jul 06 16:26:26.798972 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:26.798770 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1e2e6141-69a0-4861-a82c-fd3377cf7752","Type":"ContainerDied","Data":"e7697f0c7eb16a900aebfa73ccc5dc2a9ecd6c63f4646a27b40adf3c9c429bd8"} Jul 06 16:26:26.798972 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:26.798799 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1e2e6141-69a0-4861-a82c-fd3377cf7752","Type":"ContainerDied","Data":"54679ab2ec9c48a786e1c16c0c65200061b635139aecff858ec696f7e70d781c"} Jul 06 16:26:26.798972 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:26.798825 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1e2e6141-69a0-4861-a82c-fd3377cf7752","Type":"ContainerDied","Data":"38acc4bb61093377bae51a27d4598de5793cccfc427c42d85d7ccd54c4ca8ea2"} Jul 06 16:26:27.793699 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:27.793674 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 06 16:26:27.805850 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:27.805817 2584 generic.go:356] "Generic (PLEG): container finished" podID="1e2e6141-69a0-4861-a82c-fd3377cf7752" containerID="96f939e145c93dd8c7e48178051261d2411cacb427857b0f56512f7e631ea669" exitCode=0 Jul 06 16:26:27.805850 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:27.805847 2584 generic.go:356] "Generic (PLEG): container finished" podID="1e2e6141-69a0-4861-a82c-fd3377cf7752" containerID="da4c252a9c03fd1e5933fefce261d05aa38f0aea53c495963990fe4c2868e340" exitCode=0 Jul 06 16:26:27.806031 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:27.805873 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1e2e6141-69a0-4861-a82c-fd3377cf7752","Type":"ContainerDied","Data":"96f939e145c93dd8c7e48178051261d2411cacb427857b0f56512f7e631ea669"} Jul 06 16:26:27.806031 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:27.805911 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 06 16:26:27.806031 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:27.805949 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1e2e6141-69a0-4861-a82c-fd3377cf7752","Type":"ContainerDied","Data":"da4c252a9c03fd1e5933fefce261d05aa38f0aea53c495963990fe4c2868e340"} Jul 06 16:26:27.806031 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:27.805994 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1e2e6141-69a0-4861-a82c-fd3377cf7752","Type":"ContainerDied","Data":"bc666cc65954cc1b9ec1f94bb4122f0ce735e386f132ae8eeaf121685c87b545"} Jul 06 16:26:27.806031 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:27.806023 2584 scope.go:117] "RemoveContainer" containerID="8349133de0c4221eaa2784ee3efc11a27b5b7fc35e43bb3289c01a5fdb627030" Jul 06 16:26:27.813684 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:27.813661 2584 scope.go:117] "RemoveContainer" containerID="96f939e145c93dd8c7e48178051261d2411cacb427857b0f56512f7e631ea669" Jul 06 16:26:27.821052 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:27.821029 2584 scope.go:117] "RemoveContainer" containerID="e7697f0c7eb16a900aebfa73ccc5dc2a9ecd6c63f4646a27b40adf3c9c429bd8" Jul 06 16:26:27.827831 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:27.827812 2584 scope.go:117] "RemoveContainer" containerID="da4c252a9c03fd1e5933fefce261d05aa38f0aea53c495963990fe4c2868e340" Jul 06 16:26:27.836792 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:27.836751 2584 scope.go:117] "RemoveContainer" containerID="54679ab2ec9c48a786e1c16c0c65200061b635139aecff858ec696f7e70d781c" Jul 06 16:26:27.844930 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:27.844849 2584 scope.go:117] "RemoveContainer" containerID="38acc4bb61093377bae51a27d4598de5793cccfc427c42d85d7ccd54c4ca8ea2" Jul 06 16:26:27.856742 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:27.856701 2584 scope.go:117] "RemoveContainer" containerID="7cd6748e7ed2b1f839ce1ac7bfa64d688508336ba52c5b4b4164f9ff78019955" Jul 06 16:26:27.864330 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:27.864305 2584 scope.go:117] "RemoveContainer" containerID="8349133de0c4221eaa2784ee3efc11a27b5b7fc35e43bb3289c01a5fdb627030" Jul 06 16:26:27.864672 ip-10-0-128-22 kubenswrapper[2584]: E0706 16:26:27.864650 2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8349133de0c4221eaa2784ee3efc11a27b5b7fc35e43bb3289c01a5fdb627030\": container with ID starting with 8349133de0c4221eaa2784ee3efc11a27b5b7fc35e43bb3289c01a5fdb627030 not found: ID does not exist" containerID="8349133de0c4221eaa2784ee3efc11a27b5b7fc35e43bb3289c01a5fdb627030" Jul 06 16:26:27.864760 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:27.864682 2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8349133de0c4221eaa2784ee3efc11a27b5b7fc35e43bb3289c01a5fdb627030"} err="failed to get container status \"8349133de0c4221eaa2784ee3efc11a27b5b7fc35e43bb3289c01a5fdb627030\": rpc error: code = NotFound desc = could not find container \"8349133de0c4221eaa2784ee3efc11a27b5b7fc35e43bb3289c01a5fdb627030\": container with ID starting with 8349133de0c4221eaa2784ee3efc11a27b5b7fc35e43bb3289c01a5fdb627030 not found: ID does not exist" Jul 06 16:26:27.864760 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:27.864717 2584 scope.go:117] "RemoveContainer" containerID="96f939e145c93dd8c7e48178051261d2411cacb427857b0f56512f7e631ea669" Jul 06 16:26:27.865061 ip-10-0-128-22 kubenswrapper[2584]: E0706 16:26:27.865037 2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"96f939e145c93dd8c7e48178051261d2411cacb427857b0f56512f7e631ea669\": container with ID starting with 96f939e145c93dd8c7e48178051261d2411cacb427857b0f56512f7e631ea669 not found: ID does not exist" containerID="96f939e145c93dd8c7e48178051261d2411cacb427857b0f56512f7e631ea669" Jul 06 16:26:27.865152 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:27.865067 2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"96f939e145c93dd8c7e48178051261d2411cacb427857b0f56512f7e631ea669"} err="failed to get container status \"96f939e145c93dd8c7e48178051261d2411cacb427857b0f56512f7e631ea669\": rpc error: code = NotFound desc = could not find container \"96f939e145c93dd8c7e48178051261d2411cacb427857b0f56512f7e631ea669\": container with ID starting with 96f939e145c93dd8c7e48178051261d2411cacb427857b0f56512f7e631ea669 not found: ID does not exist" Jul 06 16:26:27.865152 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:27.865083 2584 scope.go:117] "RemoveContainer" containerID="e7697f0c7eb16a900aebfa73ccc5dc2a9ecd6c63f4646a27b40adf3c9c429bd8" Jul 06 16:26:27.865590 ip-10-0-128-22 kubenswrapper[2584]: E0706 16:26:27.865564 2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e7697f0c7eb16a900aebfa73ccc5dc2a9ecd6c63f4646a27b40adf3c9c429bd8\": container with ID starting with e7697f0c7eb16a900aebfa73ccc5dc2a9ecd6c63f4646a27b40adf3c9c429bd8 not found: ID does not exist" containerID="e7697f0c7eb16a900aebfa73ccc5dc2a9ecd6c63f4646a27b40adf3c9c429bd8" Jul 06 16:26:27.865687 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:27.865595 2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e7697f0c7eb16a900aebfa73ccc5dc2a9ecd6c63f4646a27b40adf3c9c429bd8"} err="failed to get container status \"e7697f0c7eb16a900aebfa73ccc5dc2a9ecd6c63f4646a27b40adf3c9c429bd8\": rpc error: code = NotFound desc = could not find container \"e7697f0c7eb16a900aebfa73ccc5dc2a9ecd6c63f4646a27b40adf3c9c429bd8\": container with ID starting with e7697f0c7eb16a900aebfa73ccc5dc2a9ecd6c63f4646a27b40adf3c9c429bd8 not found: ID does not exist" Jul 06 16:26:27.865687 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:27.865616 2584 scope.go:117] "RemoveContainer" containerID="da4c252a9c03fd1e5933fefce261d05aa38f0aea53c495963990fe4c2868e340" Jul 06 16:26:27.865934 ip-10-0-128-22 kubenswrapper[2584]: E0706 16:26:27.865912 2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"da4c252a9c03fd1e5933fefce261d05aa38f0aea53c495963990fe4c2868e340\": container with ID starting with da4c252a9c03fd1e5933fefce261d05aa38f0aea53c495963990fe4c2868e340 not found: ID does not exist" containerID="da4c252a9c03fd1e5933fefce261d05aa38f0aea53c495963990fe4c2868e340" Jul 06 16:26:27.866012 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:27.865941 2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"da4c252a9c03fd1e5933fefce261d05aa38f0aea53c495963990fe4c2868e340"} err="failed to get container status \"da4c252a9c03fd1e5933fefce261d05aa38f0aea53c495963990fe4c2868e340\": rpc error: code = NotFound desc = could not find container \"da4c252a9c03fd1e5933fefce261d05aa38f0aea53c495963990fe4c2868e340\": container with ID starting with da4c252a9c03fd1e5933fefce261d05aa38f0aea53c495963990fe4c2868e340 not found: ID does not exist" Jul 06 16:26:27.866012 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:27.865962 2584 scope.go:117] "RemoveContainer" containerID="54679ab2ec9c48a786e1c16c0c65200061b635139aecff858ec696f7e70d781c" Jul 06 16:26:27.866250 ip-10-0-128-22 kubenswrapper[2584]: E0706 16:26:27.866233 2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"54679ab2ec9c48a786e1c16c0c65200061b635139aecff858ec696f7e70d781c\": container with ID starting with 54679ab2ec9c48a786e1c16c0c65200061b635139aecff858ec696f7e70d781c not found: ID does not exist" containerID="54679ab2ec9c48a786e1c16c0c65200061b635139aecff858ec696f7e70d781c" Jul 06 16:26:27.866309 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:27.866256 2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"54679ab2ec9c48a786e1c16c0c65200061b635139aecff858ec696f7e70d781c"} err="failed to get container status \"54679ab2ec9c48a786e1c16c0c65200061b635139aecff858ec696f7e70d781c\": rpc error: code = NotFound desc = could not find container \"54679ab2ec9c48a786e1c16c0c65200061b635139aecff858ec696f7e70d781c\": container with ID starting with 54679ab2ec9c48a786e1c16c0c65200061b635139aecff858ec696f7e70d781c not found: ID does not exist" Jul 06 16:26:27.866309 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:27.866276 2584 scope.go:117] "RemoveContainer" containerID="38acc4bb61093377bae51a27d4598de5793cccfc427c42d85d7ccd54c4ca8ea2" Jul 06 16:26:27.866554 ip-10-0-128-22 kubenswrapper[2584]: E0706 16:26:27.866538 2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"38acc4bb61093377bae51a27d4598de5793cccfc427c42d85d7ccd54c4ca8ea2\": container with ID starting with 38acc4bb61093377bae51a27d4598de5793cccfc427c42d85d7ccd54c4ca8ea2 not found: ID does not exist" containerID="38acc4bb61093377bae51a27d4598de5793cccfc427c42d85d7ccd54c4ca8ea2" Jul 06 16:26:27.866600 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:27.866557 2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"38acc4bb61093377bae51a27d4598de5793cccfc427c42d85d7ccd54c4ca8ea2"} err="failed to get container status \"38acc4bb61093377bae51a27d4598de5793cccfc427c42d85d7ccd54c4ca8ea2\": rpc error: code = NotFound desc = could not find container \"38acc4bb61093377bae51a27d4598de5793cccfc427c42d85d7ccd54c4ca8ea2\": container with ID starting with 38acc4bb61093377bae51a27d4598de5793cccfc427c42d85d7ccd54c4ca8ea2 not found: ID does not exist" Jul 06 16:26:27.866600 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:27.866571 2584 scope.go:117] "RemoveContainer" containerID="7cd6748e7ed2b1f839ce1ac7bfa64d688508336ba52c5b4b4164f9ff78019955" Jul 06 16:26:27.866956 ip-10-0-128-22 kubenswrapper[2584]: E0706 16:26:27.866939 2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7cd6748e7ed2b1f839ce1ac7bfa64d688508336ba52c5b4b4164f9ff78019955\": container with ID starting with 7cd6748e7ed2b1f839ce1ac7bfa64d688508336ba52c5b4b4164f9ff78019955 not found: ID does not exist" containerID="7cd6748e7ed2b1f839ce1ac7bfa64d688508336ba52c5b4b4164f9ff78019955" Jul 06 16:26:27.867009 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:27.866961 2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7cd6748e7ed2b1f839ce1ac7bfa64d688508336ba52c5b4b4164f9ff78019955"} err="failed to get container status \"7cd6748e7ed2b1f839ce1ac7bfa64d688508336ba52c5b4b4164f9ff78019955\": rpc error: code = NotFound desc = could not find container \"7cd6748e7ed2b1f839ce1ac7bfa64d688508336ba52c5b4b4164f9ff78019955\": container with ID starting with 7cd6748e7ed2b1f839ce1ac7bfa64d688508336ba52c5b4b4164f9ff78019955 not found: ID does not exist" Jul 06 16:26:27.867009 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:27.866982 2584 scope.go:117] "RemoveContainer" containerID="8349133de0c4221eaa2784ee3efc11a27b5b7fc35e43bb3289c01a5fdb627030" Jul 06 16:26:27.867237 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:27.867218 2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8349133de0c4221eaa2784ee3efc11a27b5b7fc35e43bb3289c01a5fdb627030"} err="failed to get container status \"8349133de0c4221eaa2784ee3efc11a27b5b7fc35e43bb3289c01a5fdb627030\": rpc error: code = NotFound desc = could not find container \"8349133de0c4221eaa2784ee3efc11a27b5b7fc35e43bb3289c01a5fdb627030\": container with ID starting with 8349133de0c4221eaa2784ee3efc11a27b5b7fc35e43bb3289c01a5fdb627030 not found: ID does not exist" Jul 06 16:26:27.867237 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:27.867237 2584 scope.go:117] "RemoveContainer" containerID="96f939e145c93dd8c7e48178051261d2411cacb427857b0f56512f7e631ea669" Jul 06 16:26:27.867477 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:27.867457 2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"96f939e145c93dd8c7e48178051261d2411cacb427857b0f56512f7e631ea669"} err="failed to get container status \"96f939e145c93dd8c7e48178051261d2411cacb427857b0f56512f7e631ea669\": rpc error: code = NotFound desc = could not find container \"96f939e145c93dd8c7e48178051261d2411cacb427857b0f56512f7e631ea669\": container with ID starting with 96f939e145c93dd8c7e48178051261d2411cacb427857b0f56512f7e631ea669 not found: ID does not exist" Jul 06 16:26:27.867562 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:27.867482 2584 scope.go:117] "RemoveContainer" containerID="e7697f0c7eb16a900aebfa73ccc5dc2a9ecd6c63f4646a27b40adf3c9c429bd8" Jul 06 16:26:27.867753 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:27.867716 2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e7697f0c7eb16a900aebfa73ccc5dc2a9ecd6c63f4646a27b40adf3c9c429bd8"} err="failed to get container status \"e7697f0c7eb16a900aebfa73ccc5dc2a9ecd6c63f4646a27b40adf3c9c429bd8\": rpc error: code = NotFound desc = could not find container \"e7697f0c7eb16a900aebfa73ccc5dc2a9ecd6c63f4646a27b40adf3c9c429bd8\": container with ID starting with e7697f0c7eb16a900aebfa73ccc5dc2a9ecd6c63f4646a27b40adf3c9c429bd8 not found: ID does not exist" Jul 06 16:26:27.867821 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:27.867755 2584 scope.go:117] "RemoveContainer" containerID="da4c252a9c03fd1e5933fefce261d05aa38f0aea53c495963990fe4c2868e340" Jul 06 16:26:27.867952 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:27.867934 2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"da4c252a9c03fd1e5933fefce261d05aa38f0aea53c495963990fe4c2868e340"} err="failed to get container status \"da4c252a9c03fd1e5933fefce261d05aa38f0aea53c495963990fe4c2868e340\": rpc error: code = NotFound desc = could not find container \"da4c252a9c03fd1e5933fefce261d05aa38f0aea53c495963990fe4c2868e340\": container with ID starting with da4c252a9c03fd1e5933fefce261d05aa38f0aea53c495963990fe4c2868e340 not found: ID does not exist" Jul 06 16:26:27.867996 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:27.867953 2584 scope.go:117] "RemoveContainer" containerID="54679ab2ec9c48a786e1c16c0c65200061b635139aecff858ec696f7e70d781c" Jul 06 16:26:27.868201 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:27.868182 2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"54679ab2ec9c48a786e1c16c0c65200061b635139aecff858ec696f7e70d781c"} err="failed to get container status \"54679ab2ec9c48a786e1c16c0c65200061b635139aecff858ec696f7e70d781c\": rpc error: code = NotFound desc = could not find container \"54679ab2ec9c48a786e1c16c0c65200061b635139aecff858ec696f7e70d781c\": container with ID starting with 54679ab2ec9c48a786e1c16c0c65200061b635139aecff858ec696f7e70d781c not found: ID does not exist" Jul 06 16:26:27.868401 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:27.868202 2584 scope.go:117] "RemoveContainer" containerID="38acc4bb61093377bae51a27d4598de5793cccfc427c42d85d7ccd54c4ca8ea2" Jul 06 16:26:27.868457 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:27.868432 2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"38acc4bb61093377bae51a27d4598de5793cccfc427c42d85d7ccd54c4ca8ea2"} err="failed to get container status \"38acc4bb61093377bae51a27d4598de5793cccfc427c42d85d7ccd54c4ca8ea2\": rpc error: code = NotFound desc = could not find container \"38acc4bb61093377bae51a27d4598de5793cccfc427c42d85d7ccd54c4ca8ea2\": container with ID starting with 38acc4bb61093377bae51a27d4598de5793cccfc427c42d85d7ccd54c4ca8ea2 not found: ID does not exist" Jul 06 16:26:27.868501 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:27.868461 2584 scope.go:117] "RemoveContainer" containerID="7cd6748e7ed2b1f839ce1ac7bfa64d688508336ba52c5b4b4164f9ff78019955" Jul 06 16:26:27.868718 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:27.868697 2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7cd6748e7ed2b1f839ce1ac7bfa64d688508336ba52c5b4b4164f9ff78019955"} err="failed to get container status \"7cd6748e7ed2b1f839ce1ac7bfa64d688508336ba52c5b4b4164f9ff78019955\": rpc error: code = NotFound desc = could not find container \"7cd6748e7ed2b1f839ce1ac7bfa64d688508336ba52c5b4b4164f9ff78019955\": container with ID starting with 7cd6748e7ed2b1f839ce1ac7bfa64d688508336ba52c5b4b4164f9ff78019955 not found: ID does not exist" Jul 06 16:26:27.948091 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:27.948054 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qjv9x\" (UniqueName: \"kubernetes.io/projected/1e2e6141-69a0-4861-a82c-fd3377cf7752-kube-api-access-qjv9x\") pod \"1e2e6141-69a0-4861-a82c-fd3377cf7752\" (UID: \"1e2e6141-69a0-4861-a82c-fd3377cf7752\") " Jul 06 16:26:27.948276 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:27.948100 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1e2e6141-69a0-4861-a82c-fd3377cf7752-metrics-client-ca\") pod \"1e2e6141-69a0-4861-a82c-fd3377cf7752\" (UID: \"1e2e6141-69a0-4861-a82c-fd3377cf7752\") " Jul 06 16:26:27.948276 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:27.948133 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/1e2e6141-69a0-4861-a82c-fd3377cf7752-alertmanager-main-db\") pod \"1e2e6141-69a0-4861-a82c-fd3377cf7752\" (UID: \"1e2e6141-69a0-4861-a82c-fd3377cf7752\") " Jul 06 16:26:27.948276 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:27.948160 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/1e2e6141-69a0-4861-a82c-fd3377cf7752-secret-alertmanager-kube-rbac-proxy\") pod \"1e2e6141-69a0-4861-a82c-fd3377cf7752\" (UID: \"1e2e6141-69a0-4861-a82c-fd3377cf7752\") " Jul 06 16:26:27.948276 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:27.948205 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/1e2e6141-69a0-4861-a82c-fd3377cf7752-tls-assets\") pod \"1e2e6141-69a0-4861-a82c-fd3377cf7752\" (UID: \"1e2e6141-69a0-4861-a82c-fd3377cf7752\") " Jul 06 16:26:27.948482 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:27.948288 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1e2e6141-69a0-4861-a82c-fd3377cf7752-alertmanager-trusted-ca-bundle\") pod \"1e2e6141-69a0-4861-a82c-fd3377cf7752\" (UID: \"1e2e6141-69a0-4861-a82c-fd3377cf7752\") " Jul 06 16:26:27.948482 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:27.948351 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/1e2e6141-69a0-4861-a82c-fd3377cf7752-web-config\") pod \"1e2e6141-69a0-4861-a82c-fd3377cf7752\" (UID: \"1e2e6141-69a0-4861-a82c-fd3377cf7752\") " Jul 06 16:26:27.948482 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:27.948392 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/1e2e6141-69a0-4861-a82c-fd3377cf7752-secret-alertmanager-main-tls\") pod \"1e2e6141-69a0-4861-a82c-fd3377cf7752\" (UID: \"1e2e6141-69a0-4861-a82c-fd3377cf7752\") " Jul 06 16:26:27.948482 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:27.948452 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/1e2e6141-69a0-4861-a82c-fd3377cf7752-config-volume\") pod \"1e2e6141-69a0-4861-a82c-fd3377cf7752\" (UID: \"1e2e6141-69a0-4861-a82c-fd3377cf7752\") " Jul 06 16:26:27.948691 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:27.948487 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/1e2e6141-69a0-4861-a82c-fd3377cf7752-cluster-tls-config\") pod \"1e2e6141-69a0-4861-a82c-fd3377cf7752\" (UID: \"1e2e6141-69a0-4861-a82c-fd3377cf7752\") " Jul 06 16:26:27.948691 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:27.948530 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/1e2e6141-69a0-4861-a82c-fd3377cf7752-secret-alertmanager-kube-rbac-proxy-web\") pod \"1e2e6141-69a0-4861-a82c-fd3377cf7752\" (UID: \"1e2e6141-69a0-4861-a82c-fd3377cf7752\") " Jul 06 16:26:27.948691 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:27.948572 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/1e2e6141-69a0-4861-a82c-fd3377cf7752-secret-alertmanager-kube-rbac-proxy-metric\") pod \"1e2e6141-69a0-4861-a82c-fd3377cf7752\" (UID: \"1e2e6141-69a0-4861-a82c-fd3377cf7752\") " Jul 06 16:26:27.948691 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:27.948572 2584 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1e2e6141-69a0-4861-a82c-fd3377cf7752-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "1e2e6141-69a0-4861-a82c-fd3377cf7752" (UID: "1e2e6141-69a0-4861-a82c-fd3377cf7752"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 16:26:27.948691 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:27.948620 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/1e2e6141-69a0-4861-a82c-fd3377cf7752-config-out\") pod \"1e2e6141-69a0-4861-a82c-fd3377cf7752\" (UID: \"1e2e6141-69a0-4861-a82c-fd3377cf7752\") " Jul 06 16:26:27.948691 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:27.948643 2584 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1e2e6141-69a0-4861-a82c-fd3377cf7752-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "1e2e6141-69a0-4861-a82c-fd3377cf7752" (UID: "1e2e6141-69a0-4861-a82c-fd3377cf7752"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 16:26:27.949029 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:27.948932 2584 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1e2e6141-69a0-4861-a82c-fd3377cf7752-metrics-client-ca\") on node \"ip-10-0-128-22.ec2.internal\" DevicePath \"\"" Jul 06 16:26:27.949029 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:27.948951 2584 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/1e2e6141-69a0-4861-a82c-fd3377cf7752-alertmanager-main-db\") on node \"ip-10-0-128-22.ec2.internal\" DevicePath \"\"" Jul 06 16:26:27.949029 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:27.948997 2584 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1e2e6141-69a0-4861-a82c-fd3377cf7752-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "1e2e6141-69a0-4861-a82c-fd3377cf7752" (UID: "1e2e6141-69a0-4861-a82c-fd3377cf7752"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 16:26:27.951463 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:27.951436 2584 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1e2e6141-69a0-4861-a82c-fd3377cf7752-kube-api-access-qjv9x" (OuterVolumeSpecName: "kube-api-access-qjv9x") pod "1e2e6141-69a0-4861-a82c-fd3377cf7752" (UID: "1e2e6141-69a0-4861-a82c-fd3377cf7752"). InnerVolumeSpecName "kube-api-access-qjv9x". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 16:26:27.951686 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:27.951655 2584 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1e2e6141-69a0-4861-a82c-fd3377cf7752-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "1e2e6141-69a0-4861-a82c-fd3377cf7752" (UID: "1e2e6141-69a0-4861-a82c-fd3377cf7752"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 16:26:27.951922 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:27.951900 2584 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1e2e6141-69a0-4861-a82c-fd3377cf7752-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "1e2e6141-69a0-4861-a82c-fd3377cf7752" (UID: "1e2e6141-69a0-4861-a82c-fd3377cf7752"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 16:26:27.952119 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:27.952101 2584 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1e2e6141-69a0-4861-a82c-fd3377cf7752-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "1e2e6141-69a0-4861-a82c-fd3377cf7752" (UID: "1e2e6141-69a0-4861-a82c-fd3377cf7752"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 16:26:27.952392 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:27.952367 2584 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1e2e6141-69a0-4861-a82c-fd3377cf7752-config-out" (OuterVolumeSpecName: "config-out") pod "1e2e6141-69a0-4861-a82c-fd3377cf7752" (UID: "1e2e6141-69a0-4861-a82c-fd3377cf7752"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 16:26:27.952501 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:27.952390 2584 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1e2e6141-69a0-4861-a82c-fd3377cf7752-config-volume" (OuterVolumeSpecName: "config-volume") pod "1e2e6141-69a0-4861-a82c-fd3377cf7752" (UID: "1e2e6141-69a0-4861-a82c-fd3377cf7752"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 16:26:27.952782 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:27.952766 2584 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1e2e6141-69a0-4861-a82c-fd3377cf7752-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "1e2e6141-69a0-4861-a82c-fd3377cf7752" (UID: "1e2e6141-69a0-4861-a82c-fd3377cf7752"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 16:26:27.953937 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:27.953920 2584 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1e2e6141-69a0-4861-a82c-fd3377cf7752-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "1e2e6141-69a0-4861-a82c-fd3377cf7752" (UID: "1e2e6141-69a0-4861-a82c-fd3377cf7752"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 16:26:27.956713 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:27.956635 2584 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1e2e6141-69a0-4861-a82c-fd3377cf7752-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "1e2e6141-69a0-4861-a82c-fd3377cf7752" (UID: "1e2e6141-69a0-4861-a82c-fd3377cf7752"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 16:26:27.963304 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:27.963273 2584 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1e2e6141-69a0-4861-a82c-fd3377cf7752-web-config" (OuterVolumeSpecName: "web-config") pod "1e2e6141-69a0-4861-a82c-fd3377cf7752" (UID: "1e2e6141-69a0-4861-a82c-fd3377cf7752"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 16:26:28.049862 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:28.049815 2584 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/1e2e6141-69a0-4861-a82c-fd3377cf7752-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-128-22.ec2.internal\" DevicePath \"\"" Jul 06 16:26:28.049862 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:28.049860 2584 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/1e2e6141-69a0-4861-a82c-fd3377cf7752-config-out\") on node \"ip-10-0-128-22.ec2.internal\" DevicePath \"\"" Jul 06 16:26:28.049862 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:28.049875 2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qjv9x\" (UniqueName: \"kubernetes.io/projected/1e2e6141-69a0-4861-a82c-fd3377cf7752-kube-api-access-qjv9x\") on node \"ip-10-0-128-22.ec2.internal\" DevicePath \"\"" Jul 06 16:26:28.050159 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:28.049888 2584 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/1e2e6141-69a0-4861-a82c-fd3377cf7752-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-128-22.ec2.internal\" DevicePath \"\"" Jul 06 16:26:28.050159 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:28.049902 2584 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/1e2e6141-69a0-4861-a82c-fd3377cf7752-tls-assets\") on node \"ip-10-0-128-22.ec2.internal\" DevicePath \"\"" Jul 06 16:26:28.050159 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:28.049913 2584 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1e2e6141-69a0-4861-a82c-fd3377cf7752-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-128-22.ec2.internal\" DevicePath \"\"" Jul 06 16:26:28.050159 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:28.049924 2584 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/1e2e6141-69a0-4861-a82c-fd3377cf7752-web-config\") on node \"ip-10-0-128-22.ec2.internal\" DevicePath \"\"" Jul 06 16:26:28.050159 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:28.049935 2584 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/1e2e6141-69a0-4861-a82c-fd3377cf7752-secret-alertmanager-main-tls\") on node \"ip-10-0-128-22.ec2.internal\" DevicePath \"\"" Jul 06 16:26:28.050159 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:28.049946 2584 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/1e2e6141-69a0-4861-a82c-fd3377cf7752-config-volume\") on node \"ip-10-0-128-22.ec2.internal\" DevicePath \"\"" Jul 06 16:26:28.050159 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:28.049957 2584 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/1e2e6141-69a0-4861-a82c-fd3377cf7752-cluster-tls-config\") on node \"ip-10-0-128-22.ec2.internal\" DevicePath \"\"" Jul 06 16:26:28.050159 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:28.049969 2584 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/1e2e6141-69a0-4861-a82c-fd3377cf7752-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-128-22.ec2.internal\" DevicePath \"\"" Jul 06 16:26:28.136616 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:28.136583 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 06 16:26:28.141172 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:28.141146 2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 06 16:26:28.178413 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:28.178380 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 06 16:26:28.178766 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:28.178750 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1e2e6141-69a0-4861-a82c-fd3377cf7752" containerName="init-config-reloader" Jul 06 16:26:28.178766 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:28.178767 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="1e2e6141-69a0-4861-a82c-fd3377cf7752" containerName="init-config-reloader" Jul 06 16:26:28.178853 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:28.178775 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1e2e6141-69a0-4861-a82c-fd3377cf7752" containerName="kube-rbac-proxy" Jul 06 16:26:28.178853 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:28.178782 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="1e2e6141-69a0-4861-a82c-fd3377cf7752" containerName="kube-rbac-proxy" Jul 06 16:26:28.178853 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:28.178793 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1e2e6141-69a0-4861-a82c-fd3377cf7752" containerName="prom-label-proxy" Jul 06 16:26:28.178853 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:28.178798 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="1e2e6141-69a0-4861-a82c-fd3377cf7752" containerName="prom-label-proxy" Jul 06 16:26:28.178853 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:28.178804 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1e2e6141-69a0-4861-a82c-fd3377cf7752" containerName="alertmanager" Jul 06 16:26:28.178853 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:28.178810 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="1e2e6141-69a0-4861-a82c-fd3377cf7752" containerName="alertmanager" Jul 06 16:26:28.178853 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:28.178823 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1e2e6141-69a0-4861-a82c-fd3377cf7752" containerName="kube-rbac-proxy-metric" Jul 06 16:26:28.178853 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:28.178829 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="1e2e6141-69a0-4861-a82c-fd3377cf7752" containerName="kube-rbac-proxy-metric" Jul 06 16:26:28.178853 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:28.178850 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1e2e6141-69a0-4861-a82c-fd3377cf7752" containerName="config-reloader" Jul 06 16:26:28.178853 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:28.178857 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="1e2e6141-69a0-4861-a82c-fd3377cf7752" containerName="config-reloader" Jul 06 16:26:28.179196 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:28.178868 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1e2e6141-69a0-4861-a82c-fd3377cf7752" containerName="kube-rbac-proxy-web" Jul 06 16:26:28.179196 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:28.178876 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="1e2e6141-69a0-4861-a82c-fd3377cf7752" containerName="kube-rbac-proxy-web" Jul 06 16:26:28.179196 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:28.178940 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="1e2e6141-69a0-4861-a82c-fd3377cf7752" containerName="config-reloader" Jul 06 16:26:28.179196 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:28.178952 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="1e2e6141-69a0-4861-a82c-fd3377cf7752" containerName="kube-rbac-proxy-web" Jul 06 16:26:28.179196 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:28.178962 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="1e2e6141-69a0-4861-a82c-fd3377cf7752" containerName="prom-label-proxy" Jul 06 16:26:28.179196 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:28.178970 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="1e2e6141-69a0-4861-a82c-fd3377cf7752" containerName="alertmanager" Jul 06 16:26:28.179196 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:28.178981 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="1e2e6141-69a0-4861-a82c-fd3377cf7752" containerName="kube-rbac-proxy" Jul 06 16:26:28.179196 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:28.178991 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="1e2e6141-69a0-4861-a82c-fd3377cf7752" containerName="kube-rbac-proxy-metric" Jul 06 16:26:28.183095 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:28.183073 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 06 16:26:28.187575 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:28.187540 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jul 06 16:26:28.187575 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:28.187559 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jul 06 16:26:28.187800 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:28.187599 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jul 06 16:26:28.187800 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:28.187601 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-s56wg\"" Jul 06 16:26:28.187800 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:28.187549 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jul 06 16:26:28.188074 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:28.188057 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jul 06 16:26:28.188142 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:28.188109 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jul 06 16:26:28.188142 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:28.188129 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jul 06 16:26:28.190812 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:28.190795 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jul 06 16:26:28.194992 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:28.194976 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jul 06 16:26:28.202657 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:28.202587 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 06 16:26:28.352316 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:28.352267 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/772a7200-5b39-41ff-aff9-db11c2da83da-tls-assets\") pod \"alertmanager-main-0\" (UID: \"772a7200-5b39-41ff-aff9-db11c2da83da\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 16:26:28.352316 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:28.352319 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/772a7200-5b39-41ff-aff9-db11c2da83da-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"772a7200-5b39-41ff-aff9-db11c2da83da\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 16:26:28.352525 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:28.352389 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/772a7200-5b39-41ff-aff9-db11c2da83da-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"772a7200-5b39-41ff-aff9-db11c2da83da\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 16:26:28.352525 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:28.352422 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/772a7200-5b39-41ff-aff9-db11c2da83da-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"772a7200-5b39-41ff-aff9-db11c2da83da\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 16:26:28.352525 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:28.352454 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/772a7200-5b39-41ff-aff9-db11c2da83da-web-config\") pod \"alertmanager-main-0\" (UID: \"772a7200-5b39-41ff-aff9-db11c2da83da\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 16:26:28.352525 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:28.352487 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/772a7200-5b39-41ff-aff9-db11c2da83da-config-volume\") pod \"alertmanager-main-0\" (UID: \"772a7200-5b39-41ff-aff9-db11c2da83da\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 16:26:28.352525 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:28.352508 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/772a7200-5b39-41ff-aff9-db11c2da83da-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"772a7200-5b39-41ff-aff9-db11c2da83da\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 16:26:28.352525 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:28.352523 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nv4wc\" (UniqueName: \"kubernetes.io/projected/772a7200-5b39-41ff-aff9-db11c2da83da-kube-api-access-nv4wc\") pod \"alertmanager-main-0\" (UID: \"772a7200-5b39-41ff-aff9-db11c2da83da\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 16:26:28.352702 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:28.352538 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/772a7200-5b39-41ff-aff9-db11c2da83da-config-out\") pod \"alertmanager-main-0\" (UID: \"772a7200-5b39-41ff-aff9-db11c2da83da\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 16:26:28.352702 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:28.352607 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/772a7200-5b39-41ff-aff9-db11c2da83da-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"772a7200-5b39-41ff-aff9-db11c2da83da\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 16:26:28.352702 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:28.352668 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/772a7200-5b39-41ff-aff9-db11c2da83da-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"772a7200-5b39-41ff-aff9-db11c2da83da\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 16:26:28.352702 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:28.352690 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/772a7200-5b39-41ff-aff9-db11c2da83da-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"772a7200-5b39-41ff-aff9-db11c2da83da\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 16:26:28.352859 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:28.352715 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/772a7200-5b39-41ff-aff9-db11c2da83da-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"772a7200-5b39-41ff-aff9-db11c2da83da\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 16:26:28.453690 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:28.453603 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/772a7200-5b39-41ff-aff9-db11c2da83da-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"772a7200-5b39-41ff-aff9-db11c2da83da\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 16:26:28.453690 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:28.453661 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/772a7200-5b39-41ff-aff9-db11c2da83da-web-config\") pod \"alertmanager-main-0\" (UID: \"772a7200-5b39-41ff-aff9-db11c2da83da\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 16:26:28.453690 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:28.453691 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/772a7200-5b39-41ff-aff9-db11c2da83da-config-volume\") pod \"alertmanager-main-0\" (UID: \"772a7200-5b39-41ff-aff9-db11c2da83da\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 16:26:28.453949 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:28.453709 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/772a7200-5b39-41ff-aff9-db11c2da83da-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"772a7200-5b39-41ff-aff9-db11c2da83da\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 16:26:28.453949 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:28.453740 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nv4wc\" (UniqueName: \"kubernetes.io/projected/772a7200-5b39-41ff-aff9-db11c2da83da-kube-api-access-nv4wc\") pod \"alertmanager-main-0\" (UID: \"772a7200-5b39-41ff-aff9-db11c2da83da\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 16:26:28.453949 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:28.453765 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/772a7200-5b39-41ff-aff9-db11c2da83da-config-out\") pod \"alertmanager-main-0\" (UID: \"772a7200-5b39-41ff-aff9-db11c2da83da\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 16:26:28.453949 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:28.453789 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/772a7200-5b39-41ff-aff9-db11c2da83da-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"772a7200-5b39-41ff-aff9-db11c2da83da\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 16:26:28.454511 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:28.454142 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/772a7200-5b39-41ff-aff9-db11c2da83da-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"772a7200-5b39-41ff-aff9-db11c2da83da\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 16:26:28.454511 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:28.454192 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/772a7200-5b39-41ff-aff9-db11c2da83da-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"772a7200-5b39-41ff-aff9-db11c2da83da\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 16:26:28.454511 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:28.454203 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/772a7200-5b39-41ff-aff9-db11c2da83da-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"772a7200-5b39-41ff-aff9-db11c2da83da\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 16:26:28.454511 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:28.454284 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/772a7200-5b39-41ff-aff9-db11c2da83da-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"772a7200-5b39-41ff-aff9-db11c2da83da\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 16:26:28.454511 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:28.454347 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/772a7200-5b39-41ff-aff9-db11c2da83da-tls-assets\") pod \"alertmanager-main-0\" (UID: \"772a7200-5b39-41ff-aff9-db11c2da83da\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 16:26:28.454511 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:28.454386 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/772a7200-5b39-41ff-aff9-db11c2da83da-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"772a7200-5b39-41ff-aff9-db11c2da83da\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 16:26:28.454511 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:28.454433 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/772a7200-5b39-41ff-aff9-db11c2da83da-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"772a7200-5b39-41ff-aff9-db11c2da83da\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 16:26:28.454511 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:28.454438 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/772a7200-5b39-41ff-aff9-db11c2da83da-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"772a7200-5b39-41ff-aff9-db11c2da83da\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 16:26:28.456779 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:28.456624 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/772a7200-5b39-41ff-aff9-db11c2da83da-config-out\") pod \"alertmanager-main-0\" (UID: \"772a7200-5b39-41ff-aff9-db11c2da83da\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 16:26:28.456956 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:28.456928 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/772a7200-5b39-41ff-aff9-db11c2da83da-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"772a7200-5b39-41ff-aff9-db11c2da83da\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 16:26:28.457036 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:28.456962 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/772a7200-5b39-41ff-aff9-db11c2da83da-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"772a7200-5b39-41ff-aff9-db11c2da83da\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 16:26:28.457036 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:28.456984 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/772a7200-5b39-41ff-aff9-db11c2da83da-web-config\") pod \"alertmanager-main-0\" (UID: \"772a7200-5b39-41ff-aff9-db11c2da83da\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 16:26:28.457036 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:28.456997 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/772a7200-5b39-41ff-aff9-db11c2da83da-config-volume\") pod \"alertmanager-main-0\" (UID: \"772a7200-5b39-41ff-aff9-db11c2da83da\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 16:26:28.457183 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:28.457049 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/772a7200-5b39-41ff-aff9-db11c2da83da-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"772a7200-5b39-41ff-aff9-db11c2da83da\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 16:26:28.457241 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:28.457208 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/772a7200-5b39-41ff-aff9-db11c2da83da-tls-assets\") pod \"alertmanager-main-0\" (UID: \"772a7200-5b39-41ff-aff9-db11c2da83da\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 16:26:28.457374 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:28.457355 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/772a7200-5b39-41ff-aff9-db11c2da83da-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"772a7200-5b39-41ff-aff9-db11c2da83da\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 16:26:28.457749 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:28.457710 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/772a7200-5b39-41ff-aff9-db11c2da83da-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"772a7200-5b39-41ff-aff9-db11c2da83da\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 16:26:28.458761 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:28.458741 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/772a7200-5b39-41ff-aff9-db11c2da83da-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"772a7200-5b39-41ff-aff9-db11c2da83da\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 16:26:28.463984 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:28.463968 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nv4wc\" (UniqueName: \"kubernetes.io/projected/772a7200-5b39-41ff-aff9-db11c2da83da-kube-api-access-nv4wc\") pod \"alertmanager-main-0\" (UID: \"772a7200-5b39-41ff-aff9-db11c2da83da\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 16:26:28.492994 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:28.492964 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 06 16:26:28.636122 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:28.635992 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 06 16:26:28.640621 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:26:28.640592 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod772a7200_5b39_41ff_aff9_db11c2da83da.slice/crio-f866ff5795ea7bd511485521cec7bf3aba31d1f419b88eee5995b449b43f47d2 WatchSource:0}: Error finding container f866ff5795ea7bd511485521cec7bf3aba31d1f419b88eee5995b449b43f47d2: Status 404 returned error can't find the container with id f866ff5795ea7bd511485521cec7bf3aba31d1f419b88eee5995b449b43f47d2 Jul 06 16:26:28.811568 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:28.811539 2584 generic.go:356] "Generic (PLEG): container finished" podID="772a7200-5b39-41ff-aff9-db11c2da83da" containerID="155bb692e2e4d2362df9b5ef6f504669463682aa2b44d7c2249358992e0a36e0" exitCode=0 Jul 06 16:26:28.811977 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:28.811613 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"772a7200-5b39-41ff-aff9-db11c2da83da","Type":"ContainerDied","Data":"155bb692e2e4d2362df9b5ef6f504669463682aa2b44d7c2249358992e0a36e0"} Jul 06 16:26:28.811977 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:28.811659 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"772a7200-5b39-41ff-aff9-db11c2da83da","Type":"ContainerStarted","Data":"f866ff5795ea7bd511485521cec7bf3aba31d1f419b88eee5995b449b43f47d2"} Jul 06 16:26:29.818646 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:29.818614 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"772a7200-5b39-41ff-aff9-db11c2da83da","Type":"ContainerStarted","Data":"7909cc9186c8df704cf17299d769612213556ede844cb2f74588ed98f01f0e5a"} Jul 06 16:26:29.819039 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:29.818674 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"772a7200-5b39-41ff-aff9-db11c2da83da","Type":"ContainerStarted","Data":"1906d6ac2f2d1e565c6d52fcbaab10ed44e452330d03b7aa3f3b2a1cd0bfef5e"} Jul 06 16:26:29.819039 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:29.818699 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"772a7200-5b39-41ff-aff9-db11c2da83da","Type":"ContainerStarted","Data":"7f8fb2e9f8faee582baa4cd4fa00c11f6fadde48bde18b43a84f338bcf908149"} Jul 06 16:26:29.819039 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:29.818738 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"772a7200-5b39-41ff-aff9-db11c2da83da","Type":"ContainerStarted","Data":"9e9fcde605aaf1dda78764653cf5b1c7975a0f0c9a62cea78950922fbf2cf58a"} Jul 06 16:26:29.819039 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:29.818761 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"772a7200-5b39-41ff-aff9-db11c2da83da","Type":"ContainerStarted","Data":"0772e35514c0478939df180fb4562808d1952af8902c908a5484dc8a00c44cc9"} Jul 06 16:26:29.819039 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:29.818783 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"772a7200-5b39-41ff-aff9-db11c2da83da","Type":"ContainerStarted","Data":"ccf37d5729fd58d899505f478216ee76b28558d59cf69cb02c666ca5e722bc95"} Jul 06 16:26:29.857310 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:29.857210 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=1.842734755 podStartE2EDuration="1.85718762s" totalImagesPullingTime="0s" totalInitContainerRuntime="14.452865ms" isStatefulPod=false podCreationTimestamp="2026-07-06 16:26:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 16:26:29.854690775 +0000 UTC m=+140.468830911" watchObservedRunningTime="2026-07-06 16:26:29.85718762 +0000 UTC m=+140.471327747" Jul 06 16:26:30.118681 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:30.118587 2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1e2e6141-69a0-4861-a82c-fd3377cf7752" path="/var/lib/kubelet/pods/1e2e6141-69a0-4861-a82c-fd3377cf7752/volumes" Jul 06 16:26:30.825387 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:30.825350 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 06 16:26:30.825887 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:30.825619 2584 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="387d0b57-b4b5-4acb-822e-32b3d3d5fa63" containerName="prometheus" containerID="cri-o://517ce8e7eb4151ac00876501d1782eefbc1ab7a3f3286461d80b3d4d9e7e0509" gracePeriod=600 Jul 06 16:26:30.825887 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:30.825654 2584 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="387d0b57-b4b5-4acb-822e-32b3d3d5fa63" containerName="kube-rbac-proxy" containerID="cri-o://43baa1bcfdc1b60520a09c3140c28661ec60c28abbbe4df3fce9fd0995303c6f" gracePeriod=600 Jul 06 16:26:30.825887 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:30.825681 2584 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="387d0b57-b4b5-4acb-822e-32b3d3d5fa63" containerName="kube-rbac-proxy-thanos" containerID="cri-o://57c8fa983b9d033a5af1c7a0dde1a2ae9ce2ceaf5536ccc1c3f23c8b3873d529" gracePeriod=600 Jul 06 16:26:30.825887 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:30.825687 2584 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="387d0b57-b4b5-4acb-822e-32b3d3d5fa63" containerName="thanos-sidecar" containerID="cri-o://4bc77931f9c26ade34e18fe206defb68e523052dbf84703999c77a2596bd8f35" gracePeriod=600 Jul 06 16:26:30.825887 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:30.825709 2584 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="387d0b57-b4b5-4acb-822e-32b3d3d5fa63" containerName="config-reloader" containerID="cri-o://f382f0aa8afeb3f46a0491928d22547f49ffae8e9e008acc03d8e87396753007" gracePeriod=600 Jul 06 16:26:30.825887 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:30.825843 2584 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="387d0b57-b4b5-4acb-822e-32b3d3d5fa63" containerName="kube-rbac-proxy-web" containerID="cri-o://a8df07321c9f43b846bcf3aa019f35d5af28fd9be2226ec9a8b50cea219f3b9c" gracePeriod=600 Jul 06 16:26:31.062946 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:31.062633 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 06 16:26:31.181172 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:31.181135 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/387d0b57-b4b5-4acb-822e-32b3d3d5fa63-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"387d0b57-b4b5-4acb-822e-32b3d3d5fa63\" (UID: \"387d0b57-b4b5-4acb-822e-32b3d3d5fa63\") " Jul 06 16:26:31.181375 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:31.181182 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/387d0b57-b4b5-4acb-822e-32b3d3d5fa63-prometheus-k8s-rulefiles-2\") pod \"387d0b57-b4b5-4acb-822e-32b3d3d5fa63\" (UID: \"387d0b57-b4b5-4acb-822e-32b3d3d5fa63\") " Jul 06 16:26:31.181375 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:31.181212 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/387d0b57-b4b5-4acb-822e-32b3d3d5fa63-web-config\") pod \"387d0b57-b4b5-4acb-822e-32b3d3d5fa63\" (UID: \"387d0b57-b4b5-4acb-822e-32b3d3d5fa63\") " Jul 06 16:26:31.181375 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:31.181234 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/387d0b57-b4b5-4acb-822e-32b3d3d5fa63-thanos-prometheus-http-client-file\") pod \"387d0b57-b4b5-4acb-822e-32b3d3d5fa63\" (UID: \"387d0b57-b4b5-4acb-822e-32b3d3d5fa63\") " Jul 06 16:26:31.181375 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:31.181274 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/387d0b57-b4b5-4acb-822e-32b3d3d5fa63-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"387d0b57-b4b5-4acb-822e-32b3d3d5fa63\" (UID: \"387d0b57-b4b5-4acb-822e-32b3d3d5fa63\") " Jul 06 16:26:31.181375 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:31.181297 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/387d0b57-b4b5-4acb-822e-32b3d3d5fa63-prometheus-k8s-db\") pod \"387d0b57-b4b5-4acb-822e-32b3d3d5fa63\" (UID: \"387d0b57-b4b5-4acb-822e-32b3d3d5fa63\") " Jul 06 16:26:31.181375 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:31.181326 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/387d0b57-b4b5-4acb-822e-32b3d3d5fa63-tls-assets\") pod \"387d0b57-b4b5-4acb-822e-32b3d3d5fa63\" (UID: \"387d0b57-b4b5-4acb-822e-32b3d3d5fa63\") " Jul 06 16:26:31.181375 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:31.181373 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/387d0b57-b4b5-4acb-822e-32b3d3d5fa63-configmap-serving-certs-ca-bundle\") pod \"387d0b57-b4b5-4acb-822e-32b3d3d5fa63\" (UID: \"387d0b57-b4b5-4acb-822e-32b3d3d5fa63\") " Jul 06 16:26:31.181712 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:31.181400 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/387d0b57-b4b5-4acb-822e-32b3d3d5fa63-config-out\") pod \"387d0b57-b4b5-4acb-822e-32b3d3d5fa63\" (UID: \"387d0b57-b4b5-4acb-822e-32b3d3d5fa63\") " Jul 06 16:26:31.181712 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:31.181428 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/387d0b57-b4b5-4acb-822e-32b3d3d5fa63-secret-metrics-client-certs\") pod \"387d0b57-b4b5-4acb-822e-32b3d3d5fa63\" (UID: \"387d0b57-b4b5-4acb-822e-32b3d3d5fa63\") " Jul 06 16:26:31.181712 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:31.181457 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/387d0b57-b4b5-4acb-822e-32b3d3d5fa63-secret-kube-rbac-proxy\") pod \"387d0b57-b4b5-4acb-822e-32b3d3d5fa63\" (UID: \"387d0b57-b4b5-4acb-822e-32b3d3d5fa63\") " Jul 06 16:26:31.181712 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:31.181499 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/387d0b57-b4b5-4acb-822e-32b3d3d5fa63-configmap-kubelet-serving-ca-bundle\") pod \"387d0b57-b4b5-4acb-822e-32b3d3d5fa63\" (UID: \"387d0b57-b4b5-4acb-822e-32b3d3d5fa63\") " Jul 06 16:26:31.181712 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:31.181540 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/387d0b57-b4b5-4acb-822e-32b3d3d5fa63-prometheus-trusted-ca-bundle\") pod \"387d0b57-b4b5-4acb-822e-32b3d3d5fa63\" (UID: \"387d0b57-b4b5-4acb-822e-32b3d3d5fa63\") " Jul 06 16:26:31.181712 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:31.181570 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9sn27\" (UniqueName: \"kubernetes.io/projected/387d0b57-b4b5-4acb-822e-32b3d3d5fa63-kube-api-access-9sn27\") pod \"387d0b57-b4b5-4acb-822e-32b3d3d5fa63\" (UID: \"387d0b57-b4b5-4acb-822e-32b3d3d5fa63\") " Jul 06 16:26:31.181712 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:31.181587 2584 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/387d0b57-b4b5-4acb-822e-32b3d3d5fa63-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "387d0b57-b4b5-4acb-822e-32b3d3d5fa63" (UID: "387d0b57-b4b5-4acb-822e-32b3d3d5fa63"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 16:26:31.181712 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:31.181622 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/387d0b57-b4b5-4acb-822e-32b3d3d5fa63-config\") pod \"387d0b57-b4b5-4acb-822e-32b3d3d5fa63\" (UID: \"387d0b57-b4b5-4acb-822e-32b3d3d5fa63\") " Jul 06 16:26:31.181712 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:31.181654 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/387d0b57-b4b5-4acb-822e-32b3d3d5fa63-secret-grpc-tls\") pod \"387d0b57-b4b5-4acb-822e-32b3d3d5fa63\" (UID: \"387d0b57-b4b5-4acb-822e-32b3d3d5fa63\") " Jul 06 16:26:31.181712 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:31.181679 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/387d0b57-b4b5-4acb-822e-32b3d3d5fa63-prometheus-k8s-rulefiles-0\") pod \"387d0b57-b4b5-4acb-822e-32b3d3d5fa63\" (UID: \"387d0b57-b4b5-4acb-822e-32b3d3d5fa63\") " Jul 06 16:26:31.182203 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:31.181750 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/387d0b57-b4b5-4acb-822e-32b3d3d5fa63-configmap-metrics-client-ca\") pod \"387d0b57-b4b5-4acb-822e-32b3d3d5fa63\" (UID: \"387d0b57-b4b5-4acb-822e-32b3d3d5fa63\") " Jul 06 16:26:31.182203 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:31.181778 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/387d0b57-b4b5-4acb-822e-32b3d3d5fa63-prometheus-k8s-rulefiles-1\") pod \"387d0b57-b4b5-4acb-822e-32b3d3d5fa63\" (UID: \"387d0b57-b4b5-4acb-822e-32b3d3d5fa63\") " Jul 06 16:26:31.182203 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:31.181806 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/387d0b57-b4b5-4acb-822e-32b3d3d5fa63-secret-prometheus-k8s-tls\") pod \"387d0b57-b4b5-4acb-822e-32b3d3d5fa63\" (UID: \"387d0b57-b4b5-4acb-822e-32b3d3d5fa63\") " Jul 06 16:26:31.182203 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:31.182097 2584 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/387d0b57-b4b5-4acb-822e-32b3d3d5fa63-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-128-22.ec2.internal\" DevicePath \"\"" Jul 06 16:26:31.182203 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:31.182110 2584 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/387d0b57-b4b5-4acb-822e-32b3d3d5fa63-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "387d0b57-b4b5-4acb-822e-32b3d3d5fa63" (UID: "387d0b57-b4b5-4acb-822e-32b3d3d5fa63"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 16:26:31.182799 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:31.182506 2584 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/387d0b57-b4b5-4acb-822e-32b3d3d5fa63-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "387d0b57-b4b5-4acb-822e-32b3d3d5fa63" (UID: "387d0b57-b4b5-4acb-822e-32b3d3d5fa63"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 16:26:31.183894 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:31.183660 2584 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/387d0b57-b4b5-4acb-822e-32b3d3d5fa63-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "387d0b57-b4b5-4acb-822e-32b3d3d5fa63" (UID: "387d0b57-b4b5-4acb-822e-32b3d3d5fa63"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 16:26:31.184220 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:31.184084 2584 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/387d0b57-b4b5-4acb-822e-32b3d3d5fa63-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "387d0b57-b4b5-4acb-822e-32b3d3d5fa63" (UID: "387d0b57-b4b5-4acb-822e-32b3d3d5fa63"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 16:26:31.184318 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:31.184250 2584 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/387d0b57-b4b5-4acb-822e-32b3d3d5fa63-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "387d0b57-b4b5-4acb-822e-32b3d3d5fa63" (UID: "387d0b57-b4b5-4acb-822e-32b3d3d5fa63"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 16:26:31.184632 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:31.184494 2584 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/387d0b57-b4b5-4acb-822e-32b3d3d5fa63-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "387d0b57-b4b5-4acb-822e-32b3d3d5fa63" (UID: "387d0b57-b4b5-4acb-822e-32b3d3d5fa63"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 16:26:31.184632 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:31.184565 2584 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/387d0b57-b4b5-4acb-822e-32b3d3d5fa63-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "387d0b57-b4b5-4acb-822e-32b3d3d5fa63" (UID: "387d0b57-b4b5-4acb-822e-32b3d3d5fa63"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 16:26:31.185058 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:31.185030 2584 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/387d0b57-b4b5-4acb-822e-32b3d3d5fa63-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "387d0b57-b4b5-4acb-822e-32b3d3d5fa63" (UID: "387d0b57-b4b5-4acb-822e-32b3d3d5fa63"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 16:26:31.185275 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:31.185246 2584 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/387d0b57-b4b5-4acb-822e-32b3d3d5fa63-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "387d0b57-b4b5-4acb-822e-32b3d3d5fa63" (UID: "387d0b57-b4b5-4acb-822e-32b3d3d5fa63"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 16:26:31.185346 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:31.185289 2584 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/387d0b57-b4b5-4acb-822e-32b3d3d5fa63-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "387d0b57-b4b5-4acb-822e-32b3d3d5fa63" (UID: "387d0b57-b4b5-4acb-822e-32b3d3d5fa63"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 16:26:31.185537 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:31.185516 2584 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/387d0b57-b4b5-4acb-822e-32b3d3d5fa63-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "387d0b57-b4b5-4acb-822e-32b3d3d5fa63" (UID: "387d0b57-b4b5-4acb-822e-32b3d3d5fa63"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 16:26:31.185829 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:31.185804 2584 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/387d0b57-b4b5-4acb-822e-32b3d3d5fa63-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "387d0b57-b4b5-4acb-822e-32b3d3d5fa63" (UID: "387d0b57-b4b5-4acb-822e-32b3d3d5fa63"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 16:26:31.185895 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:31.185837 2584 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/387d0b57-b4b5-4acb-822e-32b3d3d5fa63-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "387d0b57-b4b5-4acb-822e-32b3d3d5fa63" (UID: "387d0b57-b4b5-4acb-822e-32b3d3d5fa63"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 16:26:31.186218 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:31.186181 2584 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/387d0b57-b4b5-4acb-822e-32b3d3d5fa63-config-out" (OuterVolumeSpecName: "config-out") pod "387d0b57-b4b5-4acb-822e-32b3d3d5fa63" (UID: "387d0b57-b4b5-4acb-822e-32b3d3d5fa63"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 16:26:31.186548 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:31.186518 2584 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/387d0b57-b4b5-4acb-822e-32b3d3d5fa63-kube-api-access-9sn27" (OuterVolumeSpecName: "kube-api-access-9sn27") pod "387d0b57-b4b5-4acb-822e-32b3d3d5fa63" (UID: "387d0b57-b4b5-4acb-822e-32b3d3d5fa63"). InnerVolumeSpecName "kube-api-access-9sn27". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 16:26:31.187302 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:31.187267 2584 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/387d0b57-b4b5-4acb-822e-32b3d3d5fa63-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "387d0b57-b4b5-4acb-822e-32b3d3d5fa63" (UID: "387d0b57-b4b5-4acb-822e-32b3d3d5fa63"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 16:26:31.187395 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:31.187381 2584 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/387d0b57-b4b5-4acb-822e-32b3d3d5fa63-config" (OuterVolumeSpecName: "config") pod "387d0b57-b4b5-4acb-822e-32b3d3d5fa63" (UID: "387d0b57-b4b5-4acb-822e-32b3d3d5fa63"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 16:26:31.187583 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:31.187557 2584 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/387d0b57-b4b5-4acb-822e-32b3d3d5fa63-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "387d0b57-b4b5-4acb-822e-32b3d3d5fa63" (UID: "387d0b57-b4b5-4acb-822e-32b3d3d5fa63"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 16:26:31.196977 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:31.196942 2584 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/387d0b57-b4b5-4acb-822e-32b3d3d5fa63-web-config" (OuterVolumeSpecName: "web-config") pod "387d0b57-b4b5-4acb-822e-32b3d3d5fa63" (UID: "387d0b57-b4b5-4acb-822e-32b3d3d5fa63"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 16:26:31.283200 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:31.283143 2584 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/387d0b57-b4b5-4acb-822e-32b3d3d5fa63-web-config\") on node \"ip-10-0-128-22.ec2.internal\" DevicePath \"\"" Jul 06 16:26:31.283200 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:31.283192 2584 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/387d0b57-b4b5-4acb-822e-32b3d3d5fa63-thanos-prometheus-http-client-file\") on node \"ip-10-0-128-22.ec2.internal\" DevicePath \"\"" Jul 06 16:26:31.283200 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:31.283208 2584 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/387d0b57-b4b5-4acb-822e-32b3d3d5fa63-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-128-22.ec2.internal\" DevicePath \"\"" Jul 06 16:26:31.283483 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:31.283223 2584 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/387d0b57-b4b5-4acb-822e-32b3d3d5fa63-prometheus-k8s-db\") on node \"ip-10-0-128-22.ec2.internal\" DevicePath \"\"" Jul 06 16:26:31.283483 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:31.283236 2584 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/387d0b57-b4b5-4acb-822e-32b3d3d5fa63-tls-assets\") on node \"ip-10-0-128-22.ec2.internal\" DevicePath \"\"" Jul 06 16:26:31.283483 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:31.283248 2584 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/387d0b57-b4b5-4acb-822e-32b3d3d5fa63-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-128-22.ec2.internal\" DevicePath \"\"" Jul 06 16:26:31.283483 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:31.283258 2584 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/387d0b57-b4b5-4acb-822e-32b3d3d5fa63-config-out\") on node \"ip-10-0-128-22.ec2.internal\" DevicePath \"\"" Jul 06 16:26:31.283483 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:31.283271 2584 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/387d0b57-b4b5-4acb-822e-32b3d3d5fa63-secret-metrics-client-certs\") on node \"ip-10-0-128-22.ec2.internal\" DevicePath \"\"" Jul 06 16:26:31.283483 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:31.283282 2584 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/387d0b57-b4b5-4acb-822e-32b3d3d5fa63-secret-kube-rbac-proxy\") on node \"ip-10-0-128-22.ec2.internal\" DevicePath \"\"" Jul 06 16:26:31.283483 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:31.283296 2584 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/387d0b57-b4b5-4acb-822e-32b3d3d5fa63-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-128-22.ec2.internal\" DevicePath \"\"" Jul 06 16:26:31.283483 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:31.283308 2584 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/387d0b57-b4b5-4acb-822e-32b3d3d5fa63-prometheus-trusted-ca-bundle\") on node \"ip-10-0-128-22.ec2.internal\" DevicePath \"\"" Jul 06 16:26:31.283483 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:31.283319 2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9sn27\" (UniqueName: \"kubernetes.io/projected/387d0b57-b4b5-4acb-822e-32b3d3d5fa63-kube-api-access-9sn27\") on node \"ip-10-0-128-22.ec2.internal\" DevicePath \"\"" Jul 06 16:26:31.283483 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:31.283330 2584 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/387d0b57-b4b5-4acb-822e-32b3d3d5fa63-config\") on node \"ip-10-0-128-22.ec2.internal\" DevicePath \"\"" Jul 06 16:26:31.283483 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:31.283341 2584 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/387d0b57-b4b5-4acb-822e-32b3d3d5fa63-secret-grpc-tls\") on node \"ip-10-0-128-22.ec2.internal\" DevicePath \"\"" Jul 06 16:26:31.283483 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:31.283354 2584 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/387d0b57-b4b5-4acb-822e-32b3d3d5fa63-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-128-22.ec2.internal\" DevicePath \"\"" Jul 06 16:26:31.283483 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:31.283367 2584 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/387d0b57-b4b5-4acb-822e-32b3d3d5fa63-configmap-metrics-client-ca\") on node \"ip-10-0-128-22.ec2.internal\" DevicePath \"\"" Jul 06 16:26:31.283483 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:31.283379 2584 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/387d0b57-b4b5-4acb-822e-32b3d3d5fa63-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-128-22.ec2.internal\" DevicePath \"\"" Jul 06 16:26:31.283483 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:31.283391 2584 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/387d0b57-b4b5-4acb-822e-32b3d3d5fa63-secret-prometheus-k8s-tls\") on node \"ip-10-0-128-22.ec2.internal\" DevicePath \"\"" Jul 06 16:26:31.283483 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:31.283406 2584 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/387d0b57-b4b5-4acb-822e-32b3d3d5fa63-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-128-22.ec2.internal\" DevicePath \"\"" Jul 06 16:26:31.832195 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:31.832156 2584 generic.go:356] "Generic (PLEG): container finished" podID="387d0b57-b4b5-4acb-822e-32b3d3d5fa63" containerID="57c8fa983b9d033a5af1c7a0dde1a2ae9ce2ceaf5536ccc1c3f23c8b3873d529" exitCode=0 Jul 06 16:26:31.832195 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:31.832188 2584 generic.go:356] "Generic (PLEG): container finished" podID="387d0b57-b4b5-4acb-822e-32b3d3d5fa63" containerID="43baa1bcfdc1b60520a09c3140c28661ec60c28abbbe4df3fce9fd0995303c6f" exitCode=0 Jul 06 16:26:31.832195 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:31.832198 2584 generic.go:356] "Generic (PLEG): container finished" podID="387d0b57-b4b5-4acb-822e-32b3d3d5fa63" containerID="a8df07321c9f43b846bcf3aa019f35d5af28fd9be2226ec9a8b50cea219f3b9c" exitCode=0 Jul 06 16:26:31.832642 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:31.832206 2584 generic.go:356] "Generic (PLEG): container finished" podID="387d0b57-b4b5-4acb-822e-32b3d3d5fa63" containerID="4bc77931f9c26ade34e18fe206defb68e523052dbf84703999c77a2596bd8f35" exitCode=0 Jul 06 16:26:31.832642 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:31.832215 2584 generic.go:356] "Generic (PLEG): container finished" podID="387d0b57-b4b5-4acb-822e-32b3d3d5fa63" containerID="f382f0aa8afeb3f46a0491928d22547f49ffae8e9e008acc03d8e87396753007" exitCode=0 Jul 06 16:26:31.832642 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:31.832223 2584 generic.go:356] "Generic (PLEG): container finished" podID="387d0b57-b4b5-4acb-822e-32b3d3d5fa63" containerID="517ce8e7eb4151ac00876501d1782eefbc1ab7a3f3286461d80b3d4d9e7e0509" exitCode=0 Jul 06 16:26:31.832642 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:31.832249 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 06 16:26:31.832642 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:31.832241 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"387d0b57-b4b5-4acb-822e-32b3d3d5fa63","Type":"ContainerDied","Data":"57c8fa983b9d033a5af1c7a0dde1a2ae9ce2ceaf5536ccc1c3f23c8b3873d529"} Jul 06 16:26:31.832642 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:31.832376 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"387d0b57-b4b5-4acb-822e-32b3d3d5fa63","Type":"ContainerDied","Data":"43baa1bcfdc1b60520a09c3140c28661ec60c28abbbe4df3fce9fd0995303c6f"} Jul 06 16:26:31.832642 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:31.832403 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"387d0b57-b4b5-4acb-822e-32b3d3d5fa63","Type":"ContainerDied","Data":"a8df07321c9f43b846bcf3aa019f35d5af28fd9be2226ec9a8b50cea219f3b9c"} Jul 06 16:26:31.832642 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:31.832427 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"387d0b57-b4b5-4acb-822e-32b3d3d5fa63","Type":"ContainerDied","Data":"4bc77931f9c26ade34e18fe206defb68e523052dbf84703999c77a2596bd8f35"} Jul 06 16:26:31.832642 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:31.832447 2584 scope.go:117] "RemoveContainer" containerID="57c8fa983b9d033a5af1c7a0dde1a2ae9ce2ceaf5536ccc1c3f23c8b3873d529" Jul 06 16:26:31.832642 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:31.832452 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"387d0b57-b4b5-4acb-822e-32b3d3d5fa63","Type":"ContainerDied","Data":"f382f0aa8afeb3f46a0491928d22547f49ffae8e9e008acc03d8e87396753007"} Jul 06 16:26:31.832642 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:31.832545 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"387d0b57-b4b5-4acb-822e-32b3d3d5fa63","Type":"ContainerDied","Data":"517ce8e7eb4151ac00876501d1782eefbc1ab7a3f3286461d80b3d4d9e7e0509"} Jul 06 16:26:31.832642 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:31.832573 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"387d0b57-b4b5-4acb-822e-32b3d3d5fa63","Type":"ContainerDied","Data":"9e3ff11885189848dcc73252b75dccf3ced4c12a23860c984c90164159b64294"} Jul 06 16:26:31.840468 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:31.840451 2584 scope.go:117] "RemoveContainer" containerID="43baa1bcfdc1b60520a09c3140c28661ec60c28abbbe4df3fce9fd0995303c6f" Jul 06 16:26:31.855673 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:31.855655 2584 scope.go:117] "RemoveContainer" containerID="a8df07321c9f43b846bcf3aa019f35d5af28fd9be2226ec9a8b50cea219f3b9c" Jul 06 16:26:31.862921 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:31.862903 2584 scope.go:117] "RemoveContainer" containerID="4bc77931f9c26ade34e18fe206defb68e523052dbf84703999c77a2596bd8f35" Jul 06 16:26:31.872054 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:31.871947 2584 scope.go:117] "RemoveContainer" containerID="f382f0aa8afeb3f46a0491928d22547f49ffae8e9e008acc03d8e87396753007" Jul 06 16:26:31.872658 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:31.872640 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 06 16:26:31.879304 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:31.879283 2584 scope.go:117] "RemoveContainer" containerID="517ce8e7eb4151ac00876501d1782eefbc1ab7a3f3286461d80b3d4d9e7e0509" Jul 06 16:26:31.881025 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:31.881008 2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 06 16:26:31.886735 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:31.886706 2584 scope.go:117] "RemoveContainer" containerID="5db1d4dc73ea0864daa2c9c02f7fa6f2c1d97d81a49818a4dc57ad6a62c9ee01" Jul 06 16:26:31.893809 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:31.893786 2584 scope.go:117] "RemoveContainer" containerID="57c8fa983b9d033a5af1c7a0dde1a2ae9ce2ceaf5536ccc1c3f23c8b3873d529" Jul 06 16:26:31.894132 ip-10-0-128-22 kubenswrapper[2584]: E0706 16:26:31.894114 2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"57c8fa983b9d033a5af1c7a0dde1a2ae9ce2ceaf5536ccc1c3f23c8b3873d529\": container with ID starting with 57c8fa983b9d033a5af1c7a0dde1a2ae9ce2ceaf5536ccc1c3f23c8b3873d529 not found: ID does not exist" containerID="57c8fa983b9d033a5af1c7a0dde1a2ae9ce2ceaf5536ccc1c3f23c8b3873d529" Jul 06 16:26:31.894187 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:31.894142 2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"57c8fa983b9d033a5af1c7a0dde1a2ae9ce2ceaf5536ccc1c3f23c8b3873d529"} err="failed to get container status \"57c8fa983b9d033a5af1c7a0dde1a2ae9ce2ceaf5536ccc1c3f23c8b3873d529\": rpc error: code = NotFound desc = could not find container \"57c8fa983b9d033a5af1c7a0dde1a2ae9ce2ceaf5536ccc1c3f23c8b3873d529\": container with ID starting with 57c8fa983b9d033a5af1c7a0dde1a2ae9ce2ceaf5536ccc1c3f23c8b3873d529 not found: ID does not exist" Jul 06 16:26:31.894187 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:31.894166 2584 scope.go:117] "RemoveContainer" containerID="43baa1bcfdc1b60520a09c3140c28661ec60c28abbbe4df3fce9fd0995303c6f" Jul 06 16:26:31.894415 ip-10-0-128-22 kubenswrapper[2584]: E0706 16:26:31.894396 2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"43baa1bcfdc1b60520a09c3140c28661ec60c28abbbe4df3fce9fd0995303c6f\": container with ID starting with 43baa1bcfdc1b60520a09c3140c28661ec60c28abbbe4df3fce9fd0995303c6f not found: ID does not exist" containerID="43baa1bcfdc1b60520a09c3140c28661ec60c28abbbe4df3fce9fd0995303c6f" Jul 06 16:26:31.894461 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:31.894426 2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"43baa1bcfdc1b60520a09c3140c28661ec60c28abbbe4df3fce9fd0995303c6f"} err="failed to get container status \"43baa1bcfdc1b60520a09c3140c28661ec60c28abbbe4df3fce9fd0995303c6f\": rpc error: code = NotFound desc = could not find container \"43baa1bcfdc1b60520a09c3140c28661ec60c28abbbe4df3fce9fd0995303c6f\": container with ID starting with 43baa1bcfdc1b60520a09c3140c28661ec60c28abbbe4df3fce9fd0995303c6f not found: ID does not exist" Jul 06 16:26:31.894461 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:31.894443 2584 scope.go:117] "RemoveContainer" containerID="a8df07321c9f43b846bcf3aa019f35d5af28fd9be2226ec9a8b50cea219f3b9c" Jul 06 16:26:31.894686 ip-10-0-128-22 kubenswrapper[2584]: E0706 16:26:31.894669 2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a8df07321c9f43b846bcf3aa019f35d5af28fd9be2226ec9a8b50cea219f3b9c\": container with ID starting with a8df07321c9f43b846bcf3aa019f35d5af28fd9be2226ec9a8b50cea219f3b9c not found: ID does not exist" containerID="a8df07321c9f43b846bcf3aa019f35d5af28fd9be2226ec9a8b50cea219f3b9c" Jul 06 16:26:31.894766 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:31.894689 2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a8df07321c9f43b846bcf3aa019f35d5af28fd9be2226ec9a8b50cea219f3b9c"} err="failed to get container status \"a8df07321c9f43b846bcf3aa019f35d5af28fd9be2226ec9a8b50cea219f3b9c\": rpc error: code = NotFound desc = could not find container \"a8df07321c9f43b846bcf3aa019f35d5af28fd9be2226ec9a8b50cea219f3b9c\": container with ID starting with a8df07321c9f43b846bcf3aa019f35d5af28fd9be2226ec9a8b50cea219f3b9c not found: ID does not exist" Jul 06 16:26:31.894766 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:31.894704 2584 scope.go:117] "RemoveContainer" containerID="4bc77931f9c26ade34e18fe206defb68e523052dbf84703999c77a2596bd8f35" Jul 06 16:26:31.894990 ip-10-0-128-22 kubenswrapper[2584]: E0706 16:26:31.894973 2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4bc77931f9c26ade34e18fe206defb68e523052dbf84703999c77a2596bd8f35\": container with ID starting with 4bc77931f9c26ade34e18fe206defb68e523052dbf84703999c77a2596bd8f35 not found: ID does not exist" containerID="4bc77931f9c26ade34e18fe206defb68e523052dbf84703999c77a2596bd8f35" Jul 06 16:26:31.895067 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:31.894999 2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4bc77931f9c26ade34e18fe206defb68e523052dbf84703999c77a2596bd8f35"} err="failed to get container status \"4bc77931f9c26ade34e18fe206defb68e523052dbf84703999c77a2596bd8f35\": rpc error: code = NotFound desc = could not find container \"4bc77931f9c26ade34e18fe206defb68e523052dbf84703999c77a2596bd8f35\": container with ID starting with 4bc77931f9c26ade34e18fe206defb68e523052dbf84703999c77a2596bd8f35 not found: ID does not exist" Jul 06 16:26:31.895067 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:31.895014 2584 scope.go:117] "RemoveContainer" containerID="f382f0aa8afeb3f46a0491928d22547f49ffae8e9e008acc03d8e87396753007" Jul 06 16:26:31.895214 ip-10-0-128-22 kubenswrapper[2584]: E0706 16:26:31.895199 2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f382f0aa8afeb3f46a0491928d22547f49ffae8e9e008acc03d8e87396753007\": container with ID starting with f382f0aa8afeb3f46a0491928d22547f49ffae8e9e008acc03d8e87396753007 not found: ID does not exist" containerID="f382f0aa8afeb3f46a0491928d22547f49ffae8e9e008acc03d8e87396753007" Jul 06 16:26:31.895253 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:31.895217 2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f382f0aa8afeb3f46a0491928d22547f49ffae8e9e008acc03d8e87396753007"} err="failed to get container status \"f382f0aa8afeb3f46a0491928d22547f49ffae8e9e008acc03d8e87396753007\": rpc error: code = NotFound desc = could not find container \"f382f0aa8afeb3f46a0491928d22547f49ffae8e9e008acc03d8e87396753007\": container with ID starting with f382f0aa8afeb3f46a0491928d22547f49ffae8e9e008acc03d8e87396753007 not found: ID does not exist" Jul 06 16:26:31.895253 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:31.895230 2584 scope.go:117] "RemoveContainer" containerID="517ce8e7eb4151ac00876501d1782eefbc1ab7a3f3286461d80b3d4d9e7e0509" Jul 06 16:26:31.895419 ip-10-0-128-22 kubenswrapper[2584]: E0706 16:26:31.895403 2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"517ce8e7eb4151ac00876501d1782eefbc1ab7a3f3286461d80b3d4d9e7e0509\": container with ID starting with 517ce8e7eb4151ac00876501d1782eefbc1ab7a3f3286461d80b3d4d9e7e0509 not found: ID does not exist" containerID="517ce8e7eb4151ac00876501d1782eefbc1ab7a3f3286461d80b3d4d9e7e0509" Jul 06 16:26:31.895458 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:31.895422 2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"517ce8e7eb4151ac00876501d1782eefbc1ab7a3f3286461d80b3d4d9e7e0509"} err="failed to get container status \"517ce8e7eb4151ac00876501d1782eefbc1ab7a3f3286461d80b3d4d9e7e0509\": rpc error: code = NotFound desc = could not find container \"517ce8e7eb4151ac00876501d1782eefbc1ab7a3f3286461d80b3d4d9e7e0509\": container with ID starting with 517ce8e7eb4151ac00876501d1782eefbc1ab7a3f3286461d80b3d4d9e7e0509 not found: ID does not exist" Jul 06 16:26:31.895458 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:31.895436 2584 scope.go:117] "RemoveContainer" containerID="5db1d4dc73ea0864daa2c9c02f7fa6f2c1d97d81a49818a4dc57ad6a62c9ee01" Jul 06 16:26:31.895607 ip-10-0-128-22 kubenswrapper[2584]: E0706 16:26:31.895594 2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5db1d4dc73ea0864daa2c9c02f7fa6f2c1d97d81a49818a4dc57ad6a62c9ee01\": container with ID starting with 5db1d4dc73ea0864daa2c9c02f7fa6f2c1d97d81a49818a4dc57ad6a62c9ee01 not found: ID does not exist" containerID="5db1d4dc73ea0864daa2c9c02f7fa6f2c1d97d81a49818a4dc57ad6a62c9ee01" Jul 06 16:26:31.895649 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:31.895609 2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5db1d4dc73ea0864daa2c9c02f7fa6f2c1d97d81a49818a4dc57ad6a62c9ee01"} err="failed to get container status \"5db1d4dc73ea0864daa2c9c02f7fa6f2c1d97d81a49818a4dc57ad6a62c9ee01\": rpc error: code = NotFound desc = could not find container \"5db1d4dc73ea0864daa2c9c02f7fa6f2c1d97d81a49818a4dc57ad6a62c9ee01\": container with ID starting with 5db1d4dc73ea0864daa2c9c02f7fa6f2c1d97d81a49818a4dc57ad6a62c9ee01 not found: ID does not exist" Jul 06 16:26:31.895649 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:31.895619 2584 scope.go:117] "RemoveContainer" containerID="57c8fa983b9d033a5af1c7a0dde1a2ae9ce2ceaf5536ccc1c3f23c8b3873d529" Jul 06 16:26:31.895826 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:31.895807 2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"57c8fa983b9d033a5af1c7a0dde1a2ae9ce2ceaf5536ccc1c3f23c8b3873d529"} err="failed to get container status \"57c8fa983b9d033a5af1c7a0dde1a2ae9ce2ceaf5536ccc1c3f23c8b3873d529\": rpc error: code = NotFound desc = could not find container \"57c8fa983b9d033a5af1c7a0dde1a2ae9ce2ceaf5536ccc1c3f23c8b3873d529\": container with ID starting with 57c8fa983b9d033a5af1c7a0dde1a2ae9ce2ceaf5536ccc1c3f23c8b3873d529 not found: ID does not exist" Jul 06 16:26:31.895865 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:31.895827 2584 scope.go:117] "RemoveContainer" containerID="43baa1bcfdc1b60520a09c3140c28661ec60c28abbbe4df3fce9fd0995303c6f" Jul 06 16:26:31.896012 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:31.895993 2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"43baa1bcfdc1b60520a09c3140c28661ec60c28abbbe4df3fce9fd0995303c6f"} err="failed to get container status \"43baa1bcfdc1b60520a09c3140c28661ec60c28abbbe4df3fce9fd0995303c6f\": rpc error: code = NotFound desc = could not find container \"43baa1bcfdc1b60520a09c3140c28661ec60c28abbbe4df3fce9fd0995303c6f\": container with ID starting with 43baa1bcfdc1b60520a09c3140c28661ec60c28abbbe4df3fce9fd0995303c6f not found: ID does not exist" Jul 06 16:26:31.896012 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:31.896011 2584 scope.go:117] "RemoveContainer" containerID="a8df07321c9f43b846bcf3aa019f35d5af28fd9be2226ec9a8b50cea219f3b9c" Jul 06 16:26:31.896213 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:31.896195 2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a8df07321c9f43b846bcf3aa019f35d5af28fd9be2226ec9a8b50cea219f3b9c"} err="failed to get container status \"a8df07321c9f43b846bcf3aa019f35d5af28fd9be2226ec9a8b50cea219f3b9c\": rpc error: code = NotFound desc = could not find container \"a8df07321c9f43b846bcf3aa019f35d5af28fd9be2226ec9a8b50cea219f3b9c\": container with ID starting with a8df07321c9f43b846bcf3aa019f35d5af28fd9be2226ec9a8b50cea219f3b9c not found: ID does not exist" Jul 06 16:26:31.896265 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:31.896214 2584 scope.go:117] "RemoveContainer" containerID="4bc77931f9c26ade34e18fe206defb68e523052dbf84703999c77a2596bd8f35" Jul 06 16:26:31.896451 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:31.896434 2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4bc77931f9c26ade34e18fe206defb68e523052dbf84703999c77a2596bd8f35"} err="failed to get container status \"4bc77931f9c26ade34e18fe206defb68e523052dbf84703999c77a2596bd8f35\": rpc error: code = NotFound desc = could not find container \"4bc77931f9c26ade34e18fe206defb68e523052dbf84703999c77a2596bd8f35\": container with ID starting with 4bc77931f9c26ade34e18fe206defb68e523052dbf84703999c77a2596bd8f35 not found: ID does not exist" Jul 06 16:26:31.896505 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:31.896453 2584 scope.go:117] "RemoveContainer" containerID="f382f0aa8afeb3f46a0491928d22547f49ffae8e9e008acc03d8e87396753007" Jul 06 16:26:31.896650 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:31.896630 2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f382f0aa8afeb3f46a0491928d22547f49ffae8e9e008acc03d8e87396753007"} err="failed to get container status \"f382f0aa8afeb3f46a0491928d22547f49ffae8e9e008acc03d8e87396753007\": rpc error: code = NotFound desc = could not find container \"f382f0aa8afeb3f46a0491928d22547f49ffae8e9e008acc03d8e87396753007\": container with ID starting with f382f0aa8afeb3f46a0491928d22547f49ffae8e9e008acc03d8e87396753007 not found: ID does not exist" Jul 06 16:26:31.896694 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:31.896652 2584 scope.go:117] "RemoveContainer" containerID="517ce8e7eb4151ac00876501d1782eefbc1ab7a3f3286461d80b3d4d9e7e0509" Jul 06 16:26:31.896916 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:31.896898 2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"517ce8e7eb4151ac00876501d1782eefbc1ab7a3f3286461d80b3d4d9e7e0509"} err="failed to get container status \"517ce8e7eb4151ac00876501d1782eefbc1ab7a3f3286461d80b3d4d9e7e0509\": rpc error: code = NotFound desc = could not find container \"517ce8e7eb4151ac00876501d1782eefbc1ab7a3f3286461d80b3d4d9e7e0509\": container with ID starting with 517ce8e7eb4151ac00876501d1782eefbc1ab7a3f3286461d80b3d4d9e7e0509 not found: ID does not exist" Jul 06 16:26:31.896916 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:31.896915 2584 scope.go:117] "RemoveContainer" containerID="5db1d4dc73ea0864daa2c9c02f7fa6f2c1d97d81a49818a4dc57ad6a62c9ee01" Jul 06 16:26:31.897131 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:31.897114 2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5db1d4dc73ea0864daa2c9c02f7fa6f2c1d97d81a49818a4dc57ad6a62c9ee01"} err="failed to get container status \"5db1d4dc73ea0864daa2c9c02f7fa6f2c1d97d81a49818a4dc57ad6a62c9ee01\": rpc error: code = NotFound desc = could not find container \"5db1d4dc73ea0864daa2c9c02f7fa6f2c1d97d81a49818a4dc57ad6a62c9ee01\": container with ID starting with 5db1d4dc73ea0864daa2c9c02f7fa6f2c1d97d81a49818a4dc57ad6a62c9ee01 not found: ID does not exist" Jul 06 16:26:31.897169 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:31.897132 2584 scope.go:117] "RemoveContainer" containerID="57c8fa983b9d033a5af1c7a0dde1a2ae9ce2ceaf5536ccc1c3f23c8b3873d529" Jul 06 16:26:31.897326 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:31.897309 2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"57c8fa983b9d033a5af1c7a0dde1a2ae9ce2ceaf5536ccc1c3f23c8b3873d529"} err="failed to get container status \"57c8fa983b9d033a5af1c7a0dde1a2ae9ce2ceaf5536ccc1c3f23c8b3873d529\": rpc error: code = NotFound desc = could not find container \"57c8fa983b9d033a5af1c7a0dde1a2ae9ce2ceaf5536ccc1c3f23c8b3873d529\": container with ID starting with 57c8fa983b9d033a5af1c7a0dde1a2ae9ce2ceaf5536ccc1c3f23c8b3873d529 not found: ID does not exist" Jul 06 16:26:31.897367 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:31.897326 2584 scope.go:117] "RemoveContainer" containerID="43baa1bcfdc1b60520a09c3140c28661ec60c28abbbe4df3fce9fd0995303c6f" Jul 06 16:26:31.897518 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:31.897498 2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"43baa1bcfdc1b60520a09c3140c28661ec60c28abbbe4df3fce9fd0995303c6f"} err="failed to get container status \"43baa1bcfdc1b60520a09c3140c28661ec60c28abbbe4df3fce9fd0995303c6f\": rpc error: code = NotFound desc = could not find container \"43baa1bcfdc1b60520a09c3140c28661ec60c28abbbe4df3fce9fd0995303c6f\": container with ID starting with 43baa1bcfdc1b60520a09c3140c28661ec60c28abbbe4df3fce9fd0995303c6f not found: ID does not exist" Jul 06 16:26:31.897556 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:31.897519 2584 scope.go:117] "RemoveContainer" containerID="a8df07321c9f43b846bcf3aa019f35d5af28fd9be2226ec9a8b50cea219f3b9c" Jul 06 16:26:31.897735 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:31.897704 2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a8df07321c9f43b846bcf3aa019f35d5af28fd9be2226ec9a8b50cea219f3b9c"} err="failed to get container status \"a8df07321c9f43b846bcf3aa019f35d5af28fd9be2226ec9a8b50cea219f3b9c\": rpc error: code = NotFound desc = could not find container \"a8df07321c9f43b846bcf3aa019f35d5af28fd9be2226ec9a8b50cea219f3b9c\": container with ID starting with a8df07321c9f43b846bcf3aa019f35d5af28fd9be2226ec9a8b50cea219f3b9c not found: ID does not exist" Jul 06 16:26:31.897786 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:31.897738 2584 scope.go:117] "RemoveContainer" containerID="4bc77931f9c26ade34e18fe206defb68e523052dbf84703999c77a2596bd8f35" Jul 06 16:26:31.897950 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:31.897935 2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4bc77931f9c26ade34e18fe206defb68e523052dbf84703999c77a2596bd8f35"} err="failed to get container status \"4bc77931f9c26ade34e18fe206defb68e523052dbf84703999c77a2596bd8f35\": rpc error: code = NotFound desc = could not find container \"4bc77931f9c26ade34e18fe206defb68e523052dbf84703999c77a2596bd8f35\": container with ID starting with 4bc77931f9c26ade34e18fe206defb68e523052dbf84703999c77a2596bd8f35 not found: ID does not exist" Jul 06 16:26:31.897994 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:31.897950 2584 scope.go:117] "RemoveContainer" containerID="f382f0aa8afeb3f46a0491928d22547f49ffae8e9e008acc03d8e87396753007" Jul 06 16:26:31.898218 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:31.898199 2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f382f0aa8afeb3f46a0491928d22547f49ffae8e9e008acc03d8e87396753007"} err="failed to get container status \"f382f0aa8afeb3f46a0491928d22547f49ffae8e9e008acc03d8e87396753007\": rpc error: code = NotFound desc = could not find container \"f382f0aa8afeb3f46a0491928d22547f49ffae8e9e008acc03d8e87396753007\": container with ID starting with f382f0aa8afeb3f46a0491928d22547f49ffae8e9e008acc03d8e87396753007 not found: ID does not exist" Jul 06 16:26:31.898282 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:31.898218 2584 scope.go:117] "RemoveContainer" containerID="517ce8e7eb4151ac00876501d1782eefbc1ab7a3f3286461d80b3d4d9e7e0509" Jul 06 16:26:31.898479 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:31.898456 2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"517ce8e7eb4151ac00876501d1782eefbc1ab7a3f3286461d80b3d4d9e7e0509"} err="failed to get container status \"517ce8e7eb4151ac00876501d1782eefbc1ab7a3f3286461d80b3d4d9e7e0509\": rpc error: code = NotFound desc = could not find container \"517ce8e7eb4151ac00876501d1782eefbc1ab7a3f3286461d80b3d4d9e7e0509\": container with ID starting with 517ce8e7eb4151ac00876501d1782eefbc1ab7a3f3286461d80b3d4d9e7e0509 not found: ID does not exist" Jul 06 16:26:31.898524 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:31.898480 2584 scope.go:117] "RemoveContainer" containerID="5db1d4dc73ea0864daa2c9c02f7fa6f2c1d97d81a49818a4dc57ad6a62c9ee01" Jul 06 16:26:31.898749 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:31.898697 2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5db1d4dc73ea0864daa2c9c02f7fa6f2c1d97d81a49818a4dc57ad6a62c9ee01"} err="failed to get container status \"5db1d4dc73ea0864daa2c9c02f7fa6f2c1d97d81a49818a4dc57ad6a62c9ee01\": rpc error: code = NotFound desc = could not find container \"5db1d4dc73ea0864daa2c9c02f7fa6f2c1d97d81a49818a4dc57ad6a62c9ee01\": container with ID starting with 5db1d4dc73ea0864daa2c9c02f7fa6f2c1d97d81a49818a4dc57ad6a62c9ee01 not found: ID does not exist" Jul 06 16:26:31.898749 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:31.898716 2584 scope.go:117] "RemoveContainer" containerID="57c8fa983b9d033a5af1c7a0dde1a2ae9ce2ceaf5536ccc1c3f23c8b3873d529" Jul 06 16:26:31.899071 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:31.899052 2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"57c8fa983b9d033a5af1c7a0dde1a2ae9ce2ceaf5536ccc1c3f23c8b3873d529"} err="failed to get container status \"57c8fa983b9d033a5af1c7a0dde1a2ae9ce2ceaf5536ccc1c3f23c8b3873d529\": rpc error: code = NotFound desc = could not find container \"57c8fa983b9d033a5af1c7a0dde1a2ae9ce2ceaf5536ccc1c3f23c8b3873d529\": container with ID starting with 57c8fa983b9d033a5af1c7a0dde1a2ae9ce2ceaf5536ccc1c3f23c8b3873d529 not found: ID does not exist" Jul 06 16:26:31.899128 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:31.899072 2584 scope.go:117] "RemoveContainer" containerID="43baa1bcfdc1b60520a09c3140c28661ec60c28abbbe4df3fce9fd0995303c6f" Jul 06 16:26:31.899375 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:31.899357 2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"43baa1bcfdc1b60520a09c3140c28661ec60c28abbbe4df3fce9fd0995303c6f"} err="failed to get container status \"43baa1bcfdc1b60520a09c3140c28661ec60c28abbbe4df3fce9fd0995303c6f\": rpc error: code = NotFound desc = could not find container \"43baa1bcfdc1b60520a09c3140c28661ec60c28abbbe4df3fce9fd0995303c6f\": container with ID starting with 43baa1bcfdc1b60520a09c3140c28661ec60c28abbbe4df3fce9fd0995303c6f not found: ID does not exist" Jul 06 16:26:31.899477 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:31.899375 2584 scope.go:117] "RemoveContainer" containerID="a8df07321c9f43b846bcf3aa019f35d5af28fd9be2226ec9a8b50cea219f3b9c" Jul 06 16:26:31.899585 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:31.899570 2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a8df07321c9f43b846bcf3aa019f35d5af28fd9be2226ec9a8b50cea219f3b9c"} err="failed to get container status \"a8df07321c9f43b846bcf3aa019f35d5af28fd9be2226ec9a8b50cea219f3b9c\": rpc error: code = NotFound desc = could not find container \"a8df07321c9f43b846bcf3aa019f35d5af28fd9be2226ec9a8b50cea219f3b9c\": container with ID starting with a8df07321c9f43b846bcf3aa019f35d5af28fd9be2226ec9a8b50cea219f3b9c not found: ID does not exist" Jul 06 16:26:31.899621 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:31.899586 2584 scope.go:117] "RemoveContainer" containerID="4bc77931f9c26ade34e18fe206defb68e523052dbf84703999c77a2596bd8f35" Jul 06 16:26:31.899786 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:31.899767 2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4bc77931f9c26ade34e18fe206defb68e523052dbf84703999c77a2596bd8f35"} err="failed to get container status \"4bc77931f9c26ade34e18fe206defb68e523052dbf84703999c77a2596bd8f35\": rpc error: code = NotFound desc = could not find container \"4bc77931f9c26ade34e18fe206defb68e523052dbf84703999c77a2596bd8f35\": container with ID starting with 4bc77931f9c26ade34e18fe206defb68e523052dbf84703999c77a2596bd8f35 not found: ID does not exist" Jul 06 16:26:31.899829 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:31.899789 2584 scope.go:117] "RemoveContainer" containerID="f382f0aa8afeb3f46a0491928d22547f49ffae8e9e008acc03d8e87396753007" Jul 06 16:26:31.900018 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:31.899997 2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f382f0aa8afeb3f46a0491928d22547f49ffae8e9e008acc03d8e87396753007"} err="failed to get container status \"f382f0aa8afeb3f46a0491928d22547f49ffae8e9e008acc03d8e87396753007\": rpc error: code = NotFound desc = could not find container \"f382f0aa8afeb3f46a0491928d22547f49ffae8e9e008acc03d8e87396753007\": container with ID starting with f382f0aa8afeb3f46a0491928d22547f49ffae8e9e008acc03d8e87396753007 not found: ID does not exist" Jul 06 16:26:31.900092 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:31.900020 2584 scope.go:117] "RemoveContainer" containerID="517ce8e7eb4151ac00876501d1782eefbc1ab7a3f3286461d80b3d4d9e7e0509" Jul 06 16:26:31.900233 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:31.900209 2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"517ce8e7eb4151ac00876501d1782eefbc1ab7a3f3286461d80b3d4d9e7e0509"} err="failed to get container status \"517ce8e7eb4151ac00876501d1782eefbc1ab7a3f3286461d80b3d4d9e7e0509\": rpc error: code = NotFound desc = could not find container \"517ce8e7eb4151ac00876501d1782eefbc1ab7a3f3286461d80b3d4d9e7e0509\": container with ID starting with 517ce8e7eb4151ac00876501d1782eefbc1ab7a3f3286461d80b3d4d9e7e0509 not found: ID does not exist" Jul 06 16:26:31.900233 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:31.900232 2584 scope.go:117] "RemoveContainer" containerID="5db1d4dc73ea0864daa2c9c02f7fa6f2c1d97d81a49818a4dc57ad6a62c9ee01" Jul 06 16:26:31.900472 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:31.900454 2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5db1d4dc73ea0864daa2c9c02f7fa6f2c1d97d81a49818a4dc57ad6a62c9ee01"} err="failed to get container status \"5db1d4dc73ea0864daa2c9c02f7fa6f2c1d97d81a49818a4dc57ad6a62c9ee01\": rpc error: code = NotFound desc = could not find container \"5db1d4dc73ea0864daa2c9c02f7fa6f2c1d97d81a49818a4dc57ad6a62c9ee01\": container with ID starting with 5db1d4dc73ea0864daa2c9c02f7fa6f2c1d97d81a49818a4dc57ad6a62c9ee01 not found: ID does not exist" Jul 06 16:26:31.900519 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:31.900474 2584 scope.go:117] "RemoveContainer" containerID="57c8fa983b9d033a5af1c7a0dde1a2ae9ce2ceaf5536ccc1c3f23c8b3873d529" Jul 06 16:26:31.900682 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:31.900664 2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"57c8fa983b9d033a5af1c7a0dde1a2ae9ce2ceaf5536ccc1c3f23c8b3873d529"} err="failed to get container status \"57c8fa983b9d033a5af1c7a0dde1a2ae9ce2ceaf5536ccc1c3f23c8b3873d529\": rpc error: code = NotFound desc = could not find container \"57c8fa983b9d033a5af1c7a0dde1a2ae9ce2ceaf5536ccc1c3f23c8b3873d529\": container with ID starting with 57c8fa983b9d033a5af1c7a0dde1a2ae9ce2ceaf5536ccc1c3f23c8b3873d529 not found: ID does not exist" Jul 06 16:26:31.900739 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:31.900682 2584 scope.go:117] "RemoveContainer" containerID="43baa1bcfdc1b60520a09c3140c28661ec60c28abbbe4df3fce9fd0995303c6f" Jul 06 16:26:31.900926 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:31.900907 2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"43baa1bcfdc1b60520a09c3140c28661ec60c28abbbe4df3fce9fd0995303c6f"} err="failed to get container status \"43baa1bcfdc1b60520a09c3140c28661ec60c28abbbe4df3fce9fd0995303c6f\": rpc error: code = NotFound desc = could not find container \"43baa1bcfdc1b60520a09c3140c28661ec60c28abbbe4df3fce9fd0995303c6f\": container with ID starting with 43baa1bcfdc1b60520a09c3140c28661ec60c28abbbe4df3fce9fd0995303c6f not found: ID does not exist" Jul 06 16:26:31.900994 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:31.900928 2584 scope.go:117] "RemoveContainer" containerID="a8df07321c9f43b846bcf3aa019f35d5af28fd9be2226ec9a8b50cea219f3b9c" Jul 06 16:26:31.901136 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:31.901121 2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a8df07321c9f43b846bcf3aa019f35d5af28fd9be2226ec9a8b50cea219f3b9c"} err="failed to get container status \"a8df07321c9f43b846bcf3aa019f35d5af28fd9be2226ec9a8b50cea219f3b9c\": rpc error: code = NotFound desc = could not find container \"a8df07321c9f43b846bcf3aa019f35d5af28fd9be2226ec9a8b50cea219f3b9c\": container with ID starting with a8df07321c9f43b846bcf3aa019f35d5af28fd9be2226ec9a8b50cea219f3b9c not found: ID does not exist" Jul 06 16:26:31.901189 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:31.901136 2584 scope.go:117] "RemoveContainer" containerID="4bc77931f9c26ade34e18fe206defb68e523052dbf84703999c77a2596bd8f35" Jul 06 16:26:31.901368 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:31.901352 2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4bc77931f9c26ade34e18fe206defb68e523052dbf84703999c77a2596bd8f35"} err="failed to get container status \"4bc77931f9c26ade34e18fe206defb68e523052dbf84703999c77a2596bd8f35\": rpc error: code = NotFound desc = could not find container \"4bc77931f9c26ade34e18fe206defb68e523052dbf84703999c77a2596bd8f35\": container with ID starting with 4bc77931f9c26ade34e18fe206defb68e523052dbf84703999c77a2596bd8f35 not found: ID does not exist" Jul 06 16:26:31.901428 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:31.901370 2584 scope.go:117] "RemoveContainer" containerID="f382f0aa8afeb3f46a0491928d22547f49ffae8e9e008acc03d8e87396753007" Jul 06 16:26:31.901568 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:31.901551 2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f382f0aa8afeb3f46a0491928d22547f49ffae8e9e008acc03d8e87396753007"} err="failed to get container status \"f382f0aa8afeb3f46a0491928d22547f49ffae8e9e008acc03d8e87396753007\": rpc error: code = NotFound desc = could not find container \"f382f0aa8afeb3f46a0491928d22547f49ffae8e9e008acc03d8e87396753007\": container with ID starting with f382f0aa8afeb3f46a0491928d22547f49ffae8e9e008acc03d8e87396753007 not found: ID does not exist" Jul 06 16:26:31.901568 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:31.901566 2584 scope.go:117] "RemoveContainer" containerID="517ce8e7eb4151ac00876501d1782eefbc1ab7a3f3286461d80b3d4d9e7e0509" Jul 06 16:26:31.901773 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:31.901756 2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"517ce8e7eb4151ac00876501d1782eefbc1ab7a3f3286461d80b3d4d9e7e0509"} err="failed to get container status \"517ce8e7eb4151ac00876501d1782eefbc1ab7a3f3286461d80b3d4d9e7e0509\": rpc error: code = NotFound desc = could not find container \"517ce8e7eb4151ac00876501d1782eefbc1ab7a3f3286461d80b3d4d9e7e0509\": container with ID starting with 517ce8e7eb4151ac00876501d1782eefbc1ab7a3f3286461d80b3d4d9e7e0509 not found: ID does not exist" Jul 06 16:26:31.901820 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:31.901774 2584 scope.go:117] "RemoveContainer" containerID="5db1d4dc73ea0864daa2c9c02f7fa6f2c1d97d81a49818a4dc57ad6a62c9ee01" Jul 06 16:26:31.901951 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:31.901936 2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5db1d4dc73ea0864daa2c9c02f7fa6f2c1d97d81a49818a4dc57ad6a62c9ee01"} err="failed to get container status \"5db1d4dc73ea0864daa2c9c02f7fa6f2c1d97d81a49818a4dc57ad6a62c9ee01\": rpc error: code = NotFound desc = could not find container \"5db1d4dc73ea0864daa2c9c02f7fa6f2c1d97d81a49818a4dc57ad6a62c9ee01\": container with ID starting with 5db1d4dc73ea0864daa2c9c02f7fa6f2c1d97d81a49818a4dc57ad6a62c9ee01 not found: ID does not exist" Jul 06 16:26:31.901951 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:31.901950 2584 scope.go:117] "RemoveContainer" containerID="57c8fa983b9d033a5af1c7a0dde1a2ae9ce2ceaf5536ccc1c3f23c8b3873d529" Jul 06 16:26:31.902119 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:31.902103 2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"57c8fa983b9d033a5af1c7a0dde1a2ae9ce2ceaf5536ccc1c3f23c8b3873d529"} err="failed to get container status \"57c8fa983b9d033a5af1c7a0dde1a2ae9ce2ceaf5536ccc1c3f23c8b3873d529\": rpc error: code = NotFound desc = could not find container \"57c8fa983b9d033a5af1c7a0dde1a2ae9ce2ceaf5536ccc1c3f23c8b3873d529\": container with ID starting with 57c8fa983b9d033a5af1c7a0dde1a2ae9ce2ceaf5536ccc1c3f23c8b3873d529 not found: ID does not exist" Jul 06 16:26:31.902161 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:31.902120 2584 scope.go:117] "RemoveContainer" containerID="43baa1bcfdc1b60520a09c3140c28661ec60c28abbbe4df3fce9fd0995303c6f" Jul 06 16:26:31.902304 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:31.902281 2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"43baa1bcfdc1b60520a09c3140c28661ec60c28abbbe4df3fce9fd0995303c6f"} err="failed to get container status \"43baa1bcfdc1b60520a09c3140c28661ec60c28abbbe4df3fce9fd0995303c6f\": rpc error: code = NotFound desc = could not find container \"43baa1bcfdc1b60520a09c3140c28661ec60c28abbbe4df3fce9fd0995303c6f\": container with ID starting with 43baa1bcfdc1b60520a09c3140c28661ec60c28abbbe4df3fce9fd0995303c6f not found: ID does not exist" Jul 06 16:26:31.902343 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:31.902304 2584 scope.go:117] "RemoveContainer" containerID="a8df07321c9f43b846bcf3aa019f35d5af28fd9be2226ec9a8b50cea219f3b9c" Jul 06 16:26:31.902475 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:31.902459 2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a8df07321c9f43b846bcf3aa019f35d5af28fd9be2226ec9a8b50cea219f3b9c"} err="failed to get container status \"a8df07321c9f43b846bcf3aa019f35d5af28fd9be2226ec9a8b50cea219f3b9c\": rpc error: code = NotFound desc = could not find container \"a8df07321c9f43b846bcf3aa019f35d5af28fd9be2226ec9a8b50cea219f3b9c\": container with ID starting with a8df07321c9f43b846bcf3aa019f35d5af28fd9be2226ec9a8b50cea219f3b9c not found: ID does not exist" Jul 06 16:26:31.902516 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:31.902475 2584 scope.go:117] "RemoveContainer" containerID="4bc77931f9c26ade34e18fe206defb68e523052dbf84703999c77a2596bd8f35" Jul 06 16:26:31.902648 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:31.902635 2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4bc77931f9c26ade34e18fe206defb68e523052dbf84703999c77a2596bd8f35"} err="failed to get container status \"4bc77931f9c26ade34e18fe206defb68e523052dbf84703999c77a2596bd8f35\": rpc error: code = NotFound desc = could not find container \"4bc77931f9c26ade34e18fe206defb68e523052dbf84703999c77a2596bd8f35\": container with ID starting with 4bc77931f9c26ade34e18fe206defb68e523052dbf84703999c77a2596bd8f35 not found: ID does not exist" Jul 06 16:26:31.902686 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:31.902648 2584 scope.go:117] "RemoveContainer" containerID="f382f0aa8afeb3f46a0491928d22547f49ffae8e9e008acc03d8e87396753007" Jul 06 16:26:31.902833 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:31.902819 2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f382f0aa8afeb3f46a0491928d22547f49ffae8e9e008acc03d8e87396753007"} err="failed to get container status \"f382f0aa8afeb3f46a0491928d22547f49ffae8e9e008acc03d8e87396753007\": rpc error: code = NotFound desc = could not find container \"f382f0aa8afeb3f46a0491928d22547f49ffae8e9e008acc03d8e87396753007\": container with ID starting with f382f0aa8afeb3f46a0491928d22547f49ffae8e9e008acc03d8e87396753007 not found: ID does not exist" Jul 06 16:26:31.902833 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:31.902832 2584 scope.go:117] "RemoveContainer" containerID="517ce8e7eb4151ac00876501d1782eefbc1ab7a3f3286461d80b3d4d9e7e0509" Jul 06 16:26:31.903003 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:31.902988 2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"517ce8e7eb4151ac00876501d1782eefbc1ab7a3f3286461d80b3d4d9e7e0509"} err="failed to get container status \"517ce8e7eb4151ac00876501d1782eefbc1ab7a3f3286461d80b3d4d9e7e0509\": rpc error: code = NotFound desc = could not find container \"517ce8e7eb4151ac00876501d1782eefbc1ab7a3f3286461d80b3d4d9e7e0509\": container with ID starting with 517ce8e7eb4151ac00876501d1782eefbc1ab7a3f3286461d80b3d4d9e7e0509 not found: ID does not exist" Jul 06 16:26:31.903044 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:31.903002 2584 scope.go:117] "RemoveContainer" containerID="5db1d4dc73ea0864daa2c9c02f7fa6f2c1d97d81a49818a4dc57ad6a62c9ee01" Jul 06 16:26:31.903185 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:31.903168 2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5db1d4dc73ea0864daa2c9c02f7fa6f2c1d97d81a49818a4dc57ad6a62c9ee01"} err="failed to get container status \"5db1d4dc73ea0864daa2c9c02f7fa6f2c1d97d81a49818a4dc57ad6a62c9ee01\": rpc error: code = NotFound desc = could not find container \"5db1d4dc73ea0864daa2c9c02f7fa6f2c1d97d81a49818a4dc57ad6a62c9ee01\": container with ID starting with 5db1d4dc73ea0864daa2c9c02f7fa6f2c1d97d81a49818a4dc57ad6a62c9ee01 not found: ID does not exist" Jul 06 16:26:31.909432 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:31.909412 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 06 16:26:31.909712 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:31.909701 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="387d0b57-b4b5-4acb-822e-32b3d3d5fa63" containerName="config-reloader" Jul 06 16:26:31.909768 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:31.909713 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="387d0b57-b4b5-4acb-822e-32b3d3d5fa63" containerName="config-reloader" Jul 06 16:26:31.909768 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:31.909738 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="387d0b57-b4b5-4acb-822e-32b3d3d5fa63" containerName="kube-rbac-proxy" Jul 06 16:26:31.909768 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:31.909747 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="387d0b57-b4b5-4acb-822e-32b3d3d5fa63" containerName="kube-rbac-proxy" Jul 06 16:26:31.909863 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:31.909771 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="387d0b57-b4b5-4acb-822e-32b3d3d5fa63" containerName="thanos-sidecar" Jul 06 16:26:31.909863 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:31.909777 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="387d0b57-b4b5-4acb-822e-32b3d3d5fa63" containerName="thanos-sidecar" Jul 06 16:26:31.909863 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:31.909785 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="387d0b57-b4b5-4acb-822e-32b3d3d5fa63" containerName="kube-rbac-proxy-thanos" Jul 06 16:26:31.909863 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:31.909790 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="387d0b57-b4b5-4acb-822e-32b3d3d5fa63" containerName="kube-rbac-proxy-thanos" Jul 06 16:26:31.909863 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:31.909797 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="387d0b57-b4b5-4acb-822e-32b3d3d5fa63" containerName="prometheus" Jul 06 16:26:31.909863 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:31.909802 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="387d0b57-b4b5-4acb-822e-32b3d3d5fa63" containerName="prometheus" Jul 06 16:26:31.909863 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:31.909810 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="387d0b57-b4b5-4acb-822e-32b3d3d5fa63" containerName="kube-rbac-proxy-web" Jul 06 16:26:31.909863 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:31.909815 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="387d0b57-b4b5-4acb-822e-32b3d3d5fa63" containerName="kube-rbac-proxy-web" Jul 06 16:26:31.909863 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:31.909823 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="387d0b57-b4b5-4acb-822e-32b3d3d5fa63" containerName="init-config-reloader" Jul 06 16:26:31.909863 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:31.909828 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="387d0b57-b4b5-4acb-822e-32b3d3d5fa63" containerName="init-config-reloader" Jul 06 16:26:31.910183 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:31.909874 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="387d0b57-b4b5-4acb-822e-32b3d3d5fa63" containerName="prometheus" Jul 06 16:26:31.910183 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:31.909882 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="387d0b57-b4b5-4acb-822e-32b3d3d5fa63" containerName="config-reloader" Jul 06 16:26:31.910183 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:31.909888 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="387d0b57-b4b5-4acb-822e-32b3d3d5fa63" containerName="kube-rbac-proxy" Jul 06 16:26:31.910183 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:31.909894 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="387d0b57-b4b5-4acb-822e-32b3d3d5fa63" containerName="thanos-sidecar" Jul 06 16:26:31.910183 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:31.909899 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="387d0b57-b4b5-4acb-822e-32b3d3d5fa63" containerName="kube-rbac-proxy-web" Jul 06 16:26:31.910183 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:31.909903 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="387d0b57-b4b5-4acb-822e-32b3d3d5fa63" containerName="kube-rbac-proxy-thanos" Jul 06 16:26:31.913757 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:31.913716 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 06 16:26:31.917117 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:31.917100 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Jul 06 16:26:31.917221 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:31.917100 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Jul 06 16:26:31.917310 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:31.917237 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Jul 06 16:26:31.917310 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:31.917247 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Jul 06 16:26:31.917310 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:31.917266 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Jul 06 16:26:31.917310 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:31.917292 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Jul 06 16:26:31.917561 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:31.917266 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-6isj5j0bn659p\"" Jul 06 16:26:31.917561 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:31.917426 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Jul 06 16:26:31.917664 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:31.917559 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Jul 06 16:26:31.917664 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:31.917566 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-v4grf\"" Jul 06 16:26:31.917664 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:31.917590 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Jul 06 16:26:31.917913 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:31.917901 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Jul 06 16:26:31.918005 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:31.917943 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Jul 06 16:26:31.918608 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:31.918553 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Jul 06 16:26:31.921558 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:31.921541 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Jul 06 16:26:31.923224 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:31.923205 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Jul 06 16:26:31.931678 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:31.931648 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 06 16:26:32.090890 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:32.090779 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/05dd8b1c-17dd-49ce-a9f6-5e4fa25a8605-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"05dd8b1c-17dd-49ce-a9f6-5e4fa25a8605\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 16:26:32.090890 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:32.090827 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/05dd8b1c-17dd-49ce-a9f6-5e4fa25a8605-config-out\") pod \"prometheus-k8s-0\" (UID: \"05dd8b1c-17dd-49ce-a9f6-5e4fa25a8605\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 16:26:32.090890 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:32.090849 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/05dd8b1c-17dd-49ce-a9f6-5e4fa25a8605-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"05dd8b1c-17dd-49ce-a9f6-5e4fa25a8605\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 16:26:32.090890 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:32.090871 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/05dd8b1c-17dd-49ce-a9f6-5e4fa25a8605-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"05dd8b1c-17dd-49ce-a9f6-5e4fa25a8605\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 16:26:32.091154 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:32.090937 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/05dd8b1c-17dd-49ce-a9f6-5e4fa25a8605-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"05dd8b1c-17dd-49ce-a9f6-5e4fa25a8605\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 16:26:32.091154 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:32.090958 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/05dd8b1c-17dd-49ce-a9f6-5e4fa25a8605-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"05dd8b1c-17dd-49ce-a9f6-5e4fa25a8605\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 16:26:32.091154 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:32.090973 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/05dd8b1c-17dd-49ce-a9f6-5e4fa25a8605-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"05dd8b1c-17dd-49ce-a9f6-5e4fa25a8605\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 16:26:32.091154 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:32.090996 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/05dd8b1c-17dd-49ce-a9f6-5e4fa25a8605-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"05dd8b1c-17dd-49ce-a9f6-5e4fa25a8605\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 16:26:32.091154 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:32.091016 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/05dd8b1c-17dd-49ce-a9f6-5e4fa25a8605-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"05dd8b1c-17dd-49ce-a9f6-5e4fa25a8605\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 16:26:32.091154 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:32.091037 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/05dd8b1c-17dd-49ce-a9f6-5e4fa25a8605-web-config\") pod \"prometheus-k8s-0\" (UID: \"05dd8b1c-17dd-49ce-a9f6-5e4fa25a8605\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 16:26:32.091154 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:32.091092 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2x4gm\" (UniqueName: \"kubernetes.io/projected/05dd8b1c-17dd-49ce-a9f6-5e4fa25a8605-kube-api-access-2x4gm\") pod \"prometheus-k8s-0\" (UID: \"05dd8b1c-17dd-49ce-a9f6-5e4fa25a8605\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 16:26:32.091154 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:32.091126 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/05dd8b1c-17dd-49ce-a9f6-5e4fa25a8605-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"05dd8b1c-17dd-49ce-a9f6-5e4fa25a8605\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 16:26:32.091154 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:32.091155 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/05dd8b1c-17dd-49ce-a9f6-5e4fa25a8605-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"05dd8b1c-17dd-49ce-a9f6-5e4fa25a8605\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 16:26:32.091465 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:32.091176 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/05dd8b1c-17dd-49ce-a9f6-5e4fa25a8605-config\") pod \"prometheus-k8s-0\" (UID: \"05dd8b1c-17dd-49ce-a9f6-5e4fa25a8605\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 16:26:32.091465 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:32.091191 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/05dd8b1c-17dd-49ce-a9f6-5e4fa25a8605-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"05dd8b1c-17dd-49ce-a9f6-5e4fa25a8605\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 16:26:32.091465 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:32.091230 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/05dd8b1c-17dd-49ce-a9f6-5e4fa25a8605-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"05dd8b1c-17dd-49ce-a9f6-5e4fa25a8605\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 16:26:32.091465 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:32.091261 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/05dd8b1c-17dd-49ce-a9f6-5e4fa25a8605-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"05dd8b1c-17dd-49ce-a9f6-5e4fa25a8605\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 16:26:32.091465 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:32.091318 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/05dd8b1c-17dd-49ce-a9f6-5e4fa25a8605-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"05dd8b1c-17dd-49ce-a9f6-5e4fa25a8605\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 16:26:32.091465 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:32.091342 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/05dd8b1c-17dd-49ce-a9f6-5e4fa25a8605-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"05dd8b1c-17dd-49ce-a9f6-5e4fa25a8605\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 16:26:32.091465 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:32.091364 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/05dd8b1c-17dd-49ce-a9f6-5e4fa25a8605-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"05dd8b1c-17dd-49ce-a9f6-5e4fa25a8605\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 16:26:32.121709 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:32.121675 2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="387d0b57-b4b5-4acb-822e-32b3d3d5fa63" path="/var/lib/kubelet/pods/387d0b57-b4b5-4acb-822e-32b3d3d5fa63/volumes" Jul 06 16:26:32.192561 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:32.192526 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/05dd8b1c-17dd-49ce-a9f6-5e4fa25a8605-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"05dd8b1c-17dd-49ce-a9f6-5e4fa25a8605\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 16:26:32.192753 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:32.192569 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/05dd8b1c-17dd-49ce-a9f6-5e4fa25a8605-web-config\") pod \"prometheus-k8s-0\" (UID: \"05dd8b1c-17dd-49ce-a9f6-5e4fa25a8605\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 16:26:32.192753 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:32.192601 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2x4gm\" (UniqueName: \"kubernetes.io/projected/05dd8b1c-17dd-49ce-a9f6-5e4fa25a8605-kube-api-access-2x4gm\") pod \"prometheus-k8s-0\" (UID: \"05dd8b1c-17dd-49ce-a9f6-5e4fa25a8605\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 16:26:32.192753 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:32.192620 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/05dd8b1c-17dd-49ce-a9f6-5e4fa25a8605-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"05dd8b1c-17dd-49ce-a9f6-5e4fa25a8605\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 16:26:32.192753 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:32.192642 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/05dd8b1c-17dd-49ce-a9f6-5e4fa25a8605-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"05dd8b1c-17dd-49ce-a9f6-5e4fa25a8605\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 16:26:32.192753 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:32.192662 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/05dd8b1c-17dd-49ce-a9f6-5e4fa25a8605-config\") pod \"prometheus-k8s-0\" (UID: \"05dd8b1c-17dd-49ce-a9f6-5e4fa25a8605\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 16:26:32.192753 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:32.192682 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/05dd8b1c-17dd-49ce-a9f6-5e4fa25a8605-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"05dd8b1c-17dd-49ce-a9f6-5e4fa25a8605\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 16:26:32.192753 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:32.192706 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/05dd8b1c-17dd-49ce-a9f6-5e4fa25a8605-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"05dd8b1c-17dd-49ce-a9f6-5e4fa25a8605\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 16:26:32.192753 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:32.192743 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/05dd8b1c-17dd-49ce-a9f6-5e4fa25a8605-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"05dd8b1c-17dd-49ce-a9f6-5e4fa25a8605\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 16:26:32.193136 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:32.192772 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/05dd8b1c-17dd-49ce-a9f6-5e4fa25a8605-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"05dd8b1c-17dd-49ce-a9f6-5e4fa25a8605\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 16:26:32.193136 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:32.192796 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/05dd8b1c-17dd-49ce-a9f6-5e4fa25a8605-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"05dd8b1c-17dd-49ce-a9f6-5e4fa25a8605\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 16:26:32.193136 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:32.192829 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/05dd8b1c-17dd-49ce-a9f6-5e4fa25a8605-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"05dd8b1c-17dd-49ce-a9f6-5e4fa25a8605\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 16:26:32.193136 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:32.192862 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/05dd8b1c-17dd-49ce-a9f6-5e4fa25a8605-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"05dd8b1c-17dd-49ce-a9f6-5e4fa25a8605\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 16:26:32.193136 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:32.192886 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/05dd8b1c-17dd-49ce-a9f6-5e4fa25a8605-config-out\") pod \"prometheus-k8s-0\" (UID: \"05dd8b1c-17dd-49ce-a9f6-5e4fa25a8605\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 16:26:32.193136 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:32.192908 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/05dd8b1c-17dd-49ce-a9f6-5e4fa25a8605-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"05dd8b1c-17dd-49ce-a9f6-5e4fa25a8605\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 16:26:32.193136 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:32.192932 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/05dd8b1c-17dd-49ce-a9f6-5e4fa25a8605-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"05dd8b1c-17dd-49ce-a9f6-5e4fa25a8605\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 16:26:32.193136 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:32.192989 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/05dd8b1c-17dd-49ce-a9f6-5e4fa25a8605-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"05dd8b1c-17dd-49ce-a9f6-5e4fa25a8605\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 16:26:32.193136 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:32.193024 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/05dd8b1c-17dd-49ce-a9f6-5e4fa25a8605-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"05dd8b1c-17dd-49ce-a9f6-5e4fa25a8605\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 16:26:32.193136 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:32.193049 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/05dd8b1c-17dd-49ce-a9f6-5e4fa25a8605-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"05dd8b1c-17dd-49ce-a9f6-5e4fa25a8605\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 16:26:32.193136 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:32.193092 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/05dd8b1c-17dd-49ce-a9f6-5e4fa25a8605-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"05dd8b1c-17dd-49ce-a9f6-5e4fa25a8605\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 16:26:32.193738 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:32.193355 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/05dd8b1c-17dd-49ce-a9f6-5e4fa25a8605-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"05dd8b1c-17dd-49ce-a9f6-5e4fa25a8605\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 16:26:32.194696 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:32.194149 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/05dd8b1c-17dd-49ce-a9f6-5e4fa25a8605-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"05dd8b1c-17dd-49ce-a9f6-5e4fa25a8605\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 16:26:32.194696 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:32.194236 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/05dd8b1c-17dd-49ce-a9f6-5e4fa25a8605-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"05dd8b1c-17dd-49ce-a9f6-5e4fa25a8605\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 16:26:32.194696 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:32.194267 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/05dd8b1c-17dd-49ce-a9f6-5e4fa25a8605-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"05dd8b1c-17dd-49ce-a9f6-5e4fa25a8605\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 16:26:32.194696 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:32.194463 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/05dd8b1c-17dd-49ce-a9f6-5e4fa25a8605-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"05dd8b1c-17dd-49ce-a9f6-5e4fa25a8605\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 16:26:32.195006 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:32.194955 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/05dd8b1c-17dd-49ce-a9f6-5e4fa25a8605-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"05dd8b1c-17dd-49ce-a9f6-5e4fa25a8605\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 16:26:32.195416 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:32.195389 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/05dd8b1c-17dd-49ce-a9f6-5e4fa25a8605-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"05dd8b1c-17dd-49ce-a9f6-5e4fa25a8605\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 16:26:32.195913 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:32.195871 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/05dd8b1c-17dd-49ce-a9f6-5e4fa25a8605-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"05dd8b1c-17dd-49ce-a9f6-5e4fa25a8605\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 16:26:32.196454 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:32.196430 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/05dd8b1c-17dd-49ce-a9f6-5e4fa25a8605-config\") pod \"prometheus-k8s-0\" (UID: \"05dd8b1c-17dd-49ce-a9f6-5e4fa25a8605\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 16:26:32.196577 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:32.196555 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/05dd8b1c-17dd-49ce-a9f6-5e4fa25a8605-config-out\") pod \"prometheus-k8s-0\" (UID: \"05dd8b1c-17dd-49ce-a9f6-5e4fa25a8605\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 16:26:32.197680 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:32.197572 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/05dd8b1c-17dd-49ce-a9f6-5e4fa25a8605-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"05dd8b1c-17dd-49ce-a9f6-5e4fa25a8605\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 16:26:32.197680 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:32.197586 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/05dd8b1c-17dd-49ce-a9f6-5e4fa25a8605-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"05dd8b1c-17dd-49ce-a9f6-5e4fa25a8605\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 16:26:32.198283 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:32.197847 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/05dd8b1c-17dd-49ce-a9f6-5e4fa25a8605-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"05dd8b1c-17dd-49ce-a9f6-5e4fa25a8605\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 16:26:32.198283 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:32.198060 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/05dd8b1c-17dd-49ce-a9f6-5e4fa25a8605-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"05dd8b1c-17dd-49ce-a9f6-5e4fa25a8605\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 16:26:32.198283 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:32.198100 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/05dd8b1c-17dd-49ce-a9f6-5e4fa25a8605-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"05dd8b1c-17dd-49ce-a9f6-5e4fa25a8605\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 16:26:32.198446 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:32.198315 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/05dd8b1c-17dd-49ce-a9f6-5e4fa25a8605-web-config\") pod \"prometheus-k8s-0\" (UID: \"05dd8b1c-17dd-49ce-a9f6-5e4fa25a8605\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 16:26:32.198691 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:32.198664 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/05dd8b1c-17dd-49ce-a9f6-5e4fa25a8605-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"05dd8b1c-17dd-49ce-a9f6-5e4fa25a8605\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 16:26:32.198979 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:32.198963 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/05dd8b1c-17dd-49ce-a9f6-5e4fa25a8605-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"05dd8b1c-17dd-49ce-a9f6-5e4fa25a8605\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 16:26:32.199184 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:32.199164 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/05dd8b1c-17dd-49ce-a9f6-5e4fa25a8605-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"05dd8b1c-17dd-49ce-a9f6-5e4fa25a8605\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 16:26:32.209674 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:32.209650 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2x4gm\" (UniqueName: \"kubernetes.io/projected/05dd8b1c-17dd-49ce-a9f6-5e4fa25a8605-kube-api-access-2x4gm\") pod \"prometheus-k8s-0\" (UID: \"05dd8b1c-17dd-49ce-a9f6-5e4fa25a8605\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 16:26:32.223589 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:32.223563 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 06 16:26:32.370773 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:32.370676 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 06 16:26:32.373589 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:26:32.373567 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod05dd8b1c_17dd_49ce_a9f6_5e4fa25a8605.slice/crio-634044e63b242466830068928dcd6e661b80a3c04691ba6ee62e2cbab50129a4 WatchSource:0}: Error finding container 634044e63b242466830068928dcd6e661b80a3c04691ba6ee62e2cbab50129a4: Status 404 returned error can't find the container with id 634044e63b242466830068928dcd6e661b80a3c04691ba6ee62e2cbab50129a4 Jul 06 16:26:32.840179 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:32.840149 2584 generic.go:356] "Generic (PLEG): container finished" podID="05dd8b1c-17dd-49ce-a9f6-5e4fa25a8605" containerID="b8c6edb8e0b1ccd816d781b6726a2af6e41a8e8773508950e5175d0130f0e939" exitCode=0 Jul 06 16:26:32.840529 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:32.840232 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"05dd8b1c-17dd-49ce-a9f6-5e4fa25a8605","Type":"ContainerDied","Data":"b8c6edb8e0b1ccd816d781b6726a2af6e41a8e8773508950e5175d0130f0e939"} Jul 06 16:26:32.840529 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:32.840281 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"05dd8b1c-17dd-49ce-a9f6-5e4fa25a8605","Type":"ContainerStarted","Data":"634044e63b242466830068928dcd6e661b80a3c04691ba6ee62e2cbab50129a4"} Jul 06 16:26:33.160294 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:33.160262 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:26:33.259670 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:33.259635 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:26:33.848929 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:33.848889 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"05dd8b1c-17dd-49ce-a9f6-5e4fa25a8605","Type":"ContainerStarted","Data":"cad597f260deb36b80460ecae5f63a6e12ae98851e81121a1cea757420edfa98"} Jul 06 16:26:33.849281 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:33.848960 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"05dd8b1c-17dd-49ce-a9f6-5e4fa25a8605","Type":"ContainerStarted","Data":"7906b801d7d4ca78e1243adc990265eddb6bbfd028d5f4db9071dc4bdb7026be"} Jul 06 16:26:33.849281 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:33.848989 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"05dd8b1c-17dd-49ce-a9f6-5e4fa25a8605","Type":"ContainerStarted","Data":"ffca6b4a4219fe81220faecfb8d2889395936fd121d5b2f15cc8c05e60e8d825"} Jul 06 16:26:33.849281 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:33.849019 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"05dd8b1c-17dd-49ce-a9f6-5e4fa25a8605","Type":"ContainerStarted","Data":"1eaf102d45a4b819a77f7c652fc3717269e317f65cbcad6de12c156bc7b097e4"} Jul 06 16:26:33.849281 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:33.849042 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"05dd8b1c-17dd-49ce-a9f6-5e4fa25a8605","Type":"ContainerStarted","Data":"fafaf8290d441611a3d790bb86b1691ad399e8fb873f307dad27e69a7ff24fcd"} Jul 06 16:26:33.849281 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:33.849075 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"05dd8b1c-17dd-49ce-a9f6-5e4fa25a8605","Type":"ContainerStarted","Data":"491cdc5b9a5441a78ec8bb1345fe714bd4a61b7bca3204872e45533f29b57a4e"} Jul 06 16:26:33.886119 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:33.886065 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.86791641 podStartE2EDuration="2.886045886s" totalImagesPullingTime="0s" totalInitContainerRuntime="18.129476ms" isStatefulPod=false podCreationTimestamp="2026-07-06 16:26:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 16:26:33.883623158 +0000 UTC m=+144.497763278" watchObservedRunningTime="2026-07-06 16:26:33.886045886 +0000 UTC m=+144.500186007" Jul 06 16:26:34.387172 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:34.387139 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:26:34.459475 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:34.459445 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:26:37.224028 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:37.223982 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 06 16:26:38.084185 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:38.084144 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:26:38.443389 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:38.443353 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:26:56.213039 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:26:56.212996 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:27:03.085014 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:27:03.084935 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:27:24.360449 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:27:24.360413 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:27:24.462767 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:27:24.462710 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:27:32.224420 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:27:32.224369 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Jul 06 16:27:32.240040 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:27:32.240014 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Jul 06 16:27:33.108824 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:27:33.108789 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 06 16:27:43.262868 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:27:43.262829 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:27:48.798218 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:27:48.798177 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:27:51.188239 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:27:51.188207 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:27:52.142254 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:27:52.142218 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:27:55.217693 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:27:55.217647 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:27:55.354362 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:27:55.354326 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:27:57.306124 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:27:57.306085 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:27:57.534841 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:27:57.534806 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:27:57.773410 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:27:57.773379 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-966zn"] Jul 06 16:27:57.776900 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:27:57.776883 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-966zn" Jul 06 16:27:57.787463 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:27:57.787445 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Jul 06 16:27:57.787571 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:27:57.787444 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-9wdnb\"" Jul 06 16:27:57.787571 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:27:57.787445 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Jul 06 16:27:57.805347 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:27:57.805322 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-966zn"] Jul 06 16:27:57.924414 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:27:57.924375 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/94854e01-46d3-4194-afaa-23c2f27e6281-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-966zn\" (UID: \"94854e01-46d3-4194-afaa-23c2f27e6281\") " pod="cert-manager/cert-manager-webhook-6486ffd997-966zn" Jul 06 16:27:57.924592 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:27:57.924487 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2x2ws\" (UniqueName: \"kubernetes.io/projected/94854e01-46d3-4194-afaa-23c2f27e6281-kube-api-access-2x2ws\") pod \"cert-manager-webhook-6486ffd997-966zn\" (UID: \"94854e01-46d3-4194-afaa-23c2f27e6281\") " pod="cert-manager/cert-manager-webhook-6486ffd997-966zn" Jul 06 16:27:58.025052 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:27:58.024969 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2x2ws\" (UniqueName: \"kubernetes.io/projected/94854e01-46d3-4194-afaa-23c2f27e6281-kube-api-access-2x2ws\") pod \"cert-manager-webhook-6486ffd997-966zn\" (UID: \"94854e01-46d3-4194-afaa-23c2f27e6281\") " pod="cert-manager/cert-manager-webhook-6486ffd997-966zn" Jul 06 16:27:58.025190 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:27:58.025122 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/94854e01-46d3-4194-afaa-23c2f27e6281-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-966zn\" (UID: \"94854e01-46d3-4194-afaa-23c2f27e6281\") " pod="cert-manager/cert-manager-webhook-6486ffd997-966zn" Jul 06 16:27:58.061173 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:27:58.061146 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/94854e01-46d3-4194-afaa-23c2f27e6281-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-966zn\" (UID: \"94854e01-46d3-4194-afaa-23c2f27e6281\") " pod="cert-manager/cert-manager-webhook-6486ffd997-966zn" Jul 06 16:27:58.061315 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:27:58.061182 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2x2ws\" (UniqueName: \"kubernetes.io/projected/94854e01-46d3-4194-afaa-23c2f27e6281-kube-api-access-2x2ws\") pod \"cert-manager-webhook-6486ffd997-966zn\" (UID: \"94854e01-46d3-4194-afaa-23c2f27e6281\") " pod="cert-manager/cert-manager-webhook-6486ffd997-966zn" Jul 06 16:27:58.097833 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:27:58.097807 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-966zn" Jul 06 16:27:58.264811 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:27:58.264770 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-966zn"] Jul 06 16:27:58.271496 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:27:58.271467 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod94854e01_46d3_4194_afaa_23c2f27e6281.slice/crio-ff360731640866c781b98074bf0b442cd2d2bfbc4536a1a37c6f3250093d6425 WatchSource:0}: Error finding container ff360731640866c781b98074bf0b442cd2d2bfbc4536a1a37c6f3250093d6425: Status 404 returned error can't find the container with id ff360731640866c781b98074bf0b442cd2d2bfbc4536a1a37c6f3250093d6425 Jul 06 16:27:59.198023 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:27:59.197985 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-966zn" event={"ID":"94854e01-46d3-4194-afaa-23c2f27e6281","Type":"ContainerStarted","Data":"ff360731640866c781b98074bf0b442cd2d2bfbc4536a1a37c6f3250093d6425"} Jul 06 16:27:59.555913 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:27:59.555815 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:27:59.956929 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:27:59.956889 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:28:00.478511 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:28:00.478481 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-hcjgg"] Jul 06 16:28:00.490792 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:28:00.490748 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-hcjgg" Jul 06 16:28:00.492117 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:28:00.492090 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-hcjgg"] Jul 06 16:28:00.493694 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:28:00.493487 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-szhgp\"" Jul 06 16:28:00.651564 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:28:00.651525 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9x88x\" (UniqueName: \"kubernetes.io/projected/9cec14a0-c698-4bce-814e-540a072d65cb-kube-api-access-9x88x\") pod \"cert-manager-cainjector-559cdd8c87-hcjgg\" (UID: \"9cec14a0-c698-4bce-814e-540a072d65cb\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-hcjgg" Jul 06 16:28:00.651774 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:28:00.651575 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/9cec14a0-c698-4bce-814e-540a072d65cb-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-hcjgg\" (UID: \"9cec14a0-c698-4bce-814e-540a072d65cb\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-hcjgg" Jul 06 16:28:00.753130 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:28:00.753035 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9x88x\" (UniqueName: \"kubernetes.io/projected/9cec14a0-c698-4bce-814e-540a072d65cb-kube-api-access-9x88x\") pod \"cert-manager-cainjector-559cdd8c87-hcjgg\" (UID: \"9cec14a0-c698-4bce-814e-540a072d65cb\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-hcjgg" Jul 06 16:28:00.753130 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:28:00.753079 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/9cec14a0-c698-4bce-814e-540a072d65cb-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-hcjgg\" (UID: \"9cec14a0-c698-4bce-814e-540a072d65cb\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-hcjgg" Jul 06 16:28:00.764277 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:28:00.764250 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/9cec14a0-c698-4bce-814e-540a072d65cb-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-hcjgg\" (UID: \"9cec14a0-c698-4bce-814e-540a072d65cb\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-hcjgg" Jul 06 16:28:00.764439 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:28:00.764392 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9x88x\" (UniqueName: \"kubernetes.io/projected/9cec14a0-c698-4bce-814e-540a072d65cb-kube-api-access-9x88x\") pod \"cert-manager-cainjector-559cdd8c87-hcjgg\" (UID: \"9cec14a0-c698-4bce-814e-540a072d65cb\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-hcjgg" Jul 06 16:28:00.802880 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:28:00.802829 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-hcjgg" Jul 06 16:28:00.947780 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:28:00.947744 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-hcjgg"] Jul 06 16:28:00.950977 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:28:00.950948 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9cec14a0_c698_4bce_814e_540a072d65cb.slice/crio-dde1c1448e9fe6f460dfe2a495291cf23cae7adc0f419ca3e0cdf337cdbe878b WatchSource:0}: Error finding container dde1c1448e9fe6f460dfe2a495291cf23cae7adc0f419ca3e0cdf337cdbe878b: Status 404 returned error can't find the container with id dde1c1448e9fe6f460dfe2a495291cf23cae7adc0f419ca3e0cdf337cdbe878b Jul 06 16:28:01.207757 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:28:01.207706 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-hcjgg" event={"ID":"9cec14a0-c698-4bce-814e-540a072d65cb","Type":"ContainerStarted","Data":"dde1c1448e9fe6f460dfe2a495291cf23cae7adc0f419ca3e0cdf337cdbe878b"} Jul 06 16:28:01.968893 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:28:01.968857 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:28:02.182184 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:28:02.182142 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:28:05.788003 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:28:05.787939 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:28:05.887331 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:28:05.887294 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:28:07.858401 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:28:07.858365 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:28:12.065717 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:28:12.065673 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:28:12.178202 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:28:12.178137 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:28:14.130055 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:28:14.130025 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:28:20.283444 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:28:20.283410 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-hcjgg" event={"ID":"9cec14a0-c698-4bce-814e-540a072d65cb","Type":"ContainerStarted","Data":"8483451934a606a5dba63ed00b4dcc3238d5ec2c492e7e5e8b8b3af8f80db038"} Jul 06 16:28:20.302442 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:28:20.302391 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-hcjgg" podStartSLOduration=1.517340686 podStartE2EDuration="20.302375827s" totalImagesPullingTime="18.785035141s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 16:28:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 16:28:20.302187086 +0000 UTC m=+250.916327208" watchObservedRunningTime="2026-07-06 16:28:20.302375827 +0000 UTC m=+250.916515947" Jul 06 16:28:26.025743 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:28:26.025589 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:28:26.093764 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:28:26.093691 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:28:26.308935 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:28:26.308830 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-966zn" event={"ID":"94854e01-46d3-4194-afaa-23c2f27e6281","Type":"ContainerStarted","Data":"197da781bae5d38eede99536a656ae9c6348e3a992fcfacfb10316c2b72fd3aa"} Jul 06 16:28:26.308935 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:28:26.308879 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-966zn" Jul 06 16:28:26.343708 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:28:26.343654 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-966zn" podStartSLOduration=1.866071631 podStartE2EDuration="29.343637347s" totalImagesPullingTime="27.477565716s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 16:27:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 16:28:26.341757274 +0000 UTC m=+256.955897392" watchObservedRunningTime="2026-07-06 16:28:26.343637347 +0000 UTC m=+256.957777467" Jul 06 16:28:28.818781 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:28:28.818746 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:28:28.977496 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:28:28.977461 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:28:29.584215 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:28:29.584182 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:28:30.002902 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:28:30.002832 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:28:30.139054 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:28:30.139013 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:28:30.243441 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:28:30.243405 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7896b945bc-77h26"] Jul 06 16:28:30.246986 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:28:30.246963 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-7896b945bc-77h26" Jul 06 16:28:30.252570 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:28:30.252546 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 06 16:28:30.255138 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:28:30.255120 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 06 16:28:30.256118 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:28:30.256093 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-lbdzb\"" Jul 06 16:28:30.258708 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:28:30.258682 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 06 16:28:30.265137 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:28:30.265116 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 06 16:28:30.316785 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:28:30.316750 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/639cf8f4-3234-4042-a9f1-2daa5378ba85-webhook-cert\") pod \"opendatahub-operator-controller-manager-7896b945bc-77h26\" (UID: \"639cf8f4-3234-4042-a9f1-2daa5378ba85\") " pod="opendatahub/opendatahub-operator-controller-manager-7896b945bc-77h26" Jul 06 16:28:30.316964 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:28:30.316817 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cnhnm\" (UniqueName: \"kubernetes.io/projected/639cf8f4-3234-4042-a9f1-2daa5378ba85-kube-api-access-cnhnm\") pod \"opendatahub-operator-controller-manager-7896b945bc-77h26\" (UID: \"639cf8f4-3234-4042-a9f1-2daa5378ba85\") " pod="opendatahub/opendatahub-operator-controller-manager-7896b945bc-77h26" Jul 06 16:28:30.316964 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:28:30.316869 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/639cf8f4-3234-4042-a9f1-2daa5378ba85-manifests\") pod \"opendatahub-operator-controller-manager-7896b945bc-77h26\" (UID: \"639cf8f4-3234-4042-a9f1-2daa5378ba85\") " pod="opendatahub/opendatahub-operator-controller-manager-7896b945bc-77h26" Jul 06 16:28:30.316964 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:28:30.316910 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/639cf8f4-3234-4042-a9f1-2daa5378ba85-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7896b945bc-77h26\" (UID: \"639cf8f4-3234-4042-a9f1-2daa5378ba85\") " pod="opendatahub/opendatahub-operator-controller-manager-7896b945bc-77h26" Jul 06 16:28:30.316964 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:28:30.316941 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/639cf8f4-3234-4042-a9f1-2daa5378ba85-tmp\") pod \"opendatahub-operator-controller-manager-7896b945bc-77h26\" (UID: \"639cf8f4-3234-4042-a9f1-2daa5378ba85\") " pod="opendatahub/opendatahub-operator-controller-manager-7896b945bc-77h26" Jul 06 16:28:30.339964 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:28:30.339933 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7896b945bc-77h26"] Jul 06 16:28:30.417943 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:28:30.417905 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/639cf8f4-3234-4042-a9f1-2daa5378ba85-manifests\") pod \"opendatahub-operator-controller-manager-7896b945bc-77h26\" (UID: \"639cf8f4-3234-4042-a9f1-2daa5378ba85\") " pod="opendatahub/opendatahub-operator-controller-manager-7896b945bc-77h26" Jul 06 16:28:30.418126 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:28:30.417956 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/639cf8f4-3234-4042-a9f1-2daa5378ba85-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7896b945bc-77h26\" (UID: \"639cf8f4-3234-4042-a9f1-2daa5378ba85\") " pod="opendatahub/opendatahub-operator-controller-manager-7896b945bc-77h26" Jul 06 16:28:30.418126 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:28:30.417989 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/639cf8f4-3234-4042-a9f1-2daa5378ba85-tmp\") pod \"opendatahub-operator-controller-manager-7896b945bc-77h26\" (UID: \"639cf8f4-3234-4042-a9f1-2daa5378ba85\") " pod="opendatahub/opendatahub-operator-controller-manager-7896b945bc-77h26" Jul 06 16:28:30.418126 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:28:30.418022 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/639cf8f4-3234-4042-a9f1-2daa5378ba85-webhook-cert\") pod \"opendatahub-operator-controller-manager-7896b945bc-77h26\" (UID: \"639cf8f4-3234-4042-a9f1-2daa5378ba85\") " pod="opendatahub/opendatahub-operator-controller-manager-7896b945bc-77h26" Jul 06 16:28:30.418126 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:28:30.418068 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cnhnm\" (UniqueName: \"kubernetes.io/projected/639cf8f4-3234-4042-a9f1-2daa5378ba85-kube-api-access-cnhnm\") pod \"opendatahub-operator-controller-manager-7896b945bc-77h26\" (UID: \"639cf8f4-3234-4042-a9f1-2daa5378ba85\") " pod="opendatahub/opendatahub-operator-controller-manager-7896b945bc-77h26" Jul 06 16:28:30.418354 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:28:30.418317 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/639cf8f4-3234-4042-a9f1-2daa5378ba85-manifests\") pod \"opendatahub-operator-controller-manager-7896b945bc-77h26\" (UID: \"639cf8f4-3234-4042-a9f1-2daa5378ba85\") " pod="opendatahub/opendatahub-operator-controller-manager-7896b945bc-77h26" Jul 06 16:28:30.420243 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:28:30.420217 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/639cf8f4-3234-4042-a9f1-2daa5378ba85-tmp\") pod \"opendatahub-operator-controller-manager-7896b945bc-77h26\" (UID: \"639cf8f4-3234-4042-a9f1-2daa5378ba85\") " pod="opendatahub/opendatahub-operator-controller-manager-7896b945bc-77h26" Jul 06 16:28:30.420553 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:28:30.420534 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/639cf8f4-3234-4042-a9f1-2daa5378ba85-webhook-cert\") pod \"opendatahub-operator-controller-manager-7896b945bc-77h26\" (UID: \"639cf8f4-3234-4042-a9f1-2daa5378ba85\") " pod="opendatahub/opendatahub-operator-controller-manager-7896b945bc-77h26" Jul 06 16:28:30.420553 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:28:30.420544 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/639cf8f4-3234-4042-a9f1-2daa5378ba85-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7896b945bc-77h26\" (UID: \"639cf8f4-3234-4042-a9f1-2daa5378ba85\") " pod="opendatahub/opendatahub-operator-controller-manager-7896b945bc-77h26" Jul 06 16:28:30.462251 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:28:30.462213 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cnhnm\" (UniqueName: \"kubernetes.io/projected/639cf8f4-3234-4042-a9f1-2daa5378ba85-kube-api-access-cnhnm\") pod \"opendatahub-operator-controller-manager-7896b945bc-77h26\" (UID: \"639cf8f4-3234-4042-a9f1-2daa5378ba85\") " pod="opendatahub/opendatahub-operator-controller-manager-7896b945bc-77h26" Jul 06 16:28:30.558956 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:28:30.558856 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-7896b945bc-77h26" Jul 06 16:28:30.607416 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:28:30.606996 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:28:30.773794 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:28:30.773758 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7896b945bc-77h26"] Jul 06 16:28:30.778083 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:28:30.778053 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod639cf8f4_3234_4042_a9f1_2daa5378ba85.slice/crio-97c577e2e329221d688b1225037fcd7d35dc86cbb97c698f90fc57c4b2643c09 WatchSource:0}: Error finding container 97c577e2e329221d688b1225037fcd7d35dc86cbb97c698f90fc57c4b2643c09: Status 404 returned error can't find the container with id 97c577e2e329221d688b1225037fcd7d35dc86cbb97c698f90fc57c4b2643c09 Jul 06 16:28:30.822315 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:28:30.822226 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:28:30.932992 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:28:30.932961 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:28:31.065138 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:28:31.065108 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:28:31.188793 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:28:31.188707 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:28:31.330931 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:28:31.330895 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7896b945bc-77h26" event={"ID":"639cf8f4-3234-4042-a9f1-2daa5378ba85","Type":"ContainerStarted","Data":"97c577e2e329221d688b1225037fcd7d35dc86cbb97c698f90fc57c4b2643c09"} Jul 06 16:28:32.317539 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:28:32.317506 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-966zn" Jul 06 16:28:34.347249 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:28:34.347157 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7896b945bc-77h26" event={"ID":"639cf8f4-3234-4042-a9f1-2daa5378ba85","Type":"ContainerStarted","Data":"923d20598ce633444f11a1f6110fe597d6dfbc64844def8b7115c1eec751cac9"} Jul 06 16:28:35.353410 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:28:35.353324 2584 generic.go:356] "Generic (PLEG): container finished" podID="639cf8f4-3234-4042-a9f1-2daa5378ba85" containerID="923d20598ce633444f11a1f6110fe597d6dfbc64844def8b7115c1eec751cac9" exitCode=0 Jul 06 16:28:35.353410 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:28:35.353390 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7896b945bc-77h26" event={"ID":"639cf8f4-3234-4042-a9f1-2daa5378ba85","Type":"ContainerDied","Data":"923d20598ce633444f11a1f6110fe597d6dfbc64844def8b7115c1eec751cac9"} Jul 06 16:28:36.360875 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:28:36.360832 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7896b945bc-77h26" event={"ID":"639cf8f4-3234-4042-a9f1-2daa5378ba85","Type":"ContainerStarted","Data":"5bfe4946351351a247d8a4fabb43f23b9862cc7dc7edb05c376029d3ddaec80e"} Jul 06 16:28:36.361395 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:28:36.360984 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-7896b945bc-77h26" Jul 06 16:28:36.395871 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:28:36.395811 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-7896b945bc-77h26" podStartSLOduration=1.744652072 podStartE2EDuration="6.39578803s" totalImagesPullingTime="3.644590714s" totalInitContainerRuntime="1.006545244s" isStatefulPod=false podCreationTimestamp="2026-07-06 16:28:30 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 16:28:36.394221192 +0000 UTC m=+267.008361314" watchObservedRunningTime="2026-07-06 16:28:36.39578803 +0000 UTC m=+267.009928152" Jul 06 16:28:42.398185 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:28:42.398145 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:28:43.978953 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:28:43.978918 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:28:47.367716 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:28:47.367678 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-7896b945bc-77h26" Jul 06 16:28:49.726378 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:28:49.726337 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:28:49.844630 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:28:49.844590 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:28:50.733142 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:28:50.733076 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:28:50.817963 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:28:50.817930 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:28:50.910556 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:28:50.910518 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:28:51.015429 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:28:51.015354 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:28:51.122807 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:28:51.122776 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:28:51.390301 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:28:51.390223 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:28:51.518068 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:28:51.518035 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:28:51.932274 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:28:51.932241 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:28:52.033516 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:28:52.033480 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:28:53.099011 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:28:53.098980 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:28:53.167097 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:28:53.167052 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:28:54.255016 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:28:54.254985 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:28:56.662568 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:28:56.662519 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-d4b66856f-hkvth"] Jul 06 16:28:56.672249 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:28:56.672221 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-d4b66856f-hkvth" Jul 06 16:28:56.677005 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:28:56.676975 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Jul 06 16:28:56.678044 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:28:56.677999 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Jul 06 16:28:56.678044 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:28:56.678029 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 06 16:28:56.678332 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:28:56.678071 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-thvxc\"" Jul 06 16:28:56.678448 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:28:56.678333 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 06 16:28:56.680117 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:28:56.680092 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Jul 06 16:28:56.683172 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:28:56.683151 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-d4b66856f-hkvth"] Jul 06 16:28:56.767807 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:28:56.767765 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6t6w5\" (UniqueName: \"kubernetes.io/projected/0a38af7a-6fe5-4c26-8dc2-6f1be1104699-kube-api-access-6t6w5\") pod \"lws-controller-manager-d4b66856f-hkvth\" (UID: \"0a38af7a-6fe5-4c26-8dc2-6f1be1104699\") " pod="openshift-lws-operator/lws-controller-manager-d4b66856f-hkvth" Jul 06 16:28:56.767989 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:28:56.767816 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/0a38af7a-6fe5-4c26-8dc2-6f1be1104699-metrics-cert\") pod \"lws-controller-manager-d4b66856f-hkvth\" (UID: \"0a38af7a-6fe5-4c26-8dc2-6f1be1104699\") " pod="openshift-lws-operator/lws-controller-manager-d4b66856f-hkvth" Jul 06 16:28:56.767989 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:28:56.767907 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/0a38af7a-6fe5-4c26-8dc2-6f1be1104699-manager-config\") pod \"lws-controller-manager-d4b66856f-hkvth\" (UID: \"0a38af7a-6fe5-4c26-8dc2-6f1be1104699\") " pod="openshift-lws-operator/lws-controller-manager-d4b66856f-hkvth" Jul 06 16:28:56.767989 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:28:56.767928 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0a38af7a-6fe5-4c26-8dc2-6f1be1104699-cert\") pod \"lws-controller-manager-d4b66856f-hkvth\" (UID: \"0a38af7a-6fe5-4c26-8dc2-6f1be1104699\") " pod="openshift-lws-operator/lws-controller-manager-d4b66856f-hkvth" Jul 06 16:28:56.869234 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:28:56.869179 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6t6w5\" (UniqueName: \"kubernetes.io/projected/0a38af7a-6fe5-4c26-8dc2-6f1be1104699-kube-api-access-6t6w5\") pod \"lws-controller-manager-d4b66856f-hkvth\" (UID: \"0a38af7a-6fe5-4c26-8dc2-6f1be1104699\") " pod="openshift-lws-operator/lws-controller-manager-d4b66856f-hkvth" Jul 06 16:28:56.869423 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:28:56.869250 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/0a38af7a-6fe5-4c26-8dc2-6f1be1104699-metrics-cert\") pod \"lws-controller-manager-d4b66856f-hkvth\" (UID: \"0a38af7a-6fe5-4c26-8dc2-6f1be1104699\") " pod="openshift-lws-operator/lws-controller-manager-d4b66856f-hkvth" Jul 06 16:28:56.869423 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:28:56.869324 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/0a38af7a-6fe5-4c26-8dc2-6f1be1104699-manager-config\") pod \"lws-controller-manager-d4b66856f-hkvth\" (UID: \"0a38af7a-6fe5-4c26-8dc2-6f1be1104699\") " pod="openshift-lws-operator/lws-controller-manager-d4b66856f-hkvth" Jul 06 16:28:56.869423 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:28:56.869347 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0a38af7a-6fe5-4c26-8dc2-6f1be1104699-cert\") pod \"lws-controller-manager-d4b66856f-hkvth\" (UID: \"0a38af7a-6fe5-4c26-8dc2-6f1be1104699\") " pod="openshift-lws-operator/lws-controller-manager-d4b66856f-hkvth" Jul 06 16:28:56.870061 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:28:56.870037 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/0a38af7a-6fe5-4c26-8dc2-6f1be1104699-manager-config\") pod \"lws-controller-manager-d4b66856f-hkvth\" (UID: \"0a38af7a-6fe5-4c26-8dc2-6f1be1104699\") " pod="openshift-lws-operator/lws-controller-manager-d4b66856f-hkvth" Jul 06 16:28:56.871685 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:28:56.871659 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0a38af7a-6fe5-4c26-8dc2-6f1be1104699-cert\") pod \"lws-controller-manager-d4b66856f-hkvth\" (UID: \"0a38af7a-6fe5-4c26-8dc2-6f1be1104699\") " pod="openshift-lws-operator/lws-controller-manager-d4b66856f-hkvth" Jul 06 16:28:56.871810 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:28:56.871699 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/0a38af7a-6fe5-4c26-8dc2-6f1be1104699-metrics-cert\") pod \"lws-controller-manager-d4b66856f-hkvth\" (UID: \"0a38af7a-6fe5-4c26-8dc2-6f1be1104699\") " pod="openshift-lws-operator/lws-controller-manager-d4b66856f-hkvth" Jul 06 16:28:56.881131 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:28:56.881109 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6t6w5\" (UniqueName: \"kubernetes.io/projected/0a38af7a-6fe5-4c26-8dc2-6f1be1104699-kube-api-access-6t6w5\") pod \"lws-controller-manager-d4b66856f-hkvth\" (UID: \"0a38af7a-6fe5-4c26-8dc2-6f1be1104699\") " pod="openshift-lws-operator/lws-controller-manager-d4b66856f-hkvth" Jul 06 16:28:56.984416 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:28:56.984327 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-d4b66856f-hkvth" Jul 06 16:28:57.135369 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:28:57.135318 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-d4b66856f-hkvth"] Jul 06 16:28:57.141508 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:28:57.141479 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0a38af7a_6fe5_4c26_8dc2_6f1be1104699.slice/crio-a0f82cdf592ee1ce202626ee8e950b0540c11977c860076e5a49705b628569ae WatchSource:0}: Error finding container a0f82cdf592ee1ce202626ee8e950b0540c11977c860076e5a49705b628569ae: Status 404 returned error can't find the container with id a0f82cdf592ee1ce202626ee8e950b0540c11977c860076e5a49705b628569ae Jul 06 16:28:57.459154 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:28:57.459116 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-d4b66856f-hkvth" event={"ID":"0a38af7a-6fe5-4c26-8dc2-6f1be1104699","Type":"ContainerStarted","Data":"a0f82cdf592ee1ce202626ee8e950b0540c11977c860076e5a49705b628569ae"} Jul 06 16:28:58.099788 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:28:58.099751 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-6b4fs"] Jul 06 16:28:58.103246 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:28:58.103227 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-6b4fs" Jul 06 16:28:58.106588 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:28:58.106402 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 06 16:28:58.106588 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:28:58.106438 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 06 16:28:58.106588 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:28:58.106449 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Jul 06 16:28:58.106588 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:28:58.106486 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Jul 06 16:28:58.106588 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:28:58.106493 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Jul 06 16:28:58.106588 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:28:58.106444 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 06 16:28:58.106588 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:28:58.106501 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-cnvcg\"" Jul 06 16:28:58.120573 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:28:58.120540 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-6b4fs"] Jul 06 16:28:58.284783 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:28:58.284739 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/fd2a40b6-f0b5-4bb2-80d1-d1806c7ade9d-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-6b4fs\" (UID: \"fd2a40b6-f0b5-4bb2-80d1-d1806c7ade9d\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-6b4fs" Jul 06 16:28:58.284955 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:28:58.284839 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fd2a40b6-f0b5-4bb2-80d1-d1806c7ade9d-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-6b4fs\" (UID: \"fd2a40b6-f0b5-4bb2-80d1-d1806c7ade9d\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-6b4fs" Jul 06 16:28:58.284955 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:28:58.284876 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/fd2a40b6-f0b5-4bb2-80d1-d1806c7ade9d-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-6b4fs\" (UID: \"fd2a40b6-f0b5-4bb2-80d1-d1806c7ade9d\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-6b4fs" Jul 06 16:28:58.284955 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:28:58.284910 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/fd2a40b6-f0b5-4bb2-80d1-d1806c7ade9d-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-6b4fs\" (UID: \"fd2a40b6-f0b5-4bb2-80d1-d1806c7ade9d\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-6b4fs" Jul 06 16:28:58.285060 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:28:58.284988 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/fd2a40b6-f0b5-4bb2-80d1-d1806c7ade9d-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-6b4fs\" (UID: \"fd2a40b6-f0b5-4bb2-80d1-d1806c7ade9d\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-6b4fs" Jul 06 16:28:58.285060 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:28:58.285017 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9tpwb\" (UniqueName: \"kubernetes.io/projected/fd2a40b6-f0b5-4bb2-80d1-d1806c7ade9d-kube-api-access-9tpwb\") pod \"istiod-openshift-gateway-94bb8fbfd-6b4fs\" (UID: \"fd2a40b6-f0b5-4bb2-80d1-d1806c7ade9d\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-6b4fs" Jul 06 16:28:58.285121 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:28:58.285063 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/fd2a40b6-f0b5-4bb2-80d1-d1806c7ade9d-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-6b4fs\" (UID: \"fd2a40b6-f0b5-4bb2-80d1-d1806c7ade9d\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-6b4fs" Jul 06 16:28:58.386164 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:28:58.385994 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/fd2a40b6-f0b5-4bb2-80d1-d1806c7ade9d-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-6b4fs\" (UID: \"fd2a40b6-f0b5-4bb2-80d1-d1806c7ade9d\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-6b4fs" Jul 06 16:28:58.386164 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:28:58.386056 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fd2a40b6-f0b5-4bb2-80d1-d1806c7ade9d-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-6b4fs\" (UID: \"fd2a40b6-f0b5-4bb2-80d1-d1806c7ade9d\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-6b4fs" Jul 06 16:28:58.386164 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:28:58.386091 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/fd2a40b6-f0b5-4bb2-80d1-d1806c7ade9d-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-6b4fs\" (UID: \"fd2a40b6-f0b5-4bb2-80d1-d1806c7ade9d\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-6b4fs" Jul 06 16:28:58.386164 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:28:58.386121 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/fd2a40b6-f0b5-4bb2-80d1-d1806c7ade9d-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-6b4fs\" (UID: \"fd2a40b6-f0b5-4bb2-80d1-d1806c7ade9d\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-6b4fs" Jul 06 16:28:58.386507 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:28:58.386191 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/fd2a40b6-f0b5-4bb2-80d1-d1806c7ade9d-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-6b4fs\" (UID: \"fd2a40b6-f0b5-4bb2-80d1-d1806c7ade9d\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-6b4fs" Jul 06 16:28:58.386507 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:28:58.386216 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9tpwb\" (UniqueName: \"kubernetes.io/projected/fd2a40b6-f0b5-4bb2-80d1-d1806c7ade9d-kube-api-access-9tpwb\") pod \"istiod-openshift-gateway-94bb8fbfd-6b4fs\" (UID: \"fd2a40b6-f0b5-4bb2-80d1-d1806c7ade9d\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-6b4fs" Jul 06 16:28:58.386507 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:28:58.386263 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/fd2a40b6-f0b5-4bb2-80d1-d1806c7ade9d-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-6b4fs\" (UID: \"fd2a40b6-f0b5-4bb2-80d1-d1806c7ade9d\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-6b4fs" Jul 06 16:28:58.387039 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:28:58.386690 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/fd2a40b6-f0b5-4bb2-80d1-d1806c7ade9d-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-6b4fs\" (UID: \"fd2a40b6-f0b5-4bb2-80d1-d1806c7ade9d\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-6b4fs" Jul 06 16:28:58.392830 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:28:58.389346 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/fd2a40b6-f0b5-4bb2-80d1-d1806c7ade9d-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-6b4fs\" (UID: \"fd2a40b6-f0b5-4bb2-80d1-d1806c7ade9d\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-6b4fs" Jul 06 16:28:58.392830 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:28:58.390230 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/fd2a40b6-f0b5-4bb2-80d1-d1806c7ade9d-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-6b4fs\" (UID: \"fd2a40b6-f0b5-4bb2-80d1-d1806c7ade9d\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-6b4fs" Jul 06 16:28:58.392830 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:28:58.390540 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/fd2a40b6-f0b5-4bb2-80d1-d1806c7ade9d-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-6b4fs\" (UID: \"fd2a40b6-f0b5-4bb2-80d1-d1806c7ade9d\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-6b4fs" Jul 06 16:28:58.392830 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:28:58.390681 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/fd2a40b6-f0b5-4bb2-80d1-d1806c7ade9d-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-6b4fs\" (UID: \"fd2a40b6-f0b5-4bb2-80d1-d1806c7ade9d\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-6b4fs" Jul 06 16:28:58.406645 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:28:58.406613 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9tpwb\" (UniqueName: \"kubernetes.io/projected/fd2a40b6-f0b5-4bb2-80d1-d1806c7ade9d-kube-api-access-9tpwb\") pod \"istiod-openshift-gateway-94bb8fbfd-6b4fs\" (UID: \"fd2a40b6-f0b5-4bb2-80d1-d1806c7ade9d\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-6b4fs" Jul 06 16:28:58.409965 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:28:58.409910 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fd2a40b6-f0b5-4bb2-80d1-d1806c7ade9d-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-6b4fs\" (UID: \"fd2a40b6-f0b5-4bb2-80d1-d1806c7ade9d\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-6b4fs" Jul 06 16:28:58.414781 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:28:58.414752 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-6b4fs" Jul 06 16:28:58.586209 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:28:58.586173 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-6b4fs"] Jul 06 16:28:58.587582 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:28:58.587546 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfd2a40b6_f0b5_4bb2_80d1_d1806c7ade9d.slice/crio-a6489bdd2a350c0df5e601eb8be1962a4718b6be6b920ea6a16dbd8eb44c02f1 WatchSource:0}: Error finding container a6489bdd2a350c0df5e601eb8be1962a4718b6be6b920ea6a16dbd8eb44c02f1: Status 404 returned error can't find the container with id a6489bdd2a350c0df5e601eb8be1962a4718b6be6b920ea6a16dbd8eb44c02f1 Jul 06 16:28:59.472425 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:28:59.472380 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-6b4fs" event={"ID":"fd2a40b6-f0b5-4bb2-80d1-d1806c7ade9d","Type":"ContainerStarted","Data":"a6489bdd2a350c0df5e601eb8be1962a4718b6be6b920ea6a16dbd8eb44c02f1"} Jul 06 16:29:00.479507 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:29:00.479469 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-d4b66856f-hkvth" event={"ID":"0a38af7a-6fe5-4c26-8dc2-6f1be1104699","Type":"ContainerStarted","Data":"8c29c3c9814867a5a6ecae2dbd0efd1682c388d8d6371c08d13be05ed6f98caa"} Jul 06 16:29:00.480012 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:29:00.479542 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-d4b66856f-hkvth" Jul 06 16:29:01.502806 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:29:01.502770 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:29:03.886329 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:29:03.886285 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 06 16:29:03.886638 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:29:03.886367 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 06 16:29:04.501915 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:29:04.501879 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-6b4fs" event={"ID":"fd2a40b6-f0b5-4bb2-80d1-d1806c7ade9d","Type":"ContainerStarted","Data":"0366bb1c9f50c974f8a59024a71e7e7808eecd35d76346aaa4a747e9ec8812e2"} Jul 06 16:29:04.502094 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:29:04.502047 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-6b4fs" Jul 06 16:29:04.503778 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:29:04.503748 2584 patch_prober.go:28] interesting pod/istiod-openshift-gateway-94bb8fbfd-6b4fs container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 06 16:29:04.503913 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:29:04.503814 2584 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-6b4fs" podUID="fd2a40b6-f0b5-4bb2-80d1-d1806c7ade9d" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 06 16:29:05.507424 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:29:05.507393 2584 patch_prober.go:28] interesting pod/istiod-openshift-gateway-94bb8fbfd-6b4fs container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 06 16:29:05.507820 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:29:05.507449 2584 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-6b4fs" podUID="fd2a40b6-f0b5-4bb2-80d1-d1806c7ade9d" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 06 16:29:05.769198 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:29:05.769120 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:29:06.424076 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:29:06.423988 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-6b4fs" podStartSLOduration=3.127480325 podStartE2EDuration="8.423964474s" totalImagesPullingTime="5.296484149s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 16:28:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 16:29:06.420392158 +0000 UTC m=+297.034532283" watchObservedRunningTime="2026-07-06 16:29:06.423964474 +0000 UTC m=+297.038104597" Jul 06 16:29:06.424657 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:29:06.424610 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-d4b66856f-hkvth" podStartSLOduration=7.256809583 podStartE2EDuration="10.424577684s" totalImagesPullingTime="3.167768101s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 16:28:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 16:29:00.541423507 +0000 UTC m=+291.155563629" watchObservedRunningTime="2026-07-06 16:29:06.424577684 +0000 UTC m=+297.038717807" Jul 06 16:29:06.596128 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:29:06.596097 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:29:06.707819 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:29:06.707706 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:29:06.822650 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:29:06.822615 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:29:06.927766 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:29:06.927706 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:29:07.022356 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:29:07.022280 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:29:08.508193 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:29:08.508163 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-6b4fs" Jul 06 16:29:08.953889 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:29:08.953852 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:29:09.053631 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:29:09.053580 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:29:09.162606 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:29:09.162575 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:29:09.287208 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:29:09.287112 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:29:09.855679 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:29:09.855650 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-294bx_6a8a4796-dbbc-43e2-a459-7acf11f9468d/ovn-acl-logging/0.log" Jul 06 16:29:09.856143 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:29:09.855735 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-294bx_6a8a4796-dbbc-43e2-a459-7acf11f9468d/ovn-acl-logging/0.log" Jul 06 16:29:09.859471 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:29:09.859440 2584 kubelet.go:1648] "Image garbage collection succeeded" Jul 06 16:29:11.486787 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:29:11.486756 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-d4b66856f-hkvth" Jul 06 16:29:15.306362 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:29:15.306324 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:29:15.386119 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:29:15.386087 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:30:11.369925 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:30:11.369844 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:30:11.460566 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:30:11.460529 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:30:11.563052 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:30:11.563025 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:30:26.000635 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:30:26.000588 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:30:26.734059 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:30:26.734022 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:30:37.356413 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:30:37.356369 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:30:37.426296 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:30:37.426263 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:30:41.095275 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:30:41.095241 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:30:44.553893 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:30:44.553849 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:30:46.471519 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:30:46.471489 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:30:46.593712 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:30:46.593679 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:30:47.580599 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:30:47.580563 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:30:47.681701 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:30:47.681668 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:30:49.113673 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:30:49.113642 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:30:50.202892 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:30:50.202850 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:30:51.471675 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:30:51.471636 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:30:52.708784 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:30:52.708746 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:30:55.670673 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:30:55.670636 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:30:55.743252 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:30:55.743221 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:30:56.714296 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:30:56.714261 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:30:56.845423 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:30:56.845393 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:31:05.354592 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:31:05.354562 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:31:05.444324 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:31:05.444293 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:31:08.496665 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:31:08.496635 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:31:08.590140 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:31:08.590112 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:31:09.229326 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:31:09.229282 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-v6klg"] Jul 06 16:31:09.231760 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:31:09.231711 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-v6klg" Jul 06 16:31:09.238316 ip-10-0-128-22 kubenswrapper[2584]: E0706 16:31:09.238278 2584 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"kuadrant-operator-controller-manager-dockercfg-n42ft\" is forbidden: User \"system:node:ip-10-0-128-22.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-22.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-n42ft\"" type="*v1.Secret" Jul 06 16:31:09.238645 ip-10-0-128-22 kubenswrapper[2584]: E0706 16:31:09.238576 2584 reflector.go:205] "Failed to watch" err="failed to list *v1.ConfigMap: configmaps \"openshift-service-ca.crt\" is forbidden: User \"system:node:ip-10-0-128-22.ec2.internal\" cannot list resource \"configmaps\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-22.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" type="*v1.ConfigMap" Jul 06 16:31:09.239397 ip-10-0-128-22 kubenswrapper[2584]: E0706 16:31:09.239268 2584 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-86878c769f-v6klg\" is forbidden: User \"system:node:ip-10-0-128-22.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-22.ec2.internal' and this object" podUID="0ac979d6-8f50-4628-bba2-0841a144c06f" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-v6klg" Jul 06 16:31:09.239791 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:31:09.239390 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 06 16:31:09.273251 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:31:09.273214 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-v6klg"] Jul 06 16:31:09.401601 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:31:09.401560 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8vzjb\" (UniqueName: \"kubernetes.io/projected/0ac979d6-8f50-4628-bba2-0841a144c06f-kube-api-access-8vzjb\") pod \"kuadrant-operator-controller-manager-86878c769f-v6klg\" (UID: \"0ac979d6-8f50-4628-bba2-0841a144c06f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-v6klg" Jul 06 16:31:09.401785 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:31:09.401663 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/0ac979d6-8f50-4628-bba2-0841a144c06f-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-v6klg\" (UID: \"0ac979d6-8f50-4628-bba2-0841a144c06f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-v6klg" Jul 06 16:31:09.502633 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:31:09.502541 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8vzjb\" (UniqueName: \"kubernetes.io/projected/0ac979d6-8f50-4628-bba2-0841a144c06f-kube-api-access-8vzjb\") pod \"kuadrant-operator-controller-manager-86878c769f-v6klg\" (UID: \"0ac979d6-8f50-4628-bba2-0841a144c06f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-v6klg" Jul 06 16:31:09.502633 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:31:09.502590 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/0ac979d6-8f50-4628-bba2-0841a144c06f-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-v6klg\" (UID: \"0ac979d6-8f50-4628-bba2-0841a144c06f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-v6klg" Jul 06 16:31:09.503066 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:31:09.502993 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/0ac979d6-8f50-4628-bba2-0841a144c06f-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-v6klg\" (UID: \"0ac979d6-8f50-4628-bba2-0841a144c06f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-v6klg" Jul 06 16:31:10.155893 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:31:10.155864 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-n42ft\"" Jul 06 16:31:10.314400 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:31:10.314367 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 06 16:31:10.326856 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:31:10.326826 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8vzjb\" (UniqueName: \"kubernetes.io/projected/0ac979d6-8f50-4628-bba2-0841a144c06f-kube-api-access-8vzjb\") pod \"kuadrant-operator-controller-manager-86878c769f-v6klg\" (UID: \"0ac979d6-8f50-4628-bba2-0841a144c06f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-v6klg" Jul 06 16:31:10.447155 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:31:10.447114 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-v6klg" Jul 06 16:31:10.598400 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:31:10.598375 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-v6klg"] Jul 06 16:31:10.601123 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:31:10.601094 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0ac979d6_8f50_4628_bba2_0841a144c06f.slice/crio-a34cd3768ad90a92bc91f6f15b7a2ede29d6901afa021c0c2932b887686e19d1 WatchSource:0}: Error finding container a34cd3768ad90a92bc91f6f15b7a2ede29d6901afa021c0c2932b887686e19d1: Status 404 returned error can't find the container with id a34cd3768ad90a92bc91f6f15b7a2ede29d6901afa021c0c2932b887686e19d1 Jul 06 16:31:10.603489 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:31:10.603472 2584 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 06 16:31:11.031168 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:31:11.031129 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:31:11.075331 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:31:11.075296 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-v6klg" event={"ID":"0ac979d6-8f50-4628-bba2-0841a144c06f","Type":"ContainerStarted","Data":"a34cd3768ad90a92bc91f6f15b7a2ede29d6901afa021c0c2932b887686e19d1"} Jul 06 16:31:11.187895 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:31:11.187842 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:31:15.935246 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:31:15.935210 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:31:16.138341 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:31:16.138296 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:31:16.797490 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:31:16.797398 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:31:16.896691 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:31:16.896653 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:31:17.114539 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:31:17.114456 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-v6klg" event={"ID":"0ac979d6-8f50-4628-bba2-0841a144c06f","Type":"ContainerStarted","Data":"fcec35e42cdbe9973db41bbc891c240f1329651bb63dca79854cb0d827b18f8d"} Jul 06 16:31:17.114940 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:31:17.114539 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-v6klg" Jul 06 16:31:17.175188 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:31:17.175118 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-v6klg" podStartSLOduration=2.348733545 podStartE2EDuration="8.175095549s" totalImagesPullingTime="5.826362004s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 16:31:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 16:31:17.168408337 +0000 UTC m=+427.782548459" watchObservedRunningTime="2026-07-06 16:31:17.175095549 +0000 UTC m=+427.789235672" Jul 06 16:31:25.763264 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:31:25.763234 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:31:25.866835 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:31:25.866804 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:31:28.123216 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:31:28.123187 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-v6klg" Jul 06 16:31:28.874189 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:31:28.874159 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:31:29.036000 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:31:29.035965 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:31:31.246785 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:31:31.246671 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:31:31.348566 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:31:31.348533 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:31:45.181694 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:31:45.181663 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:31:45.443509 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:31:45.443422 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:31:47.913913 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:31:47.913876 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:31:48.077269 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:31:48.077235 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:31:48.178923 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:31:48.178839 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:31:48.877156 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:31:48.877126 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:31:49.012058 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:31:49.012027 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:32:10.710395 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:32:10.710357 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:32:23.571833 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:32:23.571791 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:32:23.764111 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:32:23.764079 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:32:31.468341 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:32:31.468313 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:32:31.606177 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:32:31.606149 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:32:31.685838 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:32:31.685808 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:32:32.100814 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:32:32.100780 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:32:34.993532 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:32:34.993502 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:32:35.121551 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:32:35.121520 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:32:35.294286 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:32:35.294207 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:32:36.025017 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:32:36.024984 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:33:06.512161 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:33:06.512072 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:33:06.979464 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:33:06.979429 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:33:07.118676 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:33:07.118645 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-5f4b5b787d-wfpdm"] Jul 06 16:33:07.122042 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:33:07.122018 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-5f4b5b787d-wfpdm" Jul 06 16:33:07.125673 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:33:07.125645 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-gpfr9\"" Jul 06 16:33:07.125806 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:33:07.125734 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-serving-cert\"" Jul 06 16:33:07.135527 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:33:07.135502 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-5f4b5b787d-wfpdm"] Jul 06 16:33:07.168387 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:33:07.168353 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wlxsp\" (UniqueName: \"kubernetes.io/projected/277e2adb-cbb6-428b-9a13-57bcb090522f-kube-api-access-wlxsp\") pod \"maas-api-5f4b5b787d-wfpdm\" (UID: \"277e2adb-cbb6-428b-9a13-57bcb090522f\") " pod="opendatahub/maas-api-5f4b5b787d-wfpdm" Jul 06 16:33:07.168588 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:33:07.168405 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/277e2adb-cbb6-428b-9a13-57bcb090522f-openshift-service-ca\") pod \"maas-api-5f4b5b787d-wfpdm\" (UID: \"277e2adb-cbb6-428b-9a13-57bcb090522f\") " pod="opendatahub/maas-api-5f4b5b787d-wfpdm" Jul 06 16:33:07.168588 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:33:07.168436 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/277e2adb-cbb6-428b-9a13-57bcb090522f-maas-api-tls\") pod \"maas-api-5f4b5b787d-wfpdm\" (UID: \"277e2adb-cbb6-428b-9a13-57bcb090522f\") " pod="opendatahub/maas-api-5f4b5b787d-wfpdm" Jul 06 16:33:07.269574 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:33:07.269484 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/277e2adb-cbb6-428b-9a13-57bcb090522f-maas-api-tls\") pod \"maas-api-5f4b5b787d-wfpdm\" (UID: \"277e2adb-cbb6-428b-9a13-57bcb090522f\") " pod="opendatahub/maas-api-5f4b5b787d-wfpdm" Jul 06 16:33:07.269756 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:33:07.269580 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wlxsp\" (UniqueName: \"kubernetes.io/projected/277e2adb-cbb6-428b-9a13-57bcb090522f-kube-api-access-wlxsp\") pod \"maas-api-5f4b5b787d-wfpdm\" (UID: \"277e2adb-cbb6-428b-9a13-57bcb090522f\") " pod="opendatahub/maas-api-5f4b5b787d-wfpdm" Jul 06 16:33:07.269756 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:33:07.269617 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/277e2adb-cbb6-428b-9a13-57bcb090522f-openshift-service-ca\") pod \"maas-api-5f4b5b787d-wfpdm\" (UID: \"277e2adb-cbb6-428b-9a13-57bcb090522f\") " pod="opendatahub/maas-api-5f4b5b787d-wfpdm" Jul 06 16:33:07.269756 ip-10-0-128-22 kubenswrapper[2584]: E0706 16:33:07.269643 2584 secret.go:189] Couldn't get secret opendatahub/maas-api-serving-cert: secret "maas-api-serving-cert" not found Jul 06 16:33:07.269756 ip-10-0-128-22 kubenswrapper[2584]: E0706 16:33:07.269737 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/277e2adb-cbb6-428b-9a13-57bcb090522f-maas-api-tls podName:277e2adb-cbb6-428b-9a13-57bcb090522f nodeName:}" failed. No retries permitted until 2026-07-06 16:33:07.769705378 +0000 UTC m=+538.383845478 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/277e2adb-cbb6-428b-9a13-57bcb090522f-maas-api-tls") pod "maas-api-5f4b5b787d-wfpdm" (UID: "277e2adb-cbb6-428b-9a13-57bcb090522f") : secret "maas-api-serving-cert" not found Jul 06 16:33:07.270238 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:33:07.270219 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/277e2adb-cbb6-428b-9a13-57bcb090522f-openshift-service-ca\") pod \"maas-api-5f4b5b787d-wfpdm\" (UID: \"277e2adb-cbb6-428b-9a13-57bcb090522f\") " pod="opendatahub/maas-api-5f4b5b787d-wfpdm" Jul 06 16:33:07.279557 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:33:07.279528 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wlxsp\" (UniqueName: \"kubernetes.io/projected/277e2adb-cbb6-428b-9a13-57bcb090522f-kube-api-access-wlxsp\") pod \"maas-api-5f4b5b787d-wfpdm\" (UID: \"277e2adb-cbb6-428b-9a13-57bcb090522f\") " pod="opendatahub/maas-api-5f4b5b787d-wfpdm" Jul 06 16:33:07.425362 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:33:07.425328 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:33:07.772922 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:33:07.772890 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:33:07.774038 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:33:07.774020 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/277e2adb-cbb6-428b-9a13-57bcb090522f-maas-api-tls\") pod \"maas-api-5f4b5b787d-wfpdm\" (UID: \"277e2adb-cbb6-428b-9a13-57bcb090522f\") " pod="opendatahub/maas-api-5f4b5b787d-wfpdm" Jul 06 16:33:07.776353 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:33:07.776323 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/277e2adb-cbb6-428b-9a13-57bcb090522f-maas-api-tls\") pod \"maas-api-5f4b5b787d-wfpdm\" (UID: \"277e2adb-cbb6-428b-9a13-57bcb090522f\") " pod="opendatahub/maas-api-5f4b5b787d-wfpdm" Jul 06 16:33:07.804134 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:33:07.804099 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-7964457696-92twh"] Jul 06 16:33:07.807696 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:33:07.807673 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7964457696-92twh" Jul 06 16:33:07.811275 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:33:07.811256 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-s8ljj\"" Jul 06 16:33:07.812902 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:33:07.812881 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\"" Jul 06 16:33:07.822531 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:33:07.822505 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7964457696-92twh"] Jul 06 16:33:07.874891 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:33:07.874844 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5ghmn\" (UniqueName: \"kubernetes.io/projected/6dd63d9e-d113-46dc-a653-e728d115e34a-kube-api-access-5ghmn\") pod \"payload-pre-processing-7964457696-92twh\" (UID: \"6dd63d9e-d113-46dc-a653-e728d115e34a\") " pod="openshift-ingress/payload-pre-processing-7964457696-92twh" Jul 06 16:33:07.875069 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:33:07.874923 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6dd63d9e-d113-46dc-a653-e728d115e34a-plugins-config-volume\") pod \"payload-pre-processing-7964457696-92twh\" (UID: \"6dd63d9e-d113-46dc-a653-e728d115e34a\") " pod="openshift-ingress/payload-pre-processing-7964457696-92twh" Jul 06 16:33:07.889507 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:33:07.889474 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:33:07.976400 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:33:07.976368 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6dd63d9e-d113-46dc-a653-e728d115e34a-plugins-config-volume\") pod \"payload-pre-processing-7964457696-92twh\" (UID: \"6dd63d9e-d113-46dc-a653-e728d115e34a\") " pod="openshift-ingress/payload-pre-processing-7964457696-92twh" Jul 06 16:33:07.976581 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:33:07.976444 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5ghmn\" (UniqueName: \"kubernetes.io/projected/6dd63d9e-d113-46dc-a653-e728d115e34a-kube-api-access-5ghmn\") pod \"payload-pre-processing-7964457696-92twh\" (UID: \"6dd63d9e-d113-46dc-a653-e728d115e34a\") " pod="openshift-ingress/payload-pre-processing-7964457696-92twh" Jul 06 16:33:07.977173 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:33:07.977154 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6dd63d9e-d113-46dc-a653-e728d115e34a-plugins-config-volume\") pod \"payload-pre-processing-7964457696-92twh\" (UID: \"6dd63d9e-d113-46dc-a653-e728d115e34a\") " pod="openshift-ingress/payload-pre-processing-7964457696-92twh" Jul 06 16:33:08.004928 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:33:08.004897 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5ghmn\" (UniqueName: \"kubernetes.io/projected/6dd63d9e-d113-46dc-a653-e728d115e34a-kube-api-access-5ghmn\") pod \"payload-pre-processing-7964457696-92twh\" (UID: \"6dd63d9e-d113-46dc-a653-e728d115e34a\") " pod="openshift-ingress/payload-pre-processing-7964457696-92twh" Jul 06 16:33:08.032795 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:33:08.032698 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-5f4b5b787d-wfpdm" Jul 06 16:33:08.119187 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:33:08.119107 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7964457696-92twh" Jul 06 16:33:08.184272 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:33:08.184245 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-5f4b5b787d-wfpdm"] Jul 06 16:33:08.187699 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:33:08.187651 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod277e2adb_cbb6_428b_9a13_57bcb090522f.slice/crio-432a9ee3fd9571a1ef1e2a341a907d1864c3a23211996ea60d81f0a7ebce0339 WatchSource:0}: Error finding container 432a9ee3fd9571a1ef1e2a341a907d1864c3a23211996ea60d81f0a7ebce0339: Status 404 returned error can't find the container with id 432a9ee3fd9571a1ef1e2a341a907d1864c3a23211996ea60d81f0a7ebce0339 Jul 06 16:33:08.267341 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:33:08.267302 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7964457696-92twh"] Jul 06 16:33:08.273442 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:33:08.273409 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6dd63d9e_d113_46dc_a653_e728d115e34a.slice/crio-8dee616f56c8c70ff2e106a3779c0b6beebab9de27c47e56074c384f34b85fdd WatchSource:0}: Error finding container 8dee616f56c8c70ff2e106a3779c0b6beebab9de27c47e56074c384f34b85fdd: Status 404 returned error can't find the container with id 8dee616f56c8c70ff2e106a3779c0b6beebab9de27c47e56074c384f34b85fdd Jul 06 16:33:08.629410 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:33:08.629368 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7964457696-92twh" event={"ID":"6dd63d9e-d113-46dc-a653-e728d115e34a","Type":"ContainerStarted","Data":"8dee616f56c8c70ff2e106a3779c0b6beebab9de27c47e56074c384f34b85fdd"} Jul 06 16:33:08.630482 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:33:08.630459 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-5f4b5b787d-wfpdm" event={"ID":"277e2adb-cbb6-428b-9a13-57bcb090522f","Type":"ContainerStarted","Data":"432a9ee3fd9571a1ef1e2a341a907d1864c3a23211996ea60d81f0a7ebce0339"} Jul 06 16:33:10.642808 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:33:10.642760 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-5f4b5b787d-wfpdm" event={"ID":"277e2adb-cbb6-428b-9a13-57bcb090522f","Type":"ContainerStarted","Data":"541389e9c0935ecd572db4467dbd816ca96a43e155938f5657f702b99df3ae97"} Jul 06 16:33:10.643285 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:33:10.642967 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-5f4b5b787d-wfpdm" Jul 06 16:33:10.644600 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:33:10.644569 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7964457696-92twh" event={"ID":"6dd63d9e-d113-46dc-a653-e728d115e34a","Type":"ContainerStarted","Data":"f2b17898b507b128bf1a2318ca0079f4529d4a4bf1130e93ddb08f16730c71a9"} Jul 06 16:33:10.644752 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:33:10.644718 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-7964457696-92twh" Jul 06 16:33:10.729474 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:33:10.729356 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-5f4b5b787d-wfpdm" podStartSLOduration=1.452112714 podStartE2EDuration="3.729333998s" totalImagesPullingTime="2.277221284s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 16:33:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 16:33:10.686333948 +0000 UTC m=+541.300474068" watchObservedRunningTime="2026-07-06 16:33:10.729333998 +0000 UTC m=+541.343474119" Jul 06 16:33:10.729658 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:33:10.729504 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-7964457696-92twh" podStartSLOduration=1.5444825789999999 podStartE2EDuration="3.729467286s" totalImagesPullingTime="2.184984707s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 16:33:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 16:33:10.726224855 +0000 UTC m=+541.340364976" watchObservedRunningTime="2026-07-06 16:33:10.729467286 +0000 UTC m=+541.343607407" Jul 06 16:33:16.655431 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:33:16.655404 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-5f4b5b787d-wfpdm" Jul 06 16:33:16.957476 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:33:16.957434 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:33:17.953295 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:33:17.953267 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:33:18.059410 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:33:18.059376 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:33:21.652267 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:33:21.652239 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-7964457696-92twh" Jul 06 16:33:23.833304 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:33:23.833253 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:33:35.080023 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:33:35.079989 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:33:57.345201 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:33:57.345167 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-m69w8"] Jul 06 16:33:57.350026 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:33:57.350006 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-m69w8" Jul 06 16:33:57.356175 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:33:57.356057 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-vgdcs\"" Jul 06 16:33:57.358366 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:33:57.358346 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 06 16:33:57.358688 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:33:57.358674 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 06 16:33:57.358875 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:33:57.358860 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\"" Jul 06 16:33:57.426657 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:33:57.426615 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8145c2cb-b892-4f0e-bc2e-0198c0a82136-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-m69w8\" (UID: \"8145c2cb-b892-4f0e-bc2e-0198c0a82136\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-m69w8" Jul 06 16:33:57.426883 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:33:57.426681 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ql9vk\" (UniqueName: \"kubernetes.io/projected/8145c2cb-b892-4f0e-bc2e-0198c0a82136-kube-api-access-ql9vk\") pod \"e2e-distinct-simulated-kserve-59975d69f9-m69w8\" (UID: \"8145c2cb-b892-4f0e-bc2e-0198c0a82136\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-m69w8" Jul 06 16:33:57.426883 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:33:57.426752 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8145c2cb-b892-4f0e-bc2e-0198c0a82136-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-m69w8\" (UID: \"8145c2cb-b892-4f0e-bc2e-0198c0a82136\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-m69w8" Jul 06 16:33:57.426883 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:33:57.426781 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8145c2cb-b892-4f0e-bc2e-0198c0a82136-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-m69w8\" (UID: \"8145c2cb-b892-4f0e-bc2e-0198c0a82136\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-m69w8" Jul 06 16:33:57.426883 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:33:57.426824 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8145c2cb-b892-4f0e-bc2e-0198c0a82136-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-m69w8\" (UID: \"8145c2cb-b892-4f0e-bc2e-0198c0a82136\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-m69w8" Jul 06 16:33:57.426883 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:33:57.426862 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8145c2cb-b892-4f0e-bc2e-0198c0a82136-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-m69w8\" (UID: \"8145c2cb-b892-4f0e-bc2e-0198c0a82136\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-m69w8" Jul 06 16:33:57.434653 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:33:57.434621 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-m69w8"] Jul 06 16:33:57.451056 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:33:57.451004 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:33:57.527517 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:33:57.527474 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8145c2cb-b892-4f0e-bc2e-0198c0a82136-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-m69w8\" (UID: \"8145c2cb-b892-4f0e-bc2e-0198c0a82136\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-m69w8" Jul 06 16:33:57.527748 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:33:57.527534 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ql9vk\" (UniqueName: \"kubernetes.io/projected/8145c2cb-b892-4f0e-bc2e-0198c0a82136-kube-api-access-ql9vk\") pod \"e2e-distinct-simulated-kserve-59975d69f9-m69w8\" (UID: \"8145c2cb-b892-4f0e-bc2e-0198c0a82136\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-m69w8" Jul 06 16:33:57.527748 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:33:57.527575 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8145c2cb-b892-4f0e-bc2e-0198c0a82136-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-m69w8\" (UID: \"8145c2cb-b892-4f0e-bc2e-0198c0a82136\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-m69w8" Jul 06 16:33:57.527748 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:33:57.527602 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8145c2cb-b892-4f0e-bc2e-0198c0a82136-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-m69w8\" (UID: \"8145c2cb-b892-4f0e-bc2e-0198c0a82136\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-m69w8" Jul 06 16:33:57.527748 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:33:57.527623 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8145c2cb-b892-4f0e-bc2e-0198c0a82136-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-m69w8\" (UID: \"8145c2cb-b892-4f0e-bc2e-0198c0a82136\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-m69w8" Jul 06 16:33:57.527748 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:33:57.527647 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8145c2cb-b892-4f0e-bc2e-0198c0a82136-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-m69w8\" (UID: \"8145c2cb-b892-4f0e-bc2e-0198c0a82136\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-m69w8" Jul 06 16:33:57.528140 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:33:57.528112 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8145c2cb-b892-4f0e-bc2e-0198c0a82136-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-m69w8\" (UID: \"8145c2cb-b892-4f0e-bc2e-0198c0a82136\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-m69w8" Jul 06 16:33:57.528274 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:33:57.528199 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8145c2cb-b892-4f0e-bc2e-0198c0a82136-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-m69w8\" (UID: \"8145c2cb-b892-4f0e-bc2e-0198c0a82136\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-m69w8" Jul 06 16:33:57.528345 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:33:57.528270 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8145c2cb-b892-4f0e-bc2e-0198c0a82136-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-m69w8\" (UID: \"8145c2cb-b892-4f0e-bc2e-0198c0a82136\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-m69w8" Jul 06 16:33:57.529938 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:33:57.529918 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8145c2cb-b892-4f0e-bc2e-0198c0a82136-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-m69w8\" (UID: \"8145c2cb-b892-4f0e-bc2e-0198c0a82136\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-m69w8" Jul 06 16:33:57.530199 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:33:57.530182 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8145c2cb-b892-4f0e-bc2e-0198c0a82136-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-m69w8\" (UID: \"8145c2cb-b892-4f0e-bc2e-0198c0a82136\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-m69w8" Jul 06 16:33:57.541842 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:33:57.541813 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ql9vk\" (UniqueName: \"kubernetes.io/projected/8145c2cb-b892-4f0e-bc2e-0198c0a82136-kube-api-access-ql9vk\") pod \"e2e-distinct-simulated-kserve-59975d69f9-m69w8\" (UID: \"8145c2cb-b892-4f0e-bc2e-0198c0a82136\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-m69w8" Jul 06 16:33:57.660207 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:33:57.660117 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-m69w8" Jul 06 16:33:57.818533 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:33:57.818448 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-m69w8"] Jul 06 16:33:57.821211 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:33:57.821185 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8145c2cb_b892_4f0e_bc2e_0198c0a82136.slice/crio-191a49dea813d3bf6386cb09cc204e5cdb4d19728c78b0a3b3469f47e4114dad WatchSource:0}: Error finding container 191a49dea813d3bf6386cb09cc204e5cdb4d19728c78b0a3b3469f47e4114dad: Status 404 returned error can't find the container with id 191a49dea813d3bf6386cb09cc204e5cdb4d19728c78b0a3b3469f47e4114dad Jul 06 16:33:57.870244 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:33:57.870210 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-m69w8" event={"ID":"8145c2cb-b892-4f0e-bc2e-0198c0a82136","Type":"ContainerStarted","Data":"191a49dea813d3bf6386cb09cc204e5cdb4d19728c78b0a3b3469f47e4114dad"} Jul 06 16:33:59.881130 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:33:59.881097 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-m69w8" event={"ID":"8145c2cb-b892-4f0e-bc2e-0198c0a82136","Type":"ContainerStarted","Data":"375883ed314ef907a0060e2eebbfb9e5382409127a51eab887f7ab14c926598b"} Jul 06 16:33:59.915414 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:33:59.915347 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-m69w8" podStartSLOduration=1.893853128 podStartE2EDuration="2.915329314s" totalImagesPullingTime="1.021476186s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 16:33:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 16:33:59.91416231 +0000 UTC m=+590.528302442" watchObservedRunningTime="2026-07-06 16:33:59.915329314 +0000 UTC m=+590.529469435" Jul 06 16:34:04.500113 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:34:04.500078 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:34:04.603016 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:34:04.602989 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:34:06.506105 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:34:06.506067 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:34:07.660291 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:34:07.660244 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-m69w8" Jul 06 16:34:07.660712 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:34:07.660366 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-m69w8" Jul 06 16:34:07.673429 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:34:07.673397 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-m69w8" Jul 06 16:34:07.936742 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:34:07.936698 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-m69w8" Jul 06 16:34:09.888776 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:34:09.888747 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-294bx_6a8a4796-dbbc-43e2-a459-7acf11f9468d/ovn-acl-logging/0.log" Jul 06 16:34:09.889756 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:34:09.889717 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-294bx_6a8a4796-dbbc-43e2-a459-7acf11f9468d/ovn-acl-logging/0.log" Jul 06 16:34:14.517352 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:34:14.517311 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:34:29.270941 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:34:29.270905 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbqtx98"] Jul 06 16:34:29.277319 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:34:29.277290 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbqtx98" Jul 06 16:34:29.280898 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:34:29.280869 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\"" Jul 06 16:34:29.287752 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:34:29.287710 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbqtx98"] Jul 06 16:34:29.333710 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:34:29.333676 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e24aae4f-7b71-4aac-b0bb-d80a238f053d-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbqtx98\" (UID: \"e24aae4f-7b71-4aac-b0bb-d80a238f053d\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbqtx98" Jul 06 16:34:29.333905 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:34:29.333783 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e24aae4f-7b71-4aac-b0bb-d80a238f053d-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbqtx98\" (UID: \"e24aae4f-7b71-4aac-b0bb-d80a238f053d\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbqtx98" Jul 06 16:34:29.333905 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:34:29.333824 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q9llb\" (UniqueName: \"kubernetes.io/projected/e24aae4f-7b71-4aac-b0bb-d80a238f053d-kube-api-access-q9llb\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbqtx98\" (UID: \"e24aae4f-7b71-4aac-b0bb-d80a238f053d\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbqtx98" Jul 06 16:34:29.333905 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:34:29.333851 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e24aae4f-7b71-4aac-b0bb-d80a238f053d-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbqtx98\" (UID: \"e24aae4f-7b71-4aac-b0bb-d80a238f053d\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbqtx98" Jul 06 16:34:29.333905 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:34:29.333879 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e24aae4f-7b71-4aac-b0bb-d80a238f053d-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbqtx98\" (UID: \"e24aae4f-7b71-4aac-b0bb-d80a238f053d\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbqtx98" Jul 06 16:34:29.334041 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:34:29.333911 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e24aae4f-7b71-4aac-b0bb-d80a238f053d-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbqtx98\" (UID: \"e24aae4f-7b71-4aac-b0bb-d80a238f053d\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbqtx98" Jul 06 16:34:29.377542 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:34:29.377514 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:34:29.434892 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:34:29.434857 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e24aae4f-7b71-4aac-b0bb-d80a238f053d-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbqtx98\" (UID: \"e24aae4f-7b71-4aac-b0bb-d80a238f053d\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbqtx98" Jul 06 16:34:29.435056 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:34:29.434902 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e24aae4f-7b71-4aac-b0bb-d80a238f053d-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbqtx98\" (UID: \"e24aae4f-7b71-4aac-b0bb-d80a238f053d\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbqtx98" Jul 06 16:34:29.435056 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:34:29.435008 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e24aae4f-7b71-4aac-b0bb-d80a238f053d-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbqtx98\" (UID: \"e24aae4f-7b71-4aac-b0bb-d80a238f053d\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbqtx98" Jul 06 16:34:29.435056 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:34:29.435042 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e24aae4f-7b71-4aac-b0bb-d80a238f053d-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbqtx98\" (UID: \"e24aae4f-7b71-4aac-b0bb-d80a238f053d\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbqtx98" Jul 06 16:34:29.435225 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:34:29.435092 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e24aae4f-7b71-4aac-b0bb-d80a238f053d-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbqtx98\" (UID: \"e24aae4f-7b71-4aac-b0bb-d80a238f053d\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbqtx98" Jul 06 16:34:29.435225 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:34:29.435124 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q9llb\" (UniqueName: \"kubernetes.io/projected/e24aae4f-7b71-4aac-b0bb-d80a238f053d-kube-api-access-q9llb\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbqtx98\" (UID: \"e24aae4f-7b71-4aac-b0bb-d80a238f053d\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbqtx98" Jul 06 16:34:29.435452 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:34:29.435420 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e24aae4f-7b71-4aac-b0bb-d80a238f053d-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbqtx98\" (UID: \"e24aae4f-7b71-4aac-b0bb-d80a238f053d\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbqtx98" Jul 06 16:34:29.435561 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:34:29.435485 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e24aae4f-7b71-4aac-b0bb-d80a238f053d-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbqtx98\" (UID: \"e24aae4f-7b71-4aac-b0bb-d80a238f053d\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbqtx98" Jul 06 16:34:29.435561 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:34:29.435516 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e24aae4f-7b71-4aac-b0bb-d80a238f053d-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbqtx98\" (UID: \"e24aae4f-7b71-4aac-b0bb-d80a238f053d\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbqtx98" Jul 06 16:34:29.437306 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:34:29.437286 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e24aae4f-7b71-4aac-b0bb-d80a238f053d-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbqtx98\" (UID: \"e24aae4f-7b71-4aac-b0bb-d80a238f053d\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbqtx98" Jul 06 16:34:29.437543 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:34:29.437525 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e24aae4f-7b71-4aac-b0bb-d80a238f053d-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbqtx98\" (UID: \"e24aae4f-7b71-4aac-b0bb-d80a238f053d\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbqtx98" Jul 06 16:34:29.450292 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:34:29.450263 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q9llb\" (UniqueName: \"kubernetes.io/projected/e24aae4f-7b71-4aac-b0bb-d80a238f053d-kube-api-access-q9llb\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbqtx98\" (UID: \"e24aae4f-7b71-4aac-b0bb-d80a238f053d\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbqtx98" Jul 06 16:34:29.588826 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:34:29.588733 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbqtx98" Jul 06 16:34:29.735119 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:34:29.735087 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbqtx98"] Jul 06 16:34:29.739387 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:34:29.739361 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode24aae4f_7b71_4aac_b0bb_d80a238f053d.slice/crio-4d1e220e79a39b6b380665d3646414715b1002263899e0bcac04e62da214431b WatchSource:0}: Error finding container 4d1e220e79a39b6b380665d3646414715b1002263899e0bcac04e62da214431b: Status 404 returned error can't find the container with id 4d1e220e79a39b6b380665d3646414715b1002263899e0bcac04e62da214431b Jul 06 16:34:30.031797 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:34:30.031764 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbqtx98" event={"ID":"e24aae4f-7b71-4aac-b0bb-d80a238f053d","Type":"ContainerStarted","Data":"488ae2fae82b226b3400a686eaf796cb733ac2e086033e45a5c2e7caa22a0c94"} Jul 06 16:34:30.031968 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:34:30.031816 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbqtx98" event={"ID":"e24aae4f-7b71-4aac-b0bb-d80a238f053d","Type":"ContainerStarted","Data":"4d1e220e79a39b6b380665d3646414715b1002263899e0bcac04e62da214431b"} Jul 06 16:34:30.083601 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:34:30.083467 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbqtx98" podStartSLOduration=0.898339807 podStartE2EDuration="1.083441158s" totalImagesPullingTime="185.101351ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 16:34:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 16:34:30.078653653 +0000 UTC m=+620.692793774" watchObservedRunningTime="2026-07-06 16:34:30.083441158 +0000 UTC m=+620.697581278" Jul 06 16:34:36.512531 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:34:36.512434 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:34:39.589140 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:34:39.589103 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbqtx98" Jul 06 16:34:39.589676 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:34:39.589172 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbqtx98" Jul 06 16:34:39.601787 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:34:39.601759 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbqtx98" Jul 06 16:34:40.091399 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:34:40.091373 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbqtx98" Jul 06 16:34:55.324483 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:34:55.324436 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:35:37.469322 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:35:37.469286 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:35:37.562948 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:35:37.562903 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:35:54.629261 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:35:54.629226 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:35:54.725489 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:35:54.725440 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:35:55.896809 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:35:55.896777 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:35:55.994295 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:35:55.994263 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:36:03.783382 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:36:03.783295 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:37:04.601073 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:37:04.601034 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:37:07.051534 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:37:07.051495 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:37:08.458348 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:37:08.458315 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:37:34.672079 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:37:34.671994 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:37:50.120031 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:37:50.119994 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:37:50.271071 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:37:50.271026 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:37:50.374847 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:37:50.374755 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:37:57.677665 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:37:57.677627 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:37:57.769662 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:37:57.769634 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:37:57.879595 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:37:57.879565 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:37:58.049883 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:37:58.049809 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:38:05.442960 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:38:05.442928 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:38:05.532268 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:38:05.532240 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:39:09.925713 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:39:09.925632 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-294bx_6a8a4796-dbbc-43e2-a459-7acf11f9468d/ovn-acl-logging/0.log" Jul 06 16:39:09.926417 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:39:09.926394 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-294bx_6a8a4796-dbbc-43e2-a459-7acf11f9468d/ovn-acl-logging/0.log" Jul 06 16:39:11.383560 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:39:11.383508 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:39:12.936498 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:39:12.936449 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:41:06.000276 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:41:06.000246 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:41:24.851419 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:41:24.851375 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:42:19.747744 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:42:19.747639 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-v6klg"] Jul 06 16:42:19.748319 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:42:19.747880 2584 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-v6klg" podUID="0ac979d6-8f50-4628-bba2-0841a144c06f" containerName="manager" containerID="cri-o://fcec35e42cdbe9973db41bbc891c240f1329651bb63dca79854cb0d827b18f8d" gracePeriod=10 Jul 06 16:42:24.104289 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:42:24.104262 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-v6klg" Jul 06 16:42:24.285269 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:42:24.285180 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/0ac979d6-8f50-4628-bba2-0841a144c06f-extensions-socket-volume\") pod \"0ac979d6-8f50-4628-bba2-0841a144c06f\" (UID: \"0ac979d6-8f50-4628-bba2-0841a144c06f\") " Jul 06 16:42:24.285269 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:42:24.285232 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8vzjb\" (UniqueName: \"kubernetes.io/projected/0ac979d6-8f50-4628-bba2-0841a144c06f-kube-api-access-8vzjb\") pod \"0ac979d6-8f50-4628-bba2-0841a144c06f\" (UID: \"0ac979d6-8f50-4628-bba2-0841a144c06f\") " Jul 06 16:42:24.285561 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:42:24.285537 2584 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0ac979d6-8f50-4628-bba2-0841a144c06f-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "0ac979d6-8f50-4628-bba2-0841a144c06f" (UID: "0ac979d6-8f50-4628-bba2-0841a144c06f"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 16:42:24.287490 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:42:24.287456 2584 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0ac979d6-8f50-4628-bba2-0841a144c06f-kube-api-access-8vzjb" (OuterVolumeSpecName: "kube-api-access-8vzjb") pod "0ac979d6-8f50-4628-bba2-0841a144c06f" (UID: "0ac979d6-8f50-4628-bba2-0841a144c06f"). InnerVolumeSpecName "kube-api-access-8vzjb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 16:42:24.354115 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:42:24.354078 2584 generic.go:356] "Generic (PLEG): container finished" podID="0ac979d6-8f50-4628-bba2-0841a144c06f" containerID="fcec35e42cdbe9973db41bbc891c240f1329651bb63dca79854cb0d827b18f8d" exitCode=0 Jul 06 16:42:24.354306 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:42:24.354120 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-v6klg" event={"ID":"0ac979d6-8f50-4628-bba2-0841a144c06f","Type":"ContainerDied","Data":"fcec35e42cdbe9973db41bbc891c240f1329651bb63dca79854cb0d827b18f8d"} Jul 06 16:42:24.354306 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:42:24.354167 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-v6klg" event={"ID":"0ac979d6-8f50-4628-bba2-0841a144c06f","Type":"ContainerDied","Data":"a34cd3768ad90a92bc91f6f15b7a2ede29d6901afa021c0c2932b887686e19d1"} Jul 06 16:42:24.354306 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:42:24.354194 2584 scope.go:117] "RemoveContainer" containerID="fcec35e42cdbe9973db41bbc891c240f1329651bb63dca79854cb0d827b18f8d" Jul 06 16:42:24.354306 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:42:24.354122 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-v6klg" Jul 06 16:42:24.363444 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:42:24.363421 2584 scope.go:117] "RemoveContainer" containerID="fcec35e42cdbe9973db41bbc891c240f1329651bb63dca79854cb0d827b18f8d" Jul 06 16:42:24.363767 ip-10-0-128-22 kubenswrapper[2584]: E0706 16:42:24.363746 2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fcec35e42cdbe9973db41bbc891c240f1329651bb63dca79854cb0d827b18f8d\": container with ID starting with fcec35e42cdbe9973db41bbc891c240f1329651bb63dca79854cb0d827b18f8d not found: ID does not exist" containerID="fcec35e42cdbe9973db41bbc891c240f1329651bb63dca79854cb0d827b18f8d" Jul 06 16:42:24.363837 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:42:24.363777 2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fcec35e42cdbe9973db41bbc891c240f1329651bb63dca79854cb0d827b18f8d"} err="failed to get container status \"fcec35e42cdbe9973db41bbc891c240f1329651bb63dca79854cb0d827b18f8d\": rpc error: code = NotFound desc = could not find container \"fcec35e42cdbe9973db41bbc891c240f1329651bb63dca79854cb0d827b18f8d\": container with ID starting with fcec35e42cdbe9973db41bbc891c240f1329651bb63dca79854cb0d827b18f8d not found: ID does not exist" Jul 06 16:42:24.385869 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:42:24.385841 2584 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/0ac979d6-8f50-4628-bba2-0841a144c06f-extensions-socket-volume\") on node \"ip-10-0-128-22.ec2.internal\" DevicePath \"\"" Jul 06 16:42:24.385869 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:42:24.385866 2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8vzjb\" (UniqueName: \"kubernetes.io/projected/0ac979d6-8f50-4628-bba2-0841a144c06f-kube-api-access-8vzjb\") on node \"ip-10-0-128-22.ec2.internal\" DevicePath \"\"" Jul 06 16:42:24.408456 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:42:24.408424 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-v6klg"] Jul 06 16:42:24.412771 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:42:24.412745 2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-v6klg"] Jul 06 16:42:26.118505 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:42:26.118463 2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0ac979d6-8f50-4628-bba2-0841a144c06f" path="/var/lib/kubelet/pods/0ac979d6-8f50-4628-bba2-0841a144c06f/volumes" Jul 06 16:42:29.015633 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:42:29.015599 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:42:29.483327 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:42:29.483286 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:43:20.940171 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:43:20.940133 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7hbqb"] Jul 06 16:43:20.940591 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:43:20.940551 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0ac979d6-8f50-4628-bba2-0841a144c06f" containerName="manager" Jul 06 16:43:20.940591 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:43:20.940564 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="0ac979d6-8f50-4628-bba2-0841a144c06f" containerName="manager" Jul 06 16:43:20.940666 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:43:20.940625 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="0ac979d6-8f50-4628-bba2-0841a144c06f" containerName="manager" Jul 06 16:43:20.943165 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:43:20.943148 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7hbqb" Jul 06 16:43:20.946943 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:43:20.946917 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 06 16:43:20.947086 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:43:20.947055 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-n42ft\"" Jul 06 16:43:20.947138 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:43:20.947094 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 06 16:43:20.960953 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:43:20.960926 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7hbqb"] Jul 06 16:43:21.005098 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:43:21.005055 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7a9f90da-048f-424b-a819-591e54a697b7-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-7hbqb\" (UID: \"7a9f90da-048f-424b-a819-591e54a697b7\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7hbqb" Jul 06 16:43:21.005271 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:43:21.005156 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mrc4z\" (UniqueName: \"kubernetes.io/projected/7a9f90da-048f-424b-a819-591e54a697b7-kube-api-access-mrc4z\") pod \"kuadrant-operator-controller-manager-86878c769f-7hbqb\" (UID: \"7a9f90da-048f-424b-a819-591e54a697b7\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7hbqb" Jul 06 16:43:21.106104 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:43:21.106060 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7a9f90da-048f-424b-a819-591e54a697b7-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-7hbqb\" (UID: \"7a9f90da-048f-424b-a819-591e54a697b7\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7hbqb" Jul 06 16:43:21.106280 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:43:21.106150 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mrc4z\" (UniqueName: \"kubernetes.io/projected/7a9f90da-048f-424b-a819-591e54a697b7-kube-api-access-mrc4z\") pod \"kuadrant-operator-controller-manager-86878c769f-7hbqb\" (UID: \"7a9f90da-048f-424b-a819-591e54a697b7\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7hbqb" Jul 06 16:43:21.106535 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:43:21.106514 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7a9f90da-048f-424b-a819-591e54a697b7-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-7hbqb\" (UID: \"7a9f90da-048f-424b-a819-591e54a697b7\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7hbqb" Jul 06 16:43:21.128822 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:43:21.128794 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mrc4z\" (UniqueName: \"kubernetes.io/projected/7a9f90da-048f-424b-a819-591e54a697b7-kube-api-access-mrc4z\") pod \"kuadrant-operator-controller-manager-86878c769f-7hbqb\" (UID: \"7a9f90da-048f-424b-a819-591e54a697b7\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7hbqb" Jul 06 16:43:21.253806 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:43:21.253679 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7hbqb" Jul 06 16:43:21.441526 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:43:21.441489 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7a9f90da_048f_424b_a819_591e54a697b7.slice/crio-4c5eca8087f766fb7d93cc6f84346858eacbb5d3873819d2e8efa44d0e8d92a1 WatchSource:0}: Error finding container 4c5eca8087f766fb7d93cc6f84346858eacbb5d3873819d2e8efa44d0e8d92a1: Status 404 returned error can't find the container with id 4c5eca8087f766fb7d93cc6f84346858eacbb5d3873819d2e8efa44d0e8d92a1 Jul 06 16:43:21.442399 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:43:21.442376 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7hbqb"] Jul 06 16:43:21.625711 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:43:21.625610 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7hbqb" event={"ID":"7a9f90da-048f-424b-a819-591e54a697b7","Type":"ContainerStarted","Data":"8a91d75713bad033190da2a079e935553a6af2aeafa297e3cedf13745df36246"} Jul 06 16:43:21.625711 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:43:21.625671 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7hbqb" event={"ID":"7a9f90da-048f-424b-a819-591e54a697b7","Type":"ContainerStarted","Data":"4c5eca8087f766fb7d93cc6f84346858eacbb5d3873819d2e8efa44d0e8d92a1"} Jul 06 16:43:21.625904 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:43:21.625840 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7hbqb" Jul 06 16:43:21.652193 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:43:21.652140 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7hbqb" podStartSLOduration=1.652122033 podStartE2EDuration="1.652122033s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 16:43:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 16:43:21.650862928 +0000 UTC m=+1152.265003049" watchObservedRunningTime="2026-07-06 16:43:21.652122033 +0000 UTC m=+1152.266262154" Jul 06 16:43:32.632694 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:43:32.632603 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7hbqb" Jul 06 16:44:09.956291 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:44:09.956263 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-294bx_6a8a4796-dbbc-43e2-a459-7acf11f9468d/ovn-acl-logging/0.log" Jul 06 16:44:09.958637 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:44:09.958617 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-294bx_6a8a4796-dbbc-43e2-a459-7acf11f9468d/ovn-acl-logging/0.log" Jul 06 16:47:42.270748 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:47:42.270703 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:47:43.726407 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:47:43.726366 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:49:09.987775 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:49:09.987748 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-294bx_6a8a4796-dbbc-43e2-a459-7acf11f9468d/ovn-acl-logging/0.log" Jul 06 16:49:09.991745 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:49:09.991712 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-294bx_6a8a4796-dbbc-43e2-a459-7acf11f9468d/ovn-acl-logging/0.log" Jul 06 16:49:10.057692 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:49:10.057658 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:49:16.679277 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:49:16.679241 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:49:24.920131 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:49:24.920081 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:49:30.134335 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:49:30.134297 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:49:33.082449 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:49:33.082365 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:49:33.182803 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:49:33.182762 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:49:33.591108 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:49:33.591070 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:49:40.448784 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:49:40.448745 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:49:40.525681 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:49:40.525617 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:49:40.680195 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:49:40.680163 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:49:40.787574 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:49:40.787484 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:49:41.852524 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:49:41.852491 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:49:41.936908 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:49:41.936880 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-ait-7efb30b2-697548f498-zw6hh"] Jul 06 16:49:41.940431 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:49:41.940408 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-7efb30b2-697548f498-zw6hh" Jul 06 16:49:41.944321 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:49:41.944300 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-ait-7efb30b2-serving-cert\"" Jul 06 16:49:41.944656 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:49:41.944617 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:49:41.956919 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:49:41.956896 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-ait-7efb30b2-697548f498-zw6hh"] Jul 06 16:49:41.975091 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:49:41.975053 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ad25326c-554c-46e2-ac34-3d5a577c288a-maas-api-tls\") pod \"maas-api-e2e-ait-7efb30b2-697548f498-zw6hh\" (UID: \"ad25326c-554c-46e2-ac34-3d5a577c288a\") " pod="opendatahub/maas-api-e2e-ait-7efb30b2-697548f498-zw6hh" Jul 06 16:49:41.975263 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:49:41.975098 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rvd6f\" (UniqueName: \"kubernetes.io/projected/ad25326c-554c-46e2-ac34-3d5a577c288a-kube-api-access-rvd6f\") pod \"maas-api-e2e-ait-7efb30b2-697548f498-zw6hh\" (UID: \"ad25326c-554c-46e2-ac34-3d5a577c288a\") " pod="opendatahub/maas-api-e2e-ait-7efb30b2-697548f498-zw6hh" Jul 06 16:49:41.975307 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:49:41.975254 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ad25326c-554c-46e2-ac34-3d5a577c288a-openshift-service-ca\") pod \"maas-api-e2e-ait-7efb30b2-697548f498-zw6hh\" (UID: \"ad25326c-554c-46e2-ac34-3d5a577c288a\") " pod="opendatahub/maas-api-e2e-ait-7efb30b2-697548f498-zw6hh" Jul 06 16:49:42.064142 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:49:42.064105 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:49:42.076289 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:49:42.076260 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ad25326c-554c-46e2-ac34-3d5a577c288a-openshift-service-ca\") pod \"maas-api-e2e-ait-7efb30b2-697548f498-zw6hh\" (UID: \"ad25326c-554c-46e2-ac34-3d5a577c288a\") " pod="opendatahub/maas-api-e2e-ait-7efb30b2-697548f498-zw6hh" Jul 06 16:49:42.076454 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:49:42.076316 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ad25326c-554c-46e2-ac34-3d5a577c288a-maas-api-tls\") pod \"maas-api-e2e-ait-7efb30b2-697548f498-zw6hh\" (UID: \"ad25326c-554c-46e2-ac34-3d5a577c288a\") " pod="opendatahub/maas-api-e2e-ait-7efb30b2-697548f498-zw6hh" Jul 06 16:49:42.076454 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:49:42.076344 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rvd6f\" (UniqueName: \"kubernetes.io/projected/ad25326c-554c-46e2-ac34-3d5a577c288a-kube-api-access-rvd6f\") pod \"maas-api-e2e-ait-7efb30b2-697548f498-zw6hh\" (UID: \"ad25326c-554c-46e2-ac34-3d5a577c288a\") " pod="opendatahub/maas-api-e2e-ait-7efb30b2-697548f498-zw6hh" Jul 06 16:49:42.076568 ip-10-0-128-22 kubenswrapper[2584]: E0706 16:49:42.076489 2584 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-ait-7efb30b2-serving-cert: secret "maas-api-e2e-ait-7efb30b2-serving-cert" not found Jul 06 16:49:42.076568 ip-10-0-128-22 kubenswrapper[2584]: E0706 16:49:42.076556 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ad25326c-554c-46e2-ac34-3d5a577c288a-maas-api-tls podName:ad25326c-554c-46e2-ac34-3d5a577c288a nodeName:}" failed. No retries permitted until 2026-07-06 16:49:42.576538343 +0000 UTC m=+1533.190678441 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/ad25326c-554c-46e2-ac34-3d5a577c288a-maas-api-tls") pod "maas-api-e2e-ait-7efb30b2-697548f498-zw6hh" (UID: "ad25326c-554c-46e2-ac34-3d5a577c288a") : secret "maas-api-e2e-ait-7efb30b2-serving-cert" not found Jul 06 16:49:42.077010 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:49:42.076986 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ad25326c-554c-46e2-ac34-3d5a577c288a-openshift-service-ca\") pod \"maas-api-e2e-ait-7efb30b2-697548f498-zw6hh\" (UID: \"ad25326c-554c-46e2-ac34-3d5a577c288a\") " pod="opendatahub/maas-api-e2e-ait-7efb30b2-697548f498-zw6hh" Jul 06 16:49:42.090175 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:49:42.090146 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rvd6f\" (UniqueName: \"kubernetes.io/projected/ad25326c-554c-46e2-ac34-3d5a577c288a-kube-api-access-rvd6f\") pod \"maas-api-e2e-ait-7efb30b2-697548f498-zw6hh\" (UID: \"ad25326c-554c-46e2-ac34-3d5a577c288a\") " pod="opendatahub/maas-api-e2e-ait-7efb30b2-697548f498-zw6hh" Jul 06 16:49:42.227981 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:49:42.227945 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:49:42.580753 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:49:42.580632 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ad25326c-554c-46e2-ac34-3d5a577c288a-maas-api-tls\") pod \"maas-api-e2e-ait-7efb30b2-697548f498-zw6hh\" (UID: \"ad25326c-554c-46e2-ac34-3d5a577c288a\") " pod="opendatahub/maas-api-e2e-ait-7efb30b2-697548f498-zw6hh" Jul 06 16:49:42.583282 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:49:42.583246 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ad25326c-554c-46e2-ac34-3d5a577c288a-maas-api-tls\") pod \"maas-api-e2e-ait-7efb30b2-697548f498-zw6hh\" (UID: \"ad25326c-554c-46e2-ac34-3d5a577c288a\") " pod="opendatahub/maas-api-e2e-ait-7efb30b2-697548f498-zw6hh" Jul 06 16:49:42.761996 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:49:42.761964 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:49:42.854111 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:49:42.854022 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-7efb30b2-697548f498-zw6hh" Jul 06 16:49:42.866280 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:49:42.866250 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:49:43.014527 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:49:43.014503 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-ait-7efb30b2-697548f498-zw6hh"] Jul 06 16:49:43.016381 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:49:43.016352 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podad25326c_554c_46e2_ac34_3d5a577c288a.slice/crio-988ea14f8f8e40c15f7ade34dc820984921126b141db05b05b8d646e656cb70d WatchSource:0}: Error finding container 988ea14f8f8e40c15f7ade34dc820984921126b141db05b05b8d646e656cb70d: Status 404 returned error can't find the container with id 988ea14f8f8e40c15f7ade34dc820984921126b141db05b05b8d646e656cb70d Jul 06 16:49:43.017780 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:49:43.017763 2584 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 06 16:49:43.475571 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:49:43.475535 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-7efb30b2-697548f498-zw6hh" event={"ID":"ad25326c-554c-46e2-ac34-3d5a577c288a","Type":"ContainerStarted","Data":"988ea14f8f8e40c15f7ade34dc820984921126b141db05b05b8d646e656cb70d"} Jul 06 16:49:44.482321 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:49:44.482287 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-7efb30b2-697548f498-zw6hh" event={"ID":"ad25326c-554c-46e2-ac34-3d5a577c288a","Type":"ContainerStarted","Data":"7326276104f08613c455975cf42658f5ca69f8eb0288789306ad961bfb1ba290"} Jul 06 16:49:44.482737 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:49:44.482486 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-ait-7efb30b2-697548f498-zw6hh" Jul 06 16:49:44.505841 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:49:44.505788 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-ait-7efb30b2-697548f498-zw6hh" podStartSLOduration=2.867491789 podStartE2EDuration="3.505769669s" totalImagesPullingTime="638.27788ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 16:49:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 16:49:44.503226448 +0000 UTC m=+1535.117366570" watchObservedRunningTime="2026-07-06 16:49:44.505769669 +0000 UTC m=+1535.119909790" Jul 06 16:49:44.512630 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:49:44.512603 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:49:46.978878 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:49:46.978830 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:49:47.184473 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:49:47.184441 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:49:47.280043 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:49:47.279953 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:49:47.363708 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:49:47.363678 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-ait-7efb30b2-697548f498-zw6hh"] Jul 06 16:49:47.363949 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:49:47.363923 2584 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-ait-7efb30b2-697548f498-zw6hh" podUID="ad25326c-554c-46e2-ac34-3d5a577c288a" containerName="maas-api" containerID="cri-o://7326276104f08613c455975cf42658f5ca69f8eb0288789306ad961bfb1ba290" gracePeriod=30 Jul 06 16:49:47.392086 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:49:47.392047 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:49:47.499156 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:49:47.499125 2584 generic.go:356] "Generic (PLEG): container finished" podID="ad25326c-554c-46e2-ac34-3d5a577c288a" containerID="7326276104f08613c455975cf42658f5ca69f8eb0288789306ad961bfb1ba290" exitCode=0 Jul 06 16:49:47.499312 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:49:47.499198 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-7efb30b2-697548f498-zw6hh" event={"ID":"ad25326c-554c-46e2-ac34-3d5a577c288a","Type":"ContainerDied","Data":"7326276104f08613c455975cf42658f5ca69f8eb0288789306ad961bfb1ba290"} Jul 06 16:49:47.623863 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:49:47.623557 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-7efb30b2-697548f498-zw6hh" Jul 06 16:49:47.730570 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:49:47.730531 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ad25326c-554c-46e2-ac34-3d5a577c288a-maas-api-tls\") pod \"ad25326c-554c-46e2-ac34-3d5a577c288a\" (UID: \"ad25326c-554c-46e2-ac34-3d5a577c288a\") " Jul 06 16:49:47.730786 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:49:47.730608 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rvd6f\" (UniqueName: \"kubernetes.io/projected/ad25326c-554c-46e2-ac34-3d5a577c288a-kube-api-access-rvd6f\") pod \"ad25326c-554c-46e2-ac34-3d5a577c288a\" (UID: \"ad25326c-554c-46e2-ac34-3d5a577c288a\") " Jul 06 16:49:47.730786 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:49:47.730656 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ad25326c-554c-46e2-ac34-3d5a577c288a-openshift-service-ca\") pod \"ad25326c-554c-46e2-ac34-3d5a577c288a\" (UID: \"ad25326c-554c-46e2-ac34-3d5a577c288a\") " Jul 06 16:49:47.731047 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:49:47.731018 2584 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ad25326c-554c-46e2-ac34-3d5a577c288a-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "ad25326c-554c-46e2-ac34-3d5a577c288a" (UID: "ad25326c-554c-46e2-ac34-3d5a577c288a"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 16:49:47.732843 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:49:47.732821 2584 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ad25326c-554c-46e2-ac34-3d5a577c288a-kube-api-access-rvd6f" (OuterVolumeSpecName: "kube-api-access-rvd6f") pod "ad25326c-554c-46e2-ac34-3d5a577c288a" (UID: "ad25326c-554c-46e2-ac34-3d5a577c288a"). InnerVolumeSpecName "kube-api-access-rvd6f". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 16:49:47.733004 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:49:47.732978 2584 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ad25326c-554c-46e2-ac34-3d5a577c288a-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "ad25326c-554c-46e2-ac34-3d5a577c288a" (UID: "ad25326c-554c-46e2-ac34-3d5a577c288a"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 16:49:47.832095 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:49:47.832009 2584 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ad25326c-554c-46e2-ac34-3d5a577c288a-openshift-service-ca\") on node \"ip-10-0-128-22.ec2.internal\" DevicePath \"\"" Jul 06 16:49:47.832095 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:49:47.832040 2584 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ad25326c-554c-46e2-ac34-3d5a577c288a-maas-api-tls\") on node \"ip-10-0-128-22.ec2.internal\" DevicePath \"\"" Jul 06 16:49:47.832095 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:49:47.832049 2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rvd6f\" (UniqueName: \"kubernetes.io/projected/ad25326c-554c-46e2-ac34-3d5a577c288a-kube-api-access-rvd6f\") on node \"ip-10-0-128-22.ec2.internal\" DevicePath \"\"" Jul 06 16:49:48.506380 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:49:48.506344 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-7efb30b2-697548f498-zw6hh" event={"ID":"ad25326c-554c-46e2-ac34-3d5a577c288a","Type":"ContainerDied","Data":"988ea14f8f8e40c15f7ade34dc820984921126b141db05b05b8d646e656cb70d"} Jul 06 16:49:48.506380 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:49:48.506378 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-7efb30b2-697548f498-zw6hh" Jul 06 16:49:48.506864 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:49:48.506423 2584 scope.go:117] "RemoveContainer" containerID="7326276104f08613c455975cf42658f5ca69f8eb0288789306ad961bfb1ba290" Jul 06 16:49:48.520173 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:49:48.520142 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:49:48.527392 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:49:48.527371 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-ait-7efb30b2-697548f498-zw6hh"] Jul 06 16:49:48.531789 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:49:48.531765 2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-ait-7efb30b2-697548f498-zw6hh"] Jul 06 16:49:50.118520 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:49:50.118486 2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ad25326c-554c-46e2-ac34-3d5a577c288a" path="/var/lib/kubelet/pods/ad25326c-554c-46e2-ac34-3d5a577c288a/volumes" Jul 06 16:49:53.512310 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:49:53.512277 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:49:53.724679 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:49:53.724644 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:49:53.871742 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:49:53.871642 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:49:54.977651 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:49:54.977605 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:49:55.051528 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:49:55.051498 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-ait-b496212d-68f95b5d5c-ncmrm"] Jul 06 16:49:55.051972 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:49:55.051946 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ad25326c-554c-46e2-ac34-3d5a577c288a" containerName="maas-api" Jul 06 16:49:55.051972 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:49:55.051974 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="ad25326c-554c-46e2-ac34-3d5a577c288a" containerName="maas-api" Jul 06 16:49:55.052077 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:49:55.052035 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="ad25326c-554c-46e2-ac34-3d5a577c288a" containerName="maas-api" Jul 06 16:49:55.055261 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:49:55.055238 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-b496212d-68f95b5d5c-ncmrm" Jul 06 16:49:55.058238 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:49:55.058220 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-ait-b496212d-serving-cert\"" Jul 06 16:49:55.066846 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:49:55.066824 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-ait-b496212d-68f95b5d5c-ncmrm"] Jul 06 16:49:55.070663 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:49:55.070636 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:49:55.144899 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:49:55.144855 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:49:55.203912 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:49:55.203879 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qphnj\" (UniqueName: \"kubernetes.io/projected/79ff8b24-4db1-4cf5-9772-3f32e096eef1-kube-api-access-qphnj\") pod \"maas-api-e2e-ait-b496212d-68f95b5d5c-ncmrm\" (UID: \"79ff8b24-4db1-4cf5-9772-3f32e096eef1\") " pod="opendatahub/maas-api-e2e-ait-b496212d-68f95b5d5c-ncmrm" Jul 06 16:49:55.204127 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:49:55.204066 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/79ff8b24-4db1-4cf5-9772-3f32e096eef1-maas-api-tls\") pod \"maas-api-e2e-ait-b496212d-68f95b5d5c-ncmrm\" (UID: \"79ff8b24-4db1-4cf5-9772-3f32e096eef1\") " pod="opendatahub/maas-api-e2e-ait-b496212d-68f95b5d5c-ncmrm" Jul 06 16:49:55.204127 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:49:55.204110 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/79ff8b24-4db1-4cf5-9772-3f32e096eef1-openshift-service-ca\") pod \"maas-api-e2e-ait-b496212d-68f95b5d5c-ncmrm\" (UID: \"79ff8b24-4db1-4cf5-9772-3f32e096eef1\") " pod="opendatahub/maas-api-e2e-ait-b496212d-68f95b5d5c-ncmrm" Jul 06 16:49:55.305360 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:49:55.305244 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/79ff8b24-4db1-4cf5-9772-3f32e096eef1-maas-api-tls\") pod \"maas-api-e2e-ait-b496212d-68f95b5d5c-ncmrm\" (UID: \"79ff8b24-4db1-4cf5-9772-3f32e096eef1\") " pod="opendatahub/maas-api-e2e-ait-b496212d-68f95b5d5c-ncmrm" Jul 06 16:49:55.305360 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:49:55.305297 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/79ff8b24-4db1-4cf5-9772-3f32e096eef1-openshift-service-ca\") pod \"maas-api-e2e-ait-b496212d-68f95b5d5c-ncmrm\" (UID: \"79ff8b24-4db1-4cf5-9772-3f32e096eef1\") " pod="opendatahub/maas-api-e2e-ait-b496212d-68f95b5d5c-ncmrm" Jul 06 16:49:55.305360 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:49:55.305346 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qphnj\" (UniqueName: \"kubernetes.io/projected/79ff8b24-4db1-4cf5-9772-3f32e096eef1-kube-api-access-qphnj\") pod \"maas-api-e2e-ait-b496212d-68f95b5d5c-ncmrm\" (UID: \"79ff8b24-4db1-4cf5-9772-3f32e096eef1\") " pod="opendatahub/maas-api-e2e-ait-b496212d-68f95b5d5c-ncmrm" Jul 06 16:49:55.305832 ip-10-0-128-22 kubenswrapper[2584]: E0706 16:49:55.305809 2584 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-ait-b496212d-serving-cert: secret "maas-api-e2e-ait-b496212d-serving-cert" not found Jul 06 16:49:55.305908 ip-10-0-128-22 kubenswrapper[2584]: E0706 16:49:55.305887 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/79ff8b24-4db1-4cf5-9772-3f32e096eef1-maas-api-tls podName:79ff8b24-4db1-4cf5-9772-3f32e096eef1 nodeName:}" failed. No retries permitted until 2026-07-06 16:49:55.805865812 +0000 UTC m=+1546.420005918 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/79ff8b24-4db1-4cf5-9772-3f32e096eef1-maas-api-tls") pod "maas-api-e2e-ait-b496212d-68f95b5d5c-ncmrm" (UID: "79ff8b24-4db1-4cf5-9772-3f32e096eef1") : secret "maas-api-e2e-ait-b496212d-serving-cert" not found Jul 06 16:49:55.306819 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:49:55.306792 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/79ff8b24-4db1-4cf5-9772-3f32e096eef1-openshift-service-ca\") pod \"maas-api-e2e-ait-b496212d-68f95b5d5c-ncmrm\" (UID: \"79ff8b24-4db1-4cf5-9772-3f32e096eef1\") " pod="opendatahub/maas-api-e2e-ait-b496212d-68f95b5d5c-ncmrm" Jul 06 16:49:55.318144 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:49:55.318117 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qphnj\" (UniqueName: \"kubernetes.io/projected/79ff8b24-4db1-4cf5-9772-3f32e096eef1-kube-api-access-qphnj\") pod \"maas-api-e2e-ait-b496212d-68f95b5d5c-ncmrm\" (UID: \"79ff8b24-4db1-4cf5-9772-3f32e096eef1\") " pod="opendatahub/maas-api-e2e-ait-b496212d-68f95b5d5c-ncmrm" Jul 06 16:49:55.360889 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:49:55.360851 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:49:55.569617 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:49:55.569529 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:49:55.670454 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:49:55.670415 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:49:55.810908 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:49:55.810868 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/79ff8b24-4db1-4cf5-9772-3f32e096eef1-maas-api-tls\") pod \"maas-api-e2e-ait-b496212d-68f95b5d5c-ncmrm\" (UID: \"79ff8b24-4db1-4cf5-9772-3f32e096eef1\") " pod="opendatahub/maas-api-e2e-ait-b496212d-68f95b5d5c-ncmrm" Jul 06 16:49:55.813342 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:49:55.813311 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/79ff8b24-4db1-4cf5-9772-3f32e096eef1-maas-api-tls\") pod \"maas-api-e2e-ait-b496212d-68f95b5d5c-ncmrm\" (UID: \"79ff8b24-4db1-4cf5-9772-3f32e096eef1\") " pod="opendatahub/maas-api-e2e-ait-b496212d-68f95b5d5c-ncmrm" Jul 06 16:49:55.968559 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:49:55.968520 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-b496212d-68f95b5d5c-ncmrm" Jul 06 16:49:56.107110 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:49:56.107080 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-ait-b496212d-68f95b5d5c-ncmrm"] Jul 06 16:49:56.109771 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:49:56.109741 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod79ff8b24_4db1_4cf5_9772_3f32e096eef1.slice/crio-4701530a1ace2341d3a55839eef4fec59c642c2850a9dba3539d5f4dbec3bcd1 WatchSource:0}: Error finding container 4701530a1ace2341d3a55839eef4fec59c642c2850a9dba3539d5f4dbec3bcd1: Status 404 returned error can't find the container with id 4701530a1ace2341d3a55839eef4fec59c642c2850a9dba3539d5f4dbec3bcd1 Jul 06 16:49:56.547925 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:49:56.547887 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-b496212d-68f95b5d5c-ncmrm" event={"ID":"79ff8b24-4db1-4cf5-9772-3f32e096eef1","Type":"ContainerStarted","Data":"921f961c3185a31127ccd73f95892cf7c9b879640f5c32039844632366c1578a"} Jul 06 16:49:56.548065 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:49:56.547944 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-b496212d-68f95b5d5c-ncmrm" event={"ID":"79ff8b24-4db1-4cf5-9772-3f32e096eef1","Type":"ContainerStarted","Data":"4701530a1ace2341d3a55839eef4fec59c642c2850a9dba3539d5f4dbec3bcd1"} Jul 06 16:49:56.548065 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:49:56.547975 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-ait-b496212d-68f95b5d5c-ncmrm" Jul 06 16:49:56.574684 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:49:56.574630 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-ait-b496212d-68f95b5d5c-ncmrm" podStartSLOduration=1.218590817 podStartE2EDuration="1.574615202s" totalImagesPullingTime="356.024385ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 16:49:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 16:49:56.572319892 +0000 UTC m=+1547.186460013" watchObservedRunningTime="2026-07-06 16:49:56.574615202 +0000 UTC m=+1547.188755322" Jul 06 16:49:57.421247 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:49:57.421213 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:50:00.036357 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:50:00.036324 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:50:00.382229 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:50:00.382135 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-ait-b496212d-68f95b5d5c-ncmrm"] Jul 06 16:50:00.382412 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:50:00.382383 2584 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-ait-b496212d-68f95b5d5c-ncmrm" podUID="79ff8b24-4db1-4cf5-9772-3f32e096eef1" containerName="maas-api" containerID="cri-o://921f961c3185a31127ccd73f95892cf7c9b879640f5c32039844632366c1578a" gracePeriod=30 Jul 06 16:50:00.419248 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:50:00.419213 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:50:00.569280 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:50:00.569245 2584 generic.go:356] "Generic (PLEG): container finished" podID="79ff8b24-4db1-4cf5-9772-3f32e096eef1" containerID="921f961c3185a31127ccd73f95892cf7c9b879640f5c32039844632366c1578a" exitCode=0 Jul 06 16:50:00.569464 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:50:00.569334 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-b496212d-68f95b5d5c-ncmrm" event={"ID":"79ff8b24-4db1-4cf5-9772-3f32e096eef1","Type":"ContainerDied","Data":"921f961c3185a31127ccd73f95892cf7c9b879640f5c32039844632366c1578a"} Jul 06 16:50:00.630388 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:50:00.630360 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-b496212d-68f95b5d5c-ncmrm" Jul 06 16:50:00.759439 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:50:00.759392 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qphnj\" (UniqueName: \"kubernetes.io/projected/79ff8b24-4db1-4cf5-9772-3f32e096eef1-kube-api-access-qphnj\") pod \"79ff8b24-4db1-4cf5-9772-3f32e096eef1\" (UID: \"79ff8b24-4db1-4cf5-9772-3f32e096eef1\") " Jul 06 16:50:00.759439 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:50:00.759443 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/79ff8b24-4db1-4cf5-9772-3f32e096eef1-maas-api-tls\") pod \"79ff8b24-4db1-4cf5-9772-3f32e096eef1\" (UID: \"79ff8b24-4db1-4cf5-9772-3f32e096eef1\") " Jul 06 16:50:00.759708 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:50:00.759491 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/79ff8b24-4db1-4cf5-9772-3f32e096eef1-openshift-service-ca\") pod \"79ff8b24-4db1-4cf5-9772-3f32e096eef1\" (UID: \"79ff8b24-4db1-4cf5-9772-3f32e096eef1\") " Jul 06 16:50:00.759933 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:50:00.759903 2584 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/79ff8b24-4db1-4cf5-9772-3f32e096eef1-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "79ff8b24-4db1-4cf5-9772-3f32e096eef1" (UID: "79ff8b24-4db1-4cf5-9772-3f32e096eef1"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 16:50:00.761487 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:50:00.761460 2584 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/79ff8b24-4db1-4cf5-9772-3f32e096eef1-kube-api-access-qphnj" (OuterVolumeSpecName: "kube-api-access-qphnj") pod "79ff8b24-4db1-4cf5-9772-3f32e096eef1" (UID: "79ff8b24-4db1-4cf5-9772-3f32e096eef1"). InnerVolumeSpecName "kube-api-access-qphnj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 16:50:00.761597 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:50:00.761583 2584 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/79ff8b24-4db1-4cf5-9772-3f32e096eef1-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "79ff8b24-4db1-4cf5-9772-3f32e096eef1" (UID: "79ff8b24-4db1-4cf5-9772-3f32e096eef1"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 16:50:00.861022 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:50:00.860989 2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qphnj\" (UniqueName: \"kubernetes.io/projected/79ff8b24-4db1-4cf5-9772-3f32e096eef1-kube-api-access-qphnj\") on node \"ip-10-0-128-22.ec2.internal\" DevicePath \"\"" Jul 06 16:50:00.861022 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:50:00.861019 2584 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/79ff8b24-4db1-4cf5-9772-3f32e096eef1-maas-api-tls\") on node \"ip-10-0-128-22.ec2.internal\" DevicePath \"\"" Jul 06 16:50:00.861022 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:50:00.861029 2584 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/79ff8b24-4db1-4cf5-9772-3f32e096eef1-openshift-service-ca\") on node \"ip-10-0-128-22.ec2.internal\" DevicePath \"\"" Jul 06 16:50:01.575146 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:50:01.575113 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-b496212d-68f95b5d5c-ncmrm" Jul 06 16:50:01.575146 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:50:01.575127 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-b496212d-68f95b5d5c-ncmrm" event={"ID":"79ff8b24-4db1-4cf5-9772-3f32e096eef1","Type":"ContainerDied","Data":"4701530a1ace2341d3a55839eef4fec59c642c2850a9dba3539d5f4dbec3bcd1"} Jul 06 16:50:01.575685 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:50:01.575194 2584 scope.go:117] "RemoveContainer" containerID="921f961c3185a31127ccd73f95892cf7c9b879640f5c32039844632366c1578a" Jul 06 16:50:01.602544 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:50:01.602503 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-ait-b496212d-68f95b5d5c-ncmrm"] Jul 06 16:50:01.608646 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:50:01.608617 2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-ait-b496212d-68f95b5d5c-ncmrm"] Jul 06 16:50:02.118700 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:50:02.118663 2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="79ff8b24-4db1-4cf5-9772-3f32e096eef1" path="/var/lib/kubelet/pods/79ff8b24-4db1-4cf5-9772-3f32e096eef1/volumes" Jul 06 16:50:06.344064 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:50:06.344034 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:50:06.445688 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:50:06.445651 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:50:08.155915 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:50:08.155882 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:50:12.663320 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:50:12.663278 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:50:12.741237 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:50:12.741206 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:50:12.900600 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:50:12.900562 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:50:12.991278 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:50:12.991183 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:50:14.109253 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:50:14.109221 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:50:14.204369 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:50:14.204333 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:50:14.282762 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:50:14.282713 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-derive-39de03b3-78586f9c45-v9gvp"] Jul 06 16:50:14.283213 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:50:14.283195 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="79ff8b24-4db1-4cf5-9772-3f32e096eef1" containerName="maas-api" Jul 06 16:50:14.283321 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:50:14.283216 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="79ff8b24-4db1-4cf5-9772-3f32e096eef1" containerName="maas-api" Jul 06 16:50:14.283321 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:50:14.283293 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="79ff8b24-4db1-4cf5-9772-3f32e096eef1" containerName="maas-api" Jul 06 16:50:14.286434 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:50:14.286413 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-derive-39de03b3-78586f9c45-v9gvp" Jul 06 16:50:14.289595 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:50:14.289574 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-derive-39de03b3-serving-cert\"" Jul 06 16:50:14.296388 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:50:14.296366 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/13fdffb9-2d0b-4ee9-afa5-3d5550c16a98-openshift-service-ca\") pod \"maas-api-e2e-derive-39de03b3-78586f9c45-v9gvp\" (UID: \"13fdffb9-2d0b-4ee9-afa5-3d5550c16a98\") " pod="opendatahub/maas-api-e2e-derive-39de03b3-78586f9c45-v9gvp" Jul 06 16:50:14.296477 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:50:14.296406 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4ptww\" (UniqueName: \"kubernetes.io/projected/13fdffb9-2d0b-4ee9-afa5-3d5550c16a98-kube-api-access-4ptww\") pod \"maas-api-e2e-derive-39de03b3-78586f9c45-v9gvp\" (UID: \"13fdffb9-2d0b-4ee9-afa5-3d5550c16a98\") " pod="opendatahub/maas-api-e2e-derive-39de03b3-78586f9c45-v9gvp" Jul 06 16:50:14.296477 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:50:14.296469 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/13fdffb9-2d0b-4ee9-afa5-3d5550c16a98-maas-api-tls\") pod \"maas-api-e2e-derive-39de03b3-78586f9c45-v9gvp\" (UID: \"13fdffb9-2d0b-4ee9-afa5-3d5550c16a98\") " pod="opendatahub/maas-api-e2e-derive-39de03b3-78586f9c45-v9gvp" Jul 06 16:50:14.300708 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:50:14.300675 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-derive-39de03b3-78586f9c45-v9gvp"] Jul 06 16:50:14.311028 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:50:14.311004 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:50:14.397556 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:50:14.397466 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/13fdffb9-2d0b-4ee9-afa5-3d5550c16a98-openshift-service-ca\") pod \"maas-api-e2e-derive-39de03b3-78586f9c45-v9gvp\" (UID: \"13fdffb9-2d0b-4ee9-afa5-3d5550c16a98\") " pod="opendatahub/maas-api-e2e-derive-39de03b3-78586f9c45-v9gvp" Jul 06 16:50:14.397556 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:50:14.397510 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4ptww\" (UniqueName: \"kubernetes.io/projected/13fdffb9-2d0b-4ee9-afa5-3d5550c16a98-kube-api-access-4ptww\") pod \"maas-api-e2e-derive-39de03b3-78586f9c45-v9gvp\" (UID: \"13fdffb9-2d0b-4ee9-afa5-3d5550c16a98\") " pod="opendatahub/maas-api-e2e-derive-39de03b3-78586f9c45-v9gvp" Jul 06 16:50:14.397556 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:50:14.397557 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/13fdffb9-2d0b-4ee9-afa5-3d5550c16a98-maas-api-tls\") pod \"maas-api-e2e-derive-39de03b3-78586f9c45-v9gvp\" (UID: \"13fdffb9-2d0b-4ee9-afa5-3d5550c16a98\") " pod="opendatahub/maas-api-e2e-derive-39de03b3-78586f9c45-v9gvp" Jul 06 16:50:14.398182 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:50:14.398163 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/13fdffb9-2d0b-4ee9-afa5-3d5550c16a98-openshift-service-ca\") pod \"maas-api-e2e-derive-39de03b3-78586f9c45-v9gvp\" (UID: \"13fdffb9-2d0b-4ee9-afa5-3d5550c16a98\") " pod="opendatahub/maas-api-e2e-derive-39de03b3-78586f9c45-v9gvp" Jul 06 16:50:14.399919 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:50:14.399903 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/13fdffb9-2d0b-4ee9-afa5-3d5550c16a98-maas-api-tls\") pod \"maas-api-e2e-derive-39de03b3-78586f9c45-v9gvp\" (UID: \"13fdffb9-2d0b-4ee9-afa5-3d5550c16a98\") " pod="opendatahub/maas-api-e2e-derive-39de03b3-78586f9c45-v9gvp" Jul 06 16:50:14.405909 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:50:14.405883 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:50:14.409751 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:50:14.409713 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4ptww\" (UniqueName: \"kubernetes.io/projected/13fdffb9-2d0b-4ee9-afa5-3d5550c16a98-kube-api-access-4ptww\") pod \"maas-api-e2e-derive-39de03b3-78586f9c45-v9gvp\" (UID: \"13fdffb9-2d0b-4ee9-afa5-3d5550c16a98\") " pod="opendatahub/maas-api-e2e-derive-39de03b3-78586f9c45-v9gvp" Jul 06 16:50:14.596504 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:50:14.596465 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-derive-39de03b3-78586f9c45-v9gvp" Jul 06 16:50:14.743129 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:50:14.743099 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-derive-39de03b3-78586f9c45-v9gvp"] Jul 06 16:50:14.745868 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:50:14.745831 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod13fdffb9_2d0b_4ee9_afa5_3d5550c16a98.slice/crio-3dd59fea15ef76c4b08ade1464f8b2f9a405c298c126b81dacdfa7a390f2dd18 WatchSource:0}: Error finding container 3dd59fea15ef76c4b08ade1464f8b2f9a405c298c126b81dacdfa7a390f2dd18: Status 404 returned error can't find the container with id 3dd59fea15ef76c4b08ade1464f8b2f9a405c298c126b81dacdfa7a390f2dd18 Jul 06 16:50:14.791378 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:50:14.791350 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:50:14.892287 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:50:14.892253 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:50:15.649153 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:50:15.649122 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-derive-39de03b3-78586f9c45-v9gvp" event={"ID":"13fdffb9-2d0b-4ee9-afa5-3d5550c16a98","Type":"ContainerStarted","Data":"9dca761197d1a60ab9b8869719ccd5d50617944933ec472df0e6119262af18cc"} Jul 06 16:50:15.649547 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:50:15.649176 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-derive-39de03b3-78586f9c45-v9gvp" event={"ID":"13fdffb9-2d0b-4ee9-afa5-3d5550c16a98","Type":"ContainerStarted","Data":"3dd59fea15ef76c4b08ade1464f8b2f9a405c298c126b81dacdfa7a390f2dd18"} Jul 06 16:50:15.649547 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:50:15.649197 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-derive-39de03b3-78586f9c45-v9gvp" Jul 06 16:50:15.669814 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:50:15.669754 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-derive-39de03b3-78586f9c45-v9gvp" podStartSLOduration=1.338861794 podStartE2EDuration="1.669714931s" totalImagesPullingTime="330.853137ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 16:50:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 16:50:15.668568442 +0000 UTC m=+1566.282708565" watchObservedRunningTime="2026-07-06 16:50:15.669714931 +0000 UTC m=+1566.283855053" Jul 06 16:50:16.304291 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:50:16.304260 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:50:18.578944 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:50:18.578891 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:50:18.779446 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:50:18.779404 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:50:18.882610 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:50:18.882521 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-derive-39de03b3-78586f9c45-v9gvp"] Jul 06 16:50:18.882903 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:50:18.882865 2584 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-derive-39de03b3-78586f9c45-v9gvp" podUID="13fdffb9-2d0b-4ee9-afa5-3d5550c16a98" containerName="maas-api" containerID="cri-o://9dca761197d1a60ab9b8869719ccd5d50617944933ec472df0e6119262af18cc" gracePeriod=30 Jul 06 16:50:18.885745 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:50:18.885694 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:50:18.987945 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:50:18.987907 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:50:19.134976 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:50:19.134912 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-derive-39de03b3-78586f9c45-v9gvp" Jul 06 16:50:19.242542 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:50:19.242502 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/13fdffb9-2d0b-4ee9-afa5-3d5550c16a98-openshift-service-ca\") pod \"13fdffb9-2d0b-4ee9-afa5-3d5550c16a98\" (UID: \"13fdffb9-2d0b-4ee9-afa5-3d5550c16a98\") " Jul 06 16:50:19.242542 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:50:19.242541 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/13fdffb9-2d0b-4ee9-afa5-3d5550c16a98-maas-api-tls\") pod \"13fdffb9-2d0b-4ee9-afa5-3d5550c16a98\" (UID: \"13fdffb9-2d0b-4ee9-afa5-3d5550c16a98\") " Jul 06 16:50:19.242777 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:50:19.242585 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4ptww\" (UniqueName: \"kubernetes.io/projected/13fdffb9-2d0b-4ee9-afa5-3d5550c16a98-kube-api-access-4ptww\") pod \"13fdffb9-2d0b-4ee9-afa5-3d5550c16a98\" (UID: \"13fdffb9-2d0b-4ee9-afa5-3d5550c16a98\") " Jul 06 16:50:19.242932 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:50:19.242901 2584 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/13fdffb9-2d0b-4ee9-afa5-3d5550c16a98-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "13fdffb9-2d0b-4ee9-afa5-3d5550c16a98" (UID: "13fdffb9-2d0b-4ee9-afa5-3d5550c16a98"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 16:50:19.244786 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:50:19.244755 2584 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/13fdffb9-2d0b-4ee9-afa5-3d5550c16a98-kube-api-access-4ptww" (OuterVolumeSpecName: "kube-api-access-4ptww") pod "13fdffb9-2d0b-4ee9-afa5-3d5550c16a98" (UID: "13fdffb9-2d0b-4ee9-afa5-3d5550c16a98"). InnerVolumeSpecName "kube-api-access-4ptww". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 16:50:19.244903 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:50:19.244760 2584 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/13fdffb9-2d0b-4ee9-afa5-3d5550c16a98-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "13fdffb9-2d0b-4ee9-afa5-3d5550c16a98" (UID: "13fdffb9-2d0b-4ee9-afa5-3d5550c16a98"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 16:50:19.344125 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:50:19.344084 2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4ptww\" (UniqueName: \"kubernetes.io/projected/13fdffb9-2d0b-4ee9-afa5-3d5550c16a98-kube-api-access-4ptww\") on node \"ip-10-0-128-22.ec2.internal\" DevicePath \"\"" Jul 06 16:50:19.344125 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:50:19.344118 2584 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/13fdffb9-2d0b-4ee9-afa5-3d5550c16a98-openshift-service-ca\") on node \"ip-10-0-128-22.ec2.internal\" DevicePath \"\"" Jul 06 16:50:19.344125 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:50:19.344131 2584 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/13fdffb9-2d0b-4ee9-afa5-3d5550c16a98-maas-api-tls\") on node \"ip-10-0-128-22.ec2.internal\" DevicePath \"\"" Jul 06 16:50:19.670408 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:50:19.670371 2584 generic.go:356] "Generic (PLEG): container finished" podID="13fdffb9-2d0b-4ee9-afa5-3d5550c16a98" containerID="9dca761197d1a60ab9b8869719ccd5d50617944933ec472df0e6119262af18cc" exitCode=0 Jul 06 16:50:19.670949 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:50:19.670421 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-derive-39de03b3-78586f9c45-v9gvp" Jul 06 16:50:19.670949 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:50:19.670464 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-derive-39de03b3-78586f9c45-v9gvp" event={"ID":"13fdffb9-2d0b-4ee9-afa5-3d5550c16a98","Type":"ContainerDied","Data":"9dca761197d1a60ab9b8869719ccd5d50617944933ec472df0e6119262af18cc"} Jul 06 16:50:19.670949 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:50:19.670526 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-derive-39de03b3-78586f9c45-v9gvp" event={"ID":"13fdffb9-2d0b-4ee9-afa5-3d5550c16a98","Type":"ContainerDied","Data":"3dd59fea15ef76c4b08ade1464f8b2f9a405c298c126b81dacdfa7a390f2dd18"} Jul 06 16:50:19.670949 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:50:19.670555 2584 scope.go:117] "RemoveContainer" containerID="9dca761197d1a60ab9b8869719ccd5d50617944933ec472df0e6119262af18cc" Jul 06 16:50:19.680960 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:50:19.680942 2584 scope.go:117] "RemoveContainer" containerID="9dca761197d1a60ab9b8869719ccd5d50617944933ec472df0e6119262af18cc" Jul 06 16:50:19.681256 ip-10-0-128-22 kubenswrapper[2584]: E0706 16:50:19.681231 2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9dca761197d1a60ab9b8869719ccd5d50617944933ec472df0e6119262af18cc\": container with ID starting with 9dca761197d1a60ab9b8869719ccd5d50617944933ec472df0e6119262af18cc not found: ID does not exist" containerID="9dca761197d1a60ab9b8869719ccd5d50617944933ec472df0e6119262af18cc" Jul 06 16:50:19.681311 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:50:19.681260 2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9dca761197d1a60ab9b8869719ccd5d50617944933ec472df0e6119262af18cc"} err="failed to get container status \"9dca761197d1a60ab9b8869719ccd5d50617944933ec472df0e6119262af18cc\": rpc error: code = NotFound desc = could not find container \"9dca761197d1a60ab9b8869719ccd5d50617944933ec472df0e6119262af18cc\": container with ID starting with 9dca761197d1a60ab9b8869719ccd5d50617944933ec472df0e6119262af18cc not found: ID does not exist" Jul 06 16:50:19.694001 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:50:19.693966 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-derive-39de03b3-78586f9c45-v9gvp"] Jul 06 16:50:19.702783 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:50:19.702759 2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-derive-39de03b3-78586f9c45-v9gvp"] Jul 06 16:50:20.088322 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:50:20.088286 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:50:20.118993 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:50:20.118956 2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="13fdffb9-2d0b-4ee9-afa5-3d5550c16a98" path="/var/lib/kubelet/pods/13fdffb9-2d0b-4ee9-afa5-3d5550c16a98/volumes" Jul 06 16:50:25.498315 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:50:25.498282 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:50:25.565054 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:50:25.565019 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:50:25.953656 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:50:25.953617 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:50:26.053085 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:50:26.053044 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:50:26.156273 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:50:26.156241 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:50:27.587843 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:50:27.587811 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:50:28.900169 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:50:28.900131 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:50:33.798388 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:50:33.798345 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:50:33.898402 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:50:33.898370 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:50:33.997876 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:50:33.997841 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:50:34.101349 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:50:34.101254 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:50:34.513430 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:50:34.513398 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:50:40.923502 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:50:40.923471 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:50:41.002863 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:50:41.002825 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:51:00.483190 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:51:00.483150 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:51:06.563560 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:51:06.563477 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:51:06.676280 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:51:06.676239 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:51:06.784685 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:51:06.784640 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:51:22.670879 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:51:22.670833 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:51:28.856449 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:51:28.856406 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:51:28.921518 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:51:28.921484 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:51:29.010499 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:51:29.010465 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:51:29.110751 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:51:29.110647 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:51:39.554817 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:51:39.554782 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:51:40.360470 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:51:40.360432 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:51:46.641790 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:51:46.641748 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:51:46.743550 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:51:46.743517 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:51:46.842282 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:51:46.842248 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:51:46.946319 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:51:46.946287 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:51:47.733472 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:51:47.733435 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:51:47.886740 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:51:47.886683 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:51:48.014796 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:51:48.014685 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:51:49.604241 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:51:49.604203 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:51:56.144045 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:51:56.144009 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:51:56.238697 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:51:56.238658 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:51:56.924236 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:51:56.924208 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:52:03.400642 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:52:03.400607 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:52:03.465641 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:52:03.465598 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:52:03.657943 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:52:03.657838 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:52:04.037674 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:52:04.037641 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:52:04.351255 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:52:04.351170 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:52:07.027612 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:52:07.027576 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:52:07.096337 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:52:07.096304 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-mt-1e5ead78-5f8c64dcc8-t7xwf"] Jul 06 16:52:07.096700 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:52:07.096688 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="13fdffb9-2d0b-4ee9-afa5-3d5550c16a98" containerName="maas-api" Jul 06 16:52:07.096785 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:52:07.096709 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="13fdffb9-2d0b-4ee9-afa5-3d5550c16a98" containerName="maas-api" Jul 06 16:52:07.096824 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:52:07.096803 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="13fdffb9-2d0b-4ee9-afa5-3d5550c16a98" containerName="maas-api" Jul 06 16:52:07.100210 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:52:07.100192 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-1e5ead78-5f8c64dcc8-t7xwf" Jul 06 16:52:07.102845 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:52:07.102822 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-mt-1e5ead78-serving-cert\"" Jul 06 16:52:07.110639 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:52:07.110613 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-mt-1e5ead78-5f8c64dcc8-t7xwf"] Jul 06 16:52:07.115324 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:52:07.115289 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:52:07.188545 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:52:07.188422 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:52:07.253585 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:52:07.253547 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qlttd\" (UniqueName: \"kubernetes.io/projected/b13b36d6-1edb-4ab6-ae88-d0984c80b3ad-kube-api-access-qlttd\") pod \"maas-api-e2e-mt-1e5ead78-5f8c64dcc8-t7xwf\" (UID: \"b13b36d6-1edb-4ab6-ae88-d0984c80b3ad\") " pod="opendatahub/maas-api-e2e-mt-1e5ead78-5f8c64dcc8-t7xwf" Jul 06 16:52:07.253787 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:52:07.253776 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b13b36d6-1edb-4ab6-ae88-d0984c80b3ad-maas-api-tls\") pod \"maas-api-e2e-mt-1e5ead78-5f8c64dcc8-t7xwf\" (UID: \"b13b36d6-1edb-4ab6-ae88-d0984c80b3ad\") " pod="opendatahub/maas-api-e2e-mt-1e5ead78-5f8c64dcc8-t7xwf" Jul 06 16:52:07.253839 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:52:07.253807 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b13b36d6-1edb-4ab6-ae88-d0984c80b3ad-openshift-service-ca\") pod \"maas-api-e2e-mt-1e5ead78-5f8c64dcc8-t7xwf\" (UID: \"b13b36d6-1edb-4ab6-ae88-d0984c80b3ad\") " pod="opendatahub/maas-api-e2e-mt-1e5ead78-5f8c64dcc8-t7xwf" Jul 06 16:52:07.289045 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:52:07.288959 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:52:07.354696 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:52:07.354655 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qlttd\" (UniqueName: \"kubernetes.io/projected/b13b36d6-1edb-4ab6-ae88-d0984c80b3ad-kube-api-access-qlttd\") pod \"maas-api-e2e-mt-1e5ead78-5f8c64dcc8-t7xwf\" (UID: \"b13b36d6-1edb-4ab6-ae88-d0984c80b3ad\") " pod="opendatahub/maas-api-e2e-mt-1e5ead78-5f8c64dcc8-t7xwf" Jul 06 16:52:07.354917 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:52:07.354718 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b13b36d6-1edb-4ab6-ae88-d0984c80b3ad-maas-api-tls\") pod \"maas-api-e2e-mt-1e5ead78-5f8c64dcc8-t7xwf\" (UID: \"b13b36d6-1edb-4ab6-ae88-d0984c80b3ad\") " pod="opendatahub/maas-api-e2e-mt-1e5ead78-5f8c64dcc8-t7xwf" Jul 06 16:52:07.354917 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:52:07.354788 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b13b36d6-1edb-4ab6-ae88-d0984c80b3ad-openshift-service-ca\") pod \"maas-api-e2e-mt-1e5ead78-5f8c64dcc8-t7xwf\" (UID: \"b13b36d6-1edb-4ab6-ae88-d0984c80b3ad\") " pod="opendatahub/maas-api-e2e-mt-1e5ead78-5f8c64dcc8-t7xwf" Jul 06 16:52:07.355519 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:52:07.355493 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b13b36d6-1edb-4ab6-ae88-d0984c80b3ad-openshift-service-ca\") pod \"maas-api-e2e-mt-1e5ead78-5f8c64dcc8-t7xwf\" (UID: \"b13b36d6-1edb-4ab6-ae88-d0984c80b3ad\") " pod="opendatahub/maas-api-e2e-mt-1e5ead78-5f8c64dcc8-t7xwf" Jul 06 16:52:07.357200 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:52:07.357182 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b13b36d6-1edb-4ab6-ae88-d0984c80b3ad-maas-api-tls\") pod \"maas-api-e2e-mt-1e5ead78-5f8c64dcc8-t7xwf\" (UID: \"b13b36d6-1edb-4ab6-ae88-d0984c80b3ad\") " pod="opendatahub/maas-api-e2e-mt-1e5ead78-5f8c64dcc8-t7xwf" Jul 06 16:52:07.365583 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:52:07.365559 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qlttd\" (UniqueName: \"kubernetes.io/projected/b13b36d6-1edb-4ab6-ae88-d0984c80b3ad-kube-api-access-qlttd\") pod \"maas-api-e2e-mt-1e5ead78-5f8c64dcc8-t7xwf\" (UID: \"b13b36d6-1edb-4ab6-ae88-d0984c80b3ad\") " pod="opendatahub/maas-api-e2e-mt-1e5ead78-5f8c64dcc8-t7xwf" Jul 06 16:52:07.411808 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:52:07.411778 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-1e5ead78-5f8c64dcc8-t7xwf" Jul 06 16:52:07.567143 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:52:07.567116 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-mt-1e5ead78-5f8c64dcc8-t7xwf"] Jul 06 16:52:07.569899 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:52:07.569861 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb13b36d6_1edb_4ab6_ae88_d0984c80b3ad.slice/crio-03082d66b971f47e39adf0b1b301a1011600b6294966437743e78d653492baf4 WatchSource:0}: Error finding container 03082d66b971f47e39adf0b1b301a1011600b6294966437743e78d653492baf4: Status 404 returned error can't find the container with id 03082d66b971f47e39adf0b1b301a1011600b6294966437743e78d653492baf4 Jul 06 16:52:07.732979 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:52:07.732945 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:52:07.832839 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:52:07.832749 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:52:08.192975 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:52:08.192935 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-1e5ead78-5f8c64dcc8-t7xwf" event={"ID":"b13b36d6-1edb-4ab6-ae88-d0984c80b3ad","Type":"ContainerStarted","Data":"fe286545a851037acd9f387ce854fa9e0f73a647582aba3d6da7a7de04ff6f50"} Jul 06 16:52:08.193467 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:52:08.192995 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-1e5ead78-5f8c64dcc8-t7xwf" event={"ID":"b13b36d6-1edb-4ab6-ae88-d0984c80b3ad","Type":"ContainerStarted","Data":"03082d66b971f47e39adf0b1b301a1011600b6294966437743e78d653492baf4"} Jul 06 16:52:08.193467 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:52:08.193101 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-mt-1e5ead78-5f8c64dcc8-t7xwf" Jul 06 16:52:08.214124 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:52:08.214068 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-mt-1e5ead78-5f8c64dcc8-t7xwf" podStartSLOduration=0.814236427 podStartE2EDuration="1.214048829s" totalImagesPullingTime="399.812402ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 16:52:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 16:52:08.212540653 +0000 UTC m=+1678.826680811" watchObservedRunningTime="2026-07-06 16:52:08.214048829 +0000 UTC m=+1678.828188949" Jul 06 16:52:11.039559 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:52:11.039519 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:52:11.557379 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:52:11.557348 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:52:12.270767 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:52:12.270717 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:52:12.366140 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:52:12.366102 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:52:12.757865 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:52:12.757834 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:52:14.203501 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:52:14.203468 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-mt-1e5ead78-5f8c64dcc8-t7xwf" Jul 06 16:52:19.111361 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:52:19.111314 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:52:19.175910 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:52:19.175869 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:52:19.562777 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:52:19.562737 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:52:19.828400 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:52:19.828305 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-mt-1e5ead78-5f8c64dcc8-t7xwf"] Jul 06 16:52:19.828582 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:52:19.828562 2584 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-mt-1e5ead78-5f8c64dcc8-t7xwf" podUID="b13b36d6-1edb-4ab6-ae88-d0984c80b3ad" containerName="maas-api" containerID="cri-o://fe286545a851037acd9f387ce854fa9e0f73a647582aba3d6da7a7de04ff6f50" gracePeriod=30 Jul 06 16:52:19.886530 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:52:19.886491 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:52:20.019279 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:52:20.019239 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:52:20.089390 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:52:20.089321 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-1e5ead78-5f8c64dcc8-t7xwf" Jul 06 16:52:20.185275 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:52:20.185239 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b13b36d6-1edb-4ab6-ae88-d0984c80b3ad-openshift-service-ca\") pod \"b13b36d6-1edb-4ab6-ae88-d0984c80b3ad\" (UID: \"b13b36d6-1edb-4ab6-ae88-d0984c80b3ad\") " Jul 06 16:52:20.185693 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:52:20.185291 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qlttd\" (UniqueName: \"kubernetes.io/projected/b13b36d6-1edb-4ab6-ae88-d0984c80b3ad-kube-api-access-qlttd\") pod \"b13b36d6-1edb-4ab6-ae88-d0984c80b3ad\" (UID: \"b13b36d6-1edb-4ab6-ae88-d0984c80b3ad\") " Jul 06 16:52:20.185693 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:52:20.185317 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b13b36d6-1edb-4ab6-ae88-d0984c80b3ad-maas-api-tls\") pod \"b13b36d6-1edb-4ab6-ae88-d0984c80b3ad\" (UID: \"b13b36d6-1edb-4ab6-ae88-d0984c80b3ad\") " Jul 06 16:52:20.185693 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:52:20.185643 2584 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b13b36d6-1edb-4ab6-ae88-d0984c80b3ad-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "b13b36d6-1edb-4ab6-ae88-d0984c80b3ad" (UID: "b13b36d6-1edb-4ab6-ae88-d0984c80b3ad"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 16:52:20.187473 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:52:20.187445 2584 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b13b36d6-1edb-4ab6-ae88-d0984c80b3ad-kube-api-access-qlttd" (OuterVolumeSpecName: "kube-api-access-qlttd") pod "b13b36d6-1edb-4ab6-ae88-d0984c80b3ad" (UID: "b13b36d6-1edb-4ab6-ae88-d0984c80b3ad"). InnerVolumeSpecName "kube-api-access-qlttd". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 16:52:20.187584 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:52:20.187511 2584 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b13b36d6-1edb-4ab6-ae88-d0984c80b3ad-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "b13b36d6-1edb-4ab6-ae88-d0984c80b3ad" (UID: "b13b36d6-1edb-4ab6-ae88-d0984c80b3ad"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 16:52:20.255309 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:52:20.255272 2584 generic.go:356] "Generic (PLEG): container finished" podID="b13b36d6-1edb-4ab6-ae88-d0984c80b3ad" containerID="fe286545a851037acd9f387ce854fa9e0f73a647582aba3d6da7a7de04ff6f50" exitCode=0 Jul 06 16:52:20.255309 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:52:20.255322 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-1e5ead78-5f8c64dcc8-t7xwf" Jul 06 16:52:20.255564 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:52:20.255375 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-1e5ead78-5f8c64dcc8-t7xwf" event={"ID":"b13b36d6-1edb-4ab6-ae88-d0984c80b3ad","Type":"ContainerDied","Data":"fe286545a851037acd9f387ce854fa9e0f73a647582aba3d6da7a7de04ff6f50"} Jul 06 16:52:20.255564 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:52:20.255436 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-1e5ead78-5f8c64dcc8-t7xwf" event={"ID":"b13b36d6-1edb-4ab6-ae88-d0984c80b3ad","Type":"ContainerDied","Data":"03082d66b971f47e39adf0b1b301a1011600b6294966437743e78d653492baf4"} Jul 06 16:52:20.255564 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:52:20.255466 2584 scope.go:117] "RemoveContainer" containerID="fe286545a851037acd9f387ce854fa9e0f73a647582aba3d6da7a7de04ff6f50" Jul 06 16:52:20.266121 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:52:20.266094 2584 scope.go:117] "RemoveContainer" containerID="fe286545a851037acd9f387ce854fa9e0f73a647582aba3d6da7a7de04ff6f50" Jul 06 16:52:20.266517 ip-10-0-128-22 kubenswrapper[2584]: E0706 16:52:20.266492 2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fe286545a851037acd9f387ce854fa9e0f73a647582aba3d6da7a7de04ff6f50\": container with ID starting with fe286545a851037acd9f387ce854fa9e0f73a647582aba3d6da7a7de04ff6f50 not found: ID does not exist" containerID="fe286545a851037acd9f387ce854fa9e0f73a647582aba3d6da7a7de04ff6f50" Jul 06 16:52:20.266624 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:52:20.266526 2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fe286545a851037acd9f387ce854fa9e0f73a647582aba3d6da7a7de04ff6f50"} err="failed to get container status \"fe286545a851037acd9f387ce854fa9e0f73a647582aba3d6da7a7de04ff6f50\": rpc error: code = NotFound desc = could not find container \"fe286545a851037acd9f387ce854fa9e0f73a647582aba3d6da7a7de04ff6f50\": container with ID starting with fe286545a851037acd9f387ce854fa9e0f73a647582aba3d6da7a7de04ff6f50 not found: ID does not exist" Jul 06 16:52:20.281448 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:52:20.281415 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-mt-1e5ead78-5f8c64dcc8-t7xwf"] Jul 06 16:52:20.286089 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:52:20.286057 2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qlttd\" (UniqueName: \"kubernetes.io/projected/b13b36d6-1edb-4ab6-ae88-d0984c80b3ad-kube-api-access-qlttd\") on node \"ip-10-0-128-22.ec2.internal\" DevicePath \"\"" Jul 06 16:52:20.286089 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:52:20.286087 2584 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b13b36d6-1edb-4ab6-ae88-d0984c80b3ad-maas-api-tls\") on node \"ip-10-0-128-22.ec2.internal\" DevicePath \"\"" Jul 06 16:52:20.286256 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:52:20.286104 2584 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b13b36d6-1edb-4ab6-ae88-d0984c80b3ad-openshift-service-ca\") on node \"ip-10-0-128-22.ec2.internal\" DevicePath \"\"" Jul 06 16:52:20.286762 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:52:20.286737 2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-mt-1e5ead78-5f8c64dcc8-t7xwf"] Jul 06 16:52:22.119013 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:52:22.118978 2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b13b36d6-1edb-4ab6-ae88-d0984c80b3ad" path="/var/lib/kubelet/pods/b13b36d6-1edb-4ab6-ae88-d0984c80b3ad/volumes" Jul 06 16:52:26.178498 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:52:26.178463 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:52:26.270704 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:52:26.270669 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:52:26.456528 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:52:26.456421 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:52:26.680874 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:52:26.680829 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:52:26.803453 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:52:26.803360 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:52:26.907832 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:52:26.907799 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:52:27.056925 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:52:27.056840 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:52:30.878157 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:52:30.878116 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:52:30.983545 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:52:30.983513 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:52:33.618367 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:52:33.618279 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:52:33.687240 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:52:33.687208 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:52:33.801040 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:52:33.801003 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:52:33.908099 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:52:33.908018 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:52:34.012345 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:52:34.012308 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:52:40.297958 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:52:40.297928 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:52:40.399633 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:52:40.399602 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:52:47.162887 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:52:47.162854 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:52:49.204381 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:52:49.204346 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:52:50.131376 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:52:50.131339 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:52:50.209577 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:52:50.209544 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-mt-8f668f5c-74b79577dd-fvf4j"] Jul 06 16:52:50.209989 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:52:50.209974 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b13b36d6-1edb-4ab6-ae88-d0984c80b3ad" containerName="maas-api" Jul 06 16:52:50.210032 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:52:50.209990 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="b13b36d6-1edb-4ab6-ae88-d0984c80b3ad" containerName="maas-api" Jul 06 16:52:50.210066 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:52:50.210046 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="b13b36d6-1edb-4ab6-ae88-d0984c80b3ad" containerName="maas-api" Jul 06 16:52:50.212957 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:52:50.212941 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-8f668f5c-74b79577dd-fvf4j" Jul 06 16:52:50.217032 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:52:50.217009 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-mt-8f668f5c-serving-cert\"" Jul 06 16:52:50.223851 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:52:50.223829 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-mt-8f668f5c-74b79577dd-fvf4j"] Jul 06 16:52:50.227442 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:52:50.227399 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:52:50.302401 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:52:50.302369 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:52:50.366081 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:52:50.366044 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/650103cb-ceae-4160-b043-d66f063f0890-maas-api-tls\") pod \"maas-api-e2e-mt-8f668f5c-74b79577dd-fvf4j\" (UID: \"650103cb-ceae-4160-b043-d66f063f0890\") " pod="opendatahub/maas-api-e2e-mt-8f668f5c-74b79577dd-fvf4j" Jul 06 16:52:50.366256 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:52:50.366112 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/650103cb-ceae-4160-b043-d66f063f0890-openshift-service-ca\") pod \"maas-api-e2e-mt-8f668f5c-74b79577dd-fvf4j\" (UID: \"650103cb-ceae-4160-b043-d66f063f0890\") " pod="opendatahub/maas-api-e2e-mt-8f668f5c-74b79577dd-fvf4j" Jul 06 16:52:50.366256 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:52:50.366183 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hkxt2\" (UniqueName: \"kubernetes.io/projected/650103cb-ceae-4160-b043-d66f063f0890-kube-api-access-hkxt2\") pod \"maas-api-e2e-mt-8f668f5c-74b79577dd-fvf4j\" (UID: \"650103cb-ceae-4160-b043-d66f063f0890\") " pod="opendatahub/maas-api-e2e-mt-8f668f5c-74b79577dd-fvf4j" Jul 06 16:52:50.466853 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:52:50.466820 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hkxt2\" (UniqueName: \"kubernetes.io/projected/650103cb-ceae-4160-b043-d66f063f0890-kube-api-access-hkxt2\") pod \"maas-api-e2e-mt-8f668f5c-74b79577dd-fvf4j\" (UID: \"650103cb-ceae-4160-b043-d66f063f0890\") " pod="opendatahub/maas-api-e2e-mt-8f668f5c-74b79577dd-fvf4j" Jul 06 16:52:50.467045 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:52:50.466879 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/650103cb-ceae-4160-b043-d66f063f0890-maas-api-tls\") pod \"maas-api-e2e-mt-8f668f5c-74b79577dd-fvf4j\" (UID: \"650103cb-ceae-4160-b043-d66f063f0890\") " pod="opendatahub/maas-api-e2e-mt-8f668f5c-74b79577dd-fvf4j" Jul 06 16:52:50.467045 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:52:50.466922 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/650103cb-ceae-4160-b043-d66f063f0890-openshift-service-ca\") pod \"maas-api-e2e-mt-8f668f5c-74b79577dd-fvf4j\" (UID: \"650103cb-ceae-4160-b043-d66f063f0890\") " pod="opendatahub/maas-api-e2e-mt-8f668f5c-74b79577dd-fvf4j" Jul 06 16:52:50.467045 ip-10-0-128-22 kubenswrapper[2584]: E0706 16:52:50.466996 2584 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-mt-8f668f5c-serving-cert: secret "maas-api-e2e-mt-8f668f5c-serving-cert" not found Jul 06 16:52:50.467150 ip-10-0-128-22 kubenswrapper[2584]: E0706 16:52:50.467068 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/650103cb-ceae-4160-b043-d66f063f0890-maas-api-tls podName:650103cb-ceae-4160-b043-d66f063f0890 nodeName:}" failed. No retries permitted until 2026-07-06 16:52:50.967049856 +0000 UTC m=+1721.581189954 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/650103cb-ceae-4160-b043-d66f063f0890-maas-api-tls") pod "maas-api-e2e-mt-8f668f5c-74b79577dd-fvf4j" (UID: "650103cb-ceae-4160-b043-d66f063f0890") : secret "maas-api-e2e-mt-8f668f5c-serving-cert" not found Jul 06 16:52:50.467504 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:52:50.467487 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/650103cb-ceae-4160-b043-d66f063f0890-openshift-service-ca\") pod \"maas-api-e2e-mt-8f668f5c-74b79577dd-fvf4j\" (UID: \"650103cb-ceae-4160-b043-d66f063f0890\") " pod="opendatahub/maas-api-e2e-mt-8f668f5c-74b79577dd-fvf4j" Jul 06 16:52:50.477223 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:52:50.477195 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hkxt2\" (UniqueName: \"kubernetes.io/projected/650103cb-ceae-4160-b043-d66f063f0890-kube-api-access-hkxt2\") pod \"maas-api-e2e-mt-8f668f5c-74b79577dd-fvf4j\" (UID: \"650103cb-ceae-4160-b043-d66f063f0890\") " pod="opendatahub/maas-api-e2e-mt-8f668f5c-74b79577dd-fvf4j" Jul 06 16:52:50.671165 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:52:50.671125 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:52:50.774380 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:52:50.774286 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:52:50.871320 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:52:50.871286 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:52:50.971435 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:52:50.971392 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/650103cb-ceae-4160-b043-d66f063f0890-maas-api-tls\") pod \"maas-api-e2e-mt-8f668f5c-74b79577dd-fvf4j\" (UID: \"650103cb-ceae-4160-b043-d66f063f0890\") " pod="opendatahub/maas-api-e2e-mt-8f668f5c-74b79577dd-fvf4j" Jul 06 16:52:50.973850 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:52:50.973830 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/650103cb-ceae-4160-b043-d66f063f0890-maas-api-tls\") pod \"maas-api-e2e-mt-8f668f5c-74b79577dd-fvf4j\" (UID: \"650103cb-ceae-4160-b043-d66f063f0890\") " pod="opendatahub/maas-api-e2e-mt-8f668f5c-74b79577dd-fvf4j" Jul 06 16:52:51.124193 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:52:51.124089 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-8f668f5c-74b79577dd-fvf4j" Jul 06 16:52:51.269008 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:52:51.268965 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-mt-8f668f5c-74b79577dd-fvf4j"] Jul 06 16:52:51.271400 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:52:51.271371 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod650103cb_ceae_4160_b043_d66f063f0890.slice/crio-d83b3716c2280c99c4db209f2d960bc37321eae4a67bcafe9b6d5c3ede2e2383 WatchSource:0}: Error finding container d83b3716c2280c99c4db209f2d960bc37321eae4a67bcafe9b6d5c3ede2e2383: Status 404 returned error can't find the container with id d83b3716c2280c99c4db209f2d960bc37321eae4a67bcafe9b6d5c3ede2e2383 Jul 06 16:52:51.411629 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:52:51.411591 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-8f668f5c-74b79577dd-fvf4j" event={"ID":"650103cb-ceae-4160-b043-d66f063f0890","Type":"ContainerStarted","Data":"d83b3716c2280c99c4db209f2d960bc37321eae4a67bcafe9b6d5c3ede2e2383"} Jul 06 16:52:52.417807 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:52:52.417767 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-8f668f5c-74b79577dd-fvf4j" event={"ID":"650103cb-ceae-4160-b043-d66f063f0890","Type":"ContainerStarted","Data":"75bfdcee0b44f789ca215353db432440e5fe79d243dac3fad1d78a50ba435242"} Jul 06 16:52:52.418212 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:52:52.417920 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-mt-8f668f5c-74b79577dd-fvf4j" Jul 06 16:52:52.439126 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:52:52.439073 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-mt-8f668f5c-74b79577dd-fvf4j" podStartSLOduration=1.9861817959999999 podStartE2EDuration="2.439055726s" totalImagesPullingTime="452.87393ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 16:52:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 16:52:52.437122543 +0000 UTC m=+1723.051262703" watchObservedRunningTime="2026-07-06 16:52:52.439055726 +0000 UTC m=+1723.053195847" Jul 06 16:52:54.671483 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:52:54.671445 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:52:58.428556 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:52:58.428526 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-mt-8f668f5c-74b79577dd-fvf4j" Jul 06 16:53:14.046138 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:53:14.046103 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:53:47.578037 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:53:47.578003 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:53:47.886752 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:53:47.886648 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-mt-8f668f5c-74b79577dd-fvf4j"] Jul 06 16:53:47.886929 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:53:47.886904 2584 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-mt-8f668f5c-74b79577dd-fvf4j" podUID="650103cb-ceae-4160-b043-d66f063f0890" containerName="maas-api" containerID="cri-o://75bfdcee0b44f789ca215353db432440e5fe79d243dac3fad1d78a50ba435242" gracePeriod=30 Jul 06 16:53:47.916564 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:53:47.916531 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:53:48.138997 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:53:48.138924 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-8f668f5c-74b79577dd-fvf4j" Jul 06 16:53:48.235491 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:53:48.235450 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hkxt2\" (UniqueName: \"kubernetes.io/projected/650103cb-ceae-4160-b043-d66f063f0890-kube-api-access-hkxt2\") pod \"650103cb-ceae-4160-b043-d66f063f0890\" (UID: \"650103cb-ceae-4160-b043-d66f063f0890\") " Jul 06 16:53:48.235675 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:53:48.235505 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/650103cb-ceae-4160-b043-d66f063f0890-openshift-service-ca\") pod \"650103cb-ceae-4160-b043-d66f063f0890\" (UID: \"650103cb-ceae-4160-b043-d66f063f0890\") " Jul 06 16:53:48.235675 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:53:48.235552 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/650103cb-ceae-4160-b043-d66f063f0890-maas-api-tls\") pod \"650103cb-ceae-4160-b043-d66f063f0890\" (UID: \"650103cb-ceae-4160-b043-d66f063f0890\") " Jul 06 16:53:48.235974 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:53:48.235951 2584 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/650103cb-ceae-4160-b043-d66f063f0890-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "650103cb-ceae-4160-b043-d66f063f0890" (UID: "650103cb-ceae-4160-b043-d66f063f0890"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 16:53:48.237683 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:53:48.237650 2584 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/650103cb-ceae-4160-b043-d66f063f0890-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "650103cb-ceae-4160-b043-d66f063f0890" (UID: "650103cb-ceae-4160-b043-d66f063f0890"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 16:53:48.237683 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:53:48.237664 2584 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/650103cb-ceae-4160-b043-d66f063f0890-kube-api-access-hkxt2" (OuterVolumeSpecName: "kube-api-access-hkxt2") pod "650103cb-ceae-4160-b043-d66f063f0890" (UID: "650103cb-ceae-4160-b043-d66f063f0890"). InnerVolumeSpecName "kube-api-access-hkxt2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 16:53:48.337014 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:53:48.336979 2584 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/650103cb-ceae-4160-b043-d66f063f0890-openshift-service-ca\") on node \"ip-10-0-128-22.ec2.internal\" DevicePath \"\"" Jul 06 16:53:48.337014 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:53:48.337011 2584 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/650103cb-ceae-4160-b043-d66f063f0890-maas-api-tls\") on node \"ip-10-0-128-22.ec2.internal\" DevicePath \"\"" Jul 06 16:53:48.337014 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:53:48.337021 2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hkxt2\" (UniqueName: \"kubernetes.io/projected/650103cb-ceae-4160-b043-d66f063f0890-kube-api-access-hkxt2\") on node \"ip-10-0-128-22.ec2.internal\" DevicePath \"\"" Jul 06 16:53:48.693369 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:53:48.693340 2584 generic.go:356] "Generic (PLEG): container finished" podID="650103cb-ceae-4160-b043-d66f063f0890" containerID="75bfdcee0b44f789ca215353db432440e5fe79d243dac3fad1d78a50ba435242" exitCode=0 Jul 06 16:53:48.693797 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:53:48.693385 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-8f668f5c-74b79577dd-fvf4j" Jul 06 16:53:48.693797 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:53:48.693436 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-8f668f5c-74b79577dd-fvf4j" event={"ID":"650103cb-ceae-4160-b043-d66f063f0890","Type":"ContainerDied","Data":"75bfdcee0b44f789ca215353db432440e5fe79d243dac3fad1d78a50ba435242"} Jul 06 16:53:48.693797 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:53:48.693485 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-8f668f5c-74b79577dd-fvf4j" event={"ID":"650103cb-ceae-4160-b043-d66f063f0890","Type":"ContainerDied","Data":"d83b3716c2280c99c4db209f2d960bc37321eae4a67bcafe9b6d5c3ede2e2383"} Jul 06 16:53:48.693797 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:53:48.693506 2584 scope.go:117] "RemoveContainer" containerID="75bfdcee0b44f789ca215353db432440e5fe79d243dac3fad1d78a50ba435242" Jul 06 16:53:48.702981 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:53:48.702958 2584 scope.go:117] "RemoveContainer" containerID="75bfdcee0b44f789ca215353db432440e5fe79d243dac3fad1d78a50ba435242" Jul 06 16:53:48.703289 ip-10-0-128-22 kubenswrapper[2584]: E0706 16:53:48.703270 2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"75bfdcee0b44f789ca215353db432440e5fe79d243dac3fad1d78a50ba435242\": container with ID starting with 75bfdcee0b44f789ca215353db432440e5fe79d243dac3fad1d78a50ba435242 not found: ID does not exist" containerID="75bfdcee0b44f789ca215353db432440e5fe79d243dac3fad1d78a50ba435242" Jul 06 16:53:48.703336 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:53:48.703300 2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"75bfdcee0b44f789ca215353db432440e5fe79d243dac3fad1d78a50ba435242"} err="failed to get container status \"75bfdcee0b44f789ca215353db432440e5fe79d243dac3fad1d78a50ba435242\": rpc error: code = NotFound desc = could not find container \"75bfdcee0b44f789ca215353db432440e5fe79d243dac3fad1d78a50ba435242\": container with ID starting with 75bfdcee0b44f789ca215353db432440e5fe79d243dac3fad1d78a50ba435242 not found: ID does not exist" Jul 06 16:53:48.717622 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:53:48.717587 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-mt-8f668f5c-74b79577dd-fvf4j"] Jul 06 16:53:48.726248 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:53:48.726218 2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-mt-8f668f5c-74b79577dd-fvf4j"] Jul 06 16:53:50.118627 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:53:50.118592 2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="650103cb-ceae-4160-b043-d66f063f0890" path="/var/lib/kubelet/pods/650103cb-ceae-4160-b043-d66f063f0890/volumes" Jul 06 16:53:53.714113 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:53:53.714080 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:53:58.711876 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:53:58.711835 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:54:09.823022 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:54:09.822982 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:54:09.922009 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:54:09.921977 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:54:10.024347 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:54:10.024317 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-294bx_6a8a4796-dbbc-43e2-a459-7acf11f9468d/ovn-acl-logging/0.log" Jul 06 16:54:10.030095 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:54:10.030070 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-294bx_6a8a4796-dbbc-43e2-a459-7acf11f9468d/ovn-acl-logging/0.log" Jul 06 16:54:35.127686 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:54:35.127648 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:54:38.636212 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:54:38.636168 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:54:45.048061 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:54:45.048021 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:54:45.143454 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:54:45.143423 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:54:45.333042 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:54:45.332952 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:54:45.575944 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:54:45.575908 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:54:46.649206 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:54:46.649164 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:54:46.752151 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:54:46.752110 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:54:47.606050 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:54:47.606016 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:54:47.702872 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:54:47.702844 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:54:47.805855 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:54:47.805821 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:54:49.283018 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:54:49.282984 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:54:55.556602 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:54:55.556570 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:54:55.657712 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:54:55.657680 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:54:56.219295 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:54:56.219262 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:55:03.794188 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:55:03.794151 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:55:03.871251 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:55:03.871222 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:55:03.991874 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:55:03.991840 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:55:04.083957 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:55:04.083859 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:55:14.573059 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:55:14.572998 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:55:14.941609 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:55:14.941576 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:55:35.940146 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:55:35.940048 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:55:42.920202 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:55:42.920161 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:55:43.015965 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:55:43.015933 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:55:43.268839 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:55:43.268714 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:55:43.596923 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:55:43.596840 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:55:43.717064 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:55:43.717024 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:55:47.084670 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:55:47.084629 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:55:47.176602 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:55:47.176569 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-inf-a-da00b0-65c5dcfb55-qzqbn"] Jul 06 16:55:47.176993 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:55:47.176979 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="650103cb-ceae-4160-b043-d66f063f0890" containerName="maas-api" Jul 06 16:55:47.177049 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:55:47.176995 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="650103cb-ceae-4160-b043-d66f063f0890" containerName="maas-api" Jul 06 16:55:47.177084 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:55:47.177077 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="650103cb-ceae-4160-b043-d66f063f0890" containerName="maas-api" Jul 06 16:55:47.180309 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:55:47.180286 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-a-da00b0-65c5dcfb55-qzqbn" Jul 06 16:55:47.185541 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:55:47.185519 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-inf-a-da00b0-serving-cert\"" Jul 06 16:55:47.188865 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:55:47.188842 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:55:47.194713 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:55:47.194692 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-inf-a-da00b0-65c5dcfb55-qzqbn"] Jul 06 16:55:47.266931 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:55:47.266899 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v4h5w\" (UniqueName: \"kubernetes.io/projected/71a024d6-a7a8-468b-8e70-3e6e2007d9ed-kube-api-access-v4h5w\") pod \"maas-api-e2e-inf-a-da00b0-65c5dcfb55-qzqbn\" (UID: \"71a024d6-a7a8-468b-8e70-3e6e2007d9ed\") " pod="opendatahub/maas-api-e2e-inf-a-da00b0-65c5dcfb55-qzqbn" Jul 06 16:55:47.267113 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:55:47.266942 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/71a024d6-a7a8-468b-8e70-3e6e2007d9ed-openshift-service-ca\") pod \"maas-api-e2e-inf-a-da00b0-65c5dcfb55-qzqbn\" (UID: \"71a024d6-a7a8-468b-8e70-3e6e2007d9ed\") " pod="opendatahub/maas-api-e2e-inf-a-da00b0-65c5dcfb55-qzqbn" Jul 06 16:55:47.267113 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:55:47.267034 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/71a024d6-a7a8-468b-8e70-3e6e2007d9ed-maas-api-tls\") pod \"maas-api-e2e-inf-a-da00b0-65c5dcfb55-qzqbn\" (UID: \"71a024d6-a7a8-468b-8e70-3e6e2007d9ed\") " pod="opendatahub/maas-api-e2e-inf-a-da00b0-65c5dcfb55-qzqbn" Jul 06 16:55:47.292070 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:55:47.292036 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:55:47.368407 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:55:47.368317 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v4h5w\" (UniqueName: \"kubernetes.io/projected/71a024d6-a7a8-468b-8e70-3e6e2007d9ed-kube-api-access-v4h5w\") pod \"maas-api-e2e-inf-a-da00b0-65c5dcfb55-qzqbn\" (UID: \"71a024d6-a7a8-468b-8e70-3e6e2007d9ed\") " pod="opendatahub/maas-api-e2e-inf-a-da00b0-65c5dcfb55-qzqbn" Jul 06 16:55:47.368407 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:55:47.368357 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/71a024d6-a7a8-468b-8e70-3e6e2007d9ed-openshift-service-ca\") pod \"maas-api-e2e-inf-a-da00b0-65c5dcfb55-qzqbn\" (UID: \"71a024d6-a7a8-468b-8e70-3e6e2007d9ed\") " pod="opendatahub/maas-api-e2e-inf-a-da00b0-65c5dcfb55-qzqbn" Jul 06 16:55:47.368611 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:55:47.368590 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/71a024d6-a7a8-468b-8e70-3e6e2007d9ed-maas-api-tls\") pod \"maas-api-e2e-inf-a-da00b0-65c5dcfb55-qzqbn\" (UID: \"71a024d6-a7a8-468b-8e70-3e6e2007d9ed\") " pod="opendatahub/maas-api-e2e-inf-a-da00b0-65c5dcfb55-qzqbn" Jul 06 16:55:47.368751 ip-10-0-128-22 kubenswrapper[2584]: E0706 16:55:47.368712 2584 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-inf-a-da00b0-serving-cert: secret "maas-api-e2e-inf-a-da00b0-serving-cert" not found Jul 06 16:55:47.368799 ip-10-0-128-22 kubenswrapper[2584]: E0706 16:55:47.368789 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/71a024d6-a7a8-468b-8e70-3e6e2007d9ed-maas-api-tls podName:71a024d6-a7a8-468b-8e70-3e6e2007d9ed nodeName:}" failed. No retries permitted until 2026-07-06 16:55:47.868774598 +0000 UTC m=+1898.482914697 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/71a024d6-a7a8-468b-8e70-3e6e2007d9ed-maas-api-tls") pod "maas-api-e2e-inf-a-da00b0-65c5dcfb55-qzqbn" (UID: "71a024d6-a7a8-468b-8e70-3e6e2007d9ed") : secret "maas-api-e2e-inf-a-da00b0-serving-cert" not found Jul 06 16:55:47.369017 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:55:47.368999 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/71a024d6-a7a8-468b-8e70-3e6e2007d9ed-openshift-service-ca\") pod \"maas-api-e2e-inf-a-da00b0-65c5dcfb55-qzqbn\" (UID: \"71a024d6-a7a8-468b-8e70-3e6e2007d9ed\") " pod="opendatahub/maas-api-e2e-inf-a-da00b0-65c5dcfb55-qzqbn" Jul 06 16:55:47.389947 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:55:47.389919 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v4h5w\" (UniqueName: \"kubernetes.io/projected/71a024d6-a7a8-468b-8e70-3e6e2007d9ed-kube-api-access-v4h5w\") pod \"maas-api-e2e-inf-a-da00b0-65c5dcfb55-qzqbn\" (UID: \"71a024d6-a7a8-468b-8e70-3e6e2007d9ed\") " pod="opendatahub/maas-api-e2e-inf-a-da00b0-65c5dcfb55-qzqbn" Jul 06 16:55:47.732895 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:55:47.732857 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:55:47.835654 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:55:47.835612 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:55:47.873785 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:55:47.873746 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/71a024d6-a7a8-468b-8e70-3e6e2007d9ed-maas-api-tls\") pod \"maas-api-e2e-inf-a-da00b0-65c5dcfb55-qzqbn\" (UID: \"71a024d6-a7a8-468b-8e70-3e6e2007d9ed\") " pod="opendatahub/maas-api-e2e-inf-a-da00b0-65c5dcfb55-qzqbn" Jul 06 16:55:47.876146 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:55:47.876124 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/71a024d6-a7a8-468b-8e70-3e6e2007d9ed-maas-api-tls\") pod \"maas-api-e2e-inf-a-da00b0-65c5dcfb55-qzqbn\" (UID: \"71a024d6-a7a8-468b-8e70-3e6e2007d9ed\") " pod="opendatahub/maas-api-e2e-inf-a-da00b0-65c5dcfb55-qzqbn" Jul 06 16:55:48.090657 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:55:48.090562 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-a-da00b0-65c5dcfb55-qzqbn" Jul 06 16:55:48.243416 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:55:48.243387 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-inf-a-da00b0-65c5dcfb55-qzqbn"] Jul 06 16:55:48.244973 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:55:48.244946 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod71a024d6_a7a8_468b_8e70_3e6e2007d9ed.slice/crio-3bd45ef5de8cf8004b15128b6d2b87a8ec90816c288661cdc37e6ed92b01f7c5 WatchSource:0}: Error finding container 3bd45ef5de8cf8004b15128b6d2b87a8ec90816c288661cdc37e6ed92b01f7c5: Status 404 returned error can't find the container with id 3bd45ef5de8cf8004b15128b6d2b87a8ec90816c288661cdc37e6ed92b01f7c5 Jul 06 16:55:48.246778 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:55:48.246761 2584 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 06 16:55:48.264898 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:55:48.264856 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-a-da00b0-65c5dcfb55-qzqbn" event={"ID":"71a024d6-a7a8-468b-8e70-3e6e2007d9ed","Type":"ContainerStarted","Data":"3bd45ef5de8cf8004b15128b6d2b87a8ec90816c288661cdc37e6ed92b01f7c5"} Jul 06 16:55:49.271500 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:55:49.271464 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-a-da00b0-65c5dcfb55-qzqbn" event={"ID":"71a024d6-a7a8-468b-8e70-3e6e2007d9ed","Type":"ContainerStarted","Data":"030266f9bdec70a2341dcdf0947cb83d50a5664eba09598016ac3c9440a74847"} Jul 06 16:55:49.271959 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:55:49.271596 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-inf-a-da00b0-65c5dcfb55-qzqbn" Jul 06 16:55:49.308214 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:55:49.308156 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-inf-a-da00b0-65c5dcfb55-qzqbn" podStartSLOduration=1.897298022 podStartE2EDuration="2.308137059s" totalImagesPullingTime="410.839037ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 16:55:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 16:55:49.30802941 +0000 UTC m=+1899.922169531" watchObservedRunningTime="2026-07-06 16:55:49.308137059 +0000 UTC m=+1899.922277179" Jul 06 16:55:51.569782 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:55:51.569717 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:55:51.939490 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:55:51.939459 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:55:52.355481 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:55:52.355396 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:55:53.036347 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:55:53.036315 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:55:55.165643 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:55:55.165601 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:55:55.263902 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:55:55.263856 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:55:55.281407 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:55:55.281372 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-inf-b-e8d6d5-884588b45-tc899"] Jul 06 16:55:55.285150 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:55:55.285127 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-inf-a-da00b0-65c5dcfb55-qzqbn" Jul 06 16:55:55.285273 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:55:55.285211 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-b-e8d6d5-884588b45-tc899" Jul 06 16:55:55.288163 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:55:55.288139 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-inf-b-e8d6d5-serving-cert\"" Jul 06 16:55:55.305749 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:55:55.305702 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-inf-b-e8d6d5-884588b45-tc899"] Jul 06 16:55:55.347645 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:55:55.347611 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3e8a2c8e-332c-4f7f-9ac5-366b65990c68-maas-api-tls\") pod \"maas-api-e2e-inf-b-e8d6d5-884588b45-tc899\" (UID: \"3e8a2c8e-332c-4f7f-9ac5-366b65990c68\") " pod="opendatahub/maas-api-e2e-inf-b-e8d6d5-884588b45-tc899" Jul 06 16:55:55.347967 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:55:55.347936 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3e8a2c8e-332c-4f7f-9ac5-366b65990c68-openshift-service-ca\") pod \"maas-api-e2e-inf-b-e8d6d5-884588b45-tc899\" (UID: \"3e8a2c8e-332c-4f7f-9ac5-366b65990c68\") " pod="opendatahub/maas-api-e2e-inf-b-e8d6d5-884588b45-tc899" Jul 06 16:55:55.348083 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:55:55.347997 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lkd2d\" (UniqueName: \"kubernetes.io/projected/3e8a2c8e-332c-4f7f-9ac5-366b65990c68-kube-api-access-lkd2d\") pod \"maas-api-e2e-inf-b-e8d6d5-884588b45-tc899\" (UID: \"3e8a2c8e-332c-4f7f-9ac5-366b65990c68\") " pod="opendatahub/maas-api-e2e-inf-b-e8d6d5-884588b45-tc899" Jul 06 16:55:55.396385 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:55:55.396350 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:55:55.449477 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:55:55.449123 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3e8a2c8e-332c-4f7f-9ac5-366b65990c68-openshift-service-ca\") pod \"maas-api-e2e-inf-b-e8d6d5-884588b45-tc899\" (UID: \"3e8a2c8e-332c-4f7f-9ac5-366b65990c68\") " pod="opendatahub/maas-api-e2e-inf-b-e8d6d5-884588b45-tc899" Jul 06 16:55:55.449477 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:55:55.449175 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lkd2d\" (UniqueName: \"kubernetes.io/projected/3e8a2c8e-332c-4f7f-9ac5-366b65990c68-kube-api-access-lkd2d\") pod \"maas-api-e2e-inf-b-e8d6d5-884588b45-tc899\" (UID: \"3e8a2c8e-332c-4f7f-9ac5-366b65990c68\") " pod="opendatahub/maas-api-e2e-inf-b-e8d6d5-884588b45-tc899" Jul 06 16:55:55.449477 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:55:55.449297 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3e8a2c8e-332c-4f7f-9ac5-366b65990c68-maas-api-tls\") pod \"maas-api-e2e-inf-b-e8d6d5-884588b45-tc899\" (UID: \"3e8a2c8e-332c-4f7f-9ac5-366b65990c68\") " pod="opendatahub/maas-api-e2e-inf-b-e8d6d5-884588b45-tc899" Jul 06 16:55:55.449477 ip-10-0-128-22 kubenswrapper[2584]: E0706 16:55:55.449433 2584 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-inf-b-e8d6d5-serving-cert: secret "maas-api-e2e-inf-b-e8d6d5-serving-cert" not found Jul 06 16:55:55.450196 ip-10-0-128-22 kubenswrapper[2584]: E0706 16:55:55.449511 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3e8a2c8e-332c-4f7f-9ac5-366b65990c68-maas-api-tls podName:3e8a2c8e-332c-4f7f-9ac5-366b65990c68 nodeName:}" failed. No retries permitted until 2026-07-06 16:55:55.949490386 +0000 UTC m=+1906.563630506 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/3e8a2c8e-332c-4f7f-9ac5-366b65990c68-maas-api-tls") pod "maas-api-e2e-inf-b-e8d6d5-884588b45-tc899" (UID: "3e8a2c8e-332c-4f7f-9ac5-366b65990c68") : secret "maas-api-e2e-inf-b-e8d6d5-serving-cert" not found Jul 06 16:55:55.450196 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:55:55.450047 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3e8a2c8e-332c-4f7f-9ac5-366b65990c68-openshift-service-ca\") pod \"maas-api-e2e-inf-b-e8d6d5-884588b45-tc899\" (UID: \"3e8a2c8e-332c-4f7f-9ac5-366b65990c68\") " pod="opendatahub/maas-api-e2e-inf-b-e8d6d5-884588b45-tc899" Jul 06 16:55:55.464002 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:55:55.463972 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lkd2d\" (UniqueName: \"kubernetes.io/projected/3e8a2c8e-332c-4f7f-9ac5-366b65990c68-kube-api-access-lkd2d\") pod \"maas-api-e2e-inf-b-e8d6d5-884588b45-tc899\" (UID: \"3e8a2c8e-332c-4f7f-9ac5-366b65990c68\") " pod="opendatahub/maas-api-e2e-inf-b-e8d6d5-884588b45-tc899" Jul 06 16:55:55.522465 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:55:55.522434 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:55:55.955849 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:55:55.955810 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3e8a2c8e-332c-4f7f-9ac5-366b65990c68-maas-api-tls\") pod \"maas-api-e2e-inf-b-e8d6d5-884588b45-tc899\" (UID: \"3e8a2c8e-332c-4f7f-9ac5-366b65990c68\") " pod="opendatahub/maas-api-e2e-inf-b-e8d6d5-884588b45-tc899" Jul 06 16:55:55.958225 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:55:55.958201 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3e8a2c8e-332c-4f7f-9ac5-366b65990c68-maas-api-tls\") pod \"maas-api-e2e-inf-b-e8d6d5-884588b45-tc899\" (UID: \"3e8a2c8e-332c-4f7f-9ac5-366b65990c68\") " pod="opendatahub/maas-api-e2e-inf-b-e8d6d5-884588b45-tc899" Jul 06 16:55:56.043060 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:55:56.043022 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:55:56.154651 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:55:56.154610 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:55:56.195677 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:55:56.195642 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-b-e8d6d5-884588b45-tc899" Jul 06 16:55:56.347905 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:55:56.347879 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-inf-b-e8d6d5-884588b45-tc899"] Jul 06 16:55:56.350516 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:55:56.350488 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3e8a2c8e_332c_4f7f_9ac5_366b65990c68.slice/crio-b674357d8d6499e611c04fe4ae274f314e3a00dc50370f9461122359ae33dc87 WatchSource:0}: Error finding container b674357d8d6499e611c04fe4ae274f314e3a00dc50370f9461122359ae33dc87: Status 404 returned error can't find the container with id b674357d8d6499e611c04fe4ae274f314e3a00dc50370f9461122359ae33dc87 Jul 06 16:55:57.315321 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:55:57.315283 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-b-e8d6d5-884588b45-tc899" event={"ID":"3e8a2c8e-332c-4f7f-9ac5-366b65990c68","Type":"ContainerStarted","Data":"96e8915fe5db0ef0a156a54a386a7220143ffebc91f0adcf0059523b94ebc0b8"} Jul 06 16:55:57.315846 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:55:57.315349 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-b-e8d6d5-884588b45-tc899" event={"ID":"3e8a2c8e-332c-4f7f-9ac5-366b65990c68","Type":"ContainerStarted","Data":"b674357d8d6499e611c04fe4ae274f314e3a00dc50370f9461122359ae33dc87"} Jul 06 16:55:57.315846 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:55:57.315405 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-inf-b-e8d6d5-884588b45-tc899" Jul 06 16:55:59.664412 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:55:59.664378 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:56:03.325468 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:56:03.325437 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-inf-b-e8d6d5-884588b45-tc899" Jul 06 16:56:03.355525 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:56:03.355461 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-inf-b-e8d6d5-884588b45-tc899" podStartSLOduration=8.036101143 podStartE2EDuration="8.35544175s" totalImagesPullingTime="319.340607ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 16:55:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 16:55:57.34540462 +0000 UTC m=+1907.959544740" watchObservedRunningTime="2026-07-06 16:56:03.35544175 +0000 UTC m=+1913.969581871" Jul 06 16:56:09.519397 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:56:09.519351 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:56:09.622581 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:56:09.622533 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:56:18.325801 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:56:18.325764 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:56:18.423417 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:56:18.423382 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:56:27.907073 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:56:27.907022 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:56:51.593184 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:56:51.593152 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:57:08.873209 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:57:08.873087 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:57:09.783176 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:57:09.783135 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:57:10.134002 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:57:10.133883 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:57:10.531483 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:57:10.531425 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:57:10.674518 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:57:10.674480 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:57:10.773846 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:57:10.773807 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:57:10.930029 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:57:10.929996 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-inf-a-da00b0-65c5dcfb55-qzqbn"] Jul 06 16:57:10.930253 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:57:10.930225 2584 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-inf-a-da00b0-65c5dcfb55-qzqbn" podUID="71a024d6-a7a8-468b-8e70-3e6e2007d9ed" containerName="maas-api" containerID="cri-o://030266f9bdec70a2341dcdf0947cb83d50a5664eba09598016ac3c9440a74847" gracePeriod=30 Jul 06 16:57:10.960905 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:57:10.960871 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:57:11.389588 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:57:11.389556 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-a-da00b0-65c5dcfb55-qzqbn" Jul 06 16:57:11.489447 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:57:11.489364 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/71a024d6-a7a8-468b-8e70-3e6e2007d9ed-openshift-service-ca\") pod \"71a024d6-a7a8-468b-8e70-3e6e2007d9ed\" (UID: \"71a024d6-a7a8-468b-8e70-3e6e2007d9ed\") " Jul 06 16:57:11.489598 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:57:11.489507 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v4h5w\" (UniqueName: \"kubernetes.io/projected/71a024d6-a7a8-468b-8e70-3e6e2007d9ed-kube-api-access-v4h5w\") pod \"71a024d6-a7a8-468b-8e70-3e6e2007d9ed\" (UID: \"71a024d6-a7a8-468b-8e70-3e6e2007d9ed\") " Jul 06 16:57:11.489598 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:57:11.489543 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/71a024d6-a7a8-468b-8e70-3e6e2007d9ed-maas-api-tls\") pod \"71a024d6-a7a8-468b-8e70-3e6e2007d9ed\" (UID: \"71a024d6-a7a8-468b-8e70-3e6e2007d9ed\") " Jul 06 16:57:11.489811 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:57:11.489787 2584 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/71a024d6-a7a8-468b-8e70-3e6e2007d9ed-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "71a024d6-a7a8-468b-8e70-3e6e2007d9ed" (UID: "71a024d6-a7a8-468b-8e70-3e6e2007d9ed"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 16:57:11.489942 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:57:11.489929 2584 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/71a024d6-a7a8-468b-8e70-3e6e2007d9ed-openshift-service-ca\") on node \"ip-10-0-128-22.ec2.internal\" DevicePath \"\"" Jul 06 16:57:11.491577 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:57:11.491544 2584 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/71a024d6-a7a8-468b-8e70-3e6e2007d9ed-kube-api-access-v4h5w" (OuterVolumeSpecName: "kube-api-access-v4h5w") pod "71a024d6-a7a8-468b-8e70-3e6e2007d9ed" (UID: "71a024d6-a7a8-468b-8e70-3e6e2007d9ed"). InnerVolumeSpecName "kube-api-access-v4h5w". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 16:57:11.491675 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:57:11.491636 2584 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/71a024d6-a7a8-468b-8e70-3e6e2007d9ed-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "71a024d6-a7a8-468b-8e70-3e6e2007d9ed" (UID: "71a024d6-a7a8-468b-8e70-3e6e2007d9ed"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 16:57:11.591278 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:57:11.591240 2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v4h5w\" (UniqueName: \"kubernetes.io/projected/71a024d6-a7a8-468b-8e70-3e6e2007d9ed-kube-api-access-v4h5w\") on node \"ip-10-0-128-22.ec2.internal\" DevicePath \"\"" Jul 06 16:57:11.591278 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:57:11.591271 2584 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/71a024d6-a7a8-468b-8e70-3e6e2007d9ed-maas-api-tls\") on node \"ip-10-0-128-22.ec2.internal\" DevicePath \"\"" Jul 06 16:57:11.678411 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:57:11.678376 2584 generic.go:356] "Generic (PLEG): container finished" podID="71a024d6-a7a8-468b-8e70-3e6e2007d9ed" containerID="030266f9bdec70a2341dcdf0947cb83d50a5664eba09598016ac3c9440a74847" exitCode=0 Jul 06 16:57:11.678589 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:57:11.678427 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-a-da00b0-65c5dcfb55-qzqbn" Jul 06 16:57:11.678589 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:57:11.678459 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-a-da00b0-65c5dcfb55-qzqbn" event={"ID":"71a024d6-a7a8-468b-8e70-3e6e2007d9ed","Type":"ContainerDied","Data":"030266f9bdec70a2341dcdf0947cb83d50a5664eba09598016ac3c9440a74847"} Jul 06 16:57:11.678589 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:57:11.678511 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-a-da00b0-65c5dcfb55-qzqbn" event={"ID":"71a024d6-a7a8-468b-8e70-3e6e2007d9ed","Type":"ContainerDied","Data":"3bd45ef5de8cf8004b15128b6d2b87a8ec90816c288661cdc37e6ed92b01f7c5"} Jul 06 16:57:11.678589 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:57:11.678535 2584 scope.go:117] "RemoveContainer" containerID="030266f9bdec70a2341dcdf0947cb83d50a5664eba09598016ac3c9440a74847" Jul 06 16:57:11.688800 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:57:11.688777 2584 scope.go:117] "RemoveContainer" containerID="030266f9bdec70a2341dcdf0947cb83d50a5664eba09598016ac3c9440a74847" Jul 06 16:57:11.689127 ip-10-0-128-22 kubenswrapper[2584]: E0706 16:57:11.689103 2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"030266f9bdec70a2341dcdf0947cb83d50a5664eba09598016ac3c9440a74847\": container with ID starting with 030266f9bdec70a2341dcdf0947cb83d50a5664eba09598016ac3c9440a74847 not found: ID does not exist" containerID="030266f9bdec70a2341dcdf0947cb83d50a5664eba09598016ac3c9440a74847" Jul 06 16:57:11.689216 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:57:11.689134 2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"030266f9bdec70a2341dcdf0947cb83d50a5664eba09598016ac3c9440a74847"} err="failed to get container status \"030266f9bdec70a2341dcdf0947cb83d50a5664eba09598016ac3c9440a74847\": rpc error: code = NotFound desc = could not find container \"030266f9bdec70a2341dcdf0947cb83d50a5664eba09598016ac3c9440a74847\": container with ID starting with 030266f9bdec70a2341dcdf0947cb83d50a5664eba09598016ac3c9440a74847 not found: ID does not exist" Jul 06 16:57:11.705204 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:57:11.705173 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-inf-a-da00b0-65c5dcfb55-qzqbn"] Jul 06 16:57:11.709882 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:57:11.709854 2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-inf-a-da00b0-65c5dcfb55-qzqbn"] Jul 06 16:57:12.119224 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:57:12.119190 2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="71a024d6-a7a8-468b-8e70-3e6e2007d9ed" path="/var/lib/kubelet/pods/71a024d6-a7a8-468b-8e70-3e6e2007d9ed/volumes" Jul 06 16:57:15.596930 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:57:15.596896 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:57:26.815206 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:57:26.815169 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:57:31.685227 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:57:31.685190 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:57:31.787111 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:57:31.787072 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:57:53.058019 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:57:53.057982 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:57:54.576471 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:57:54.576435 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-inf-b-e8d6d5-884588b45-tc899"] Jul 06 16:57:54.577013 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:57:54.576678 2584 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-inf-b-e8d6d5-884588b45-tc899" podUID="3e8a2c8e-332c-4f7f-9ac5-366b65990c68" containerName="maas-api" containerID="cri-o://96e8915fe5db0ef0a156a54a386a7220143ffebc91f0adcf0059523b94ebc0b8" gracePeriod=30 Jul 06 16:57:54.647061 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:57:54.647032 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:57:54.831044 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:57:54.830977 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-b-e8d6d5-884588b45-tc899" Jul 06 16:57:54.898460 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:57:54.898426 2584 generic.go:356] "Generic (PLEG): container finished" podID="3e8a2c8e-332c-4f7f-9ac5-366b65990c68" containerID="96e8915fe5db0ef0a156a54a386a7220143ffebc91f0adcf0059523b94ebc0b8" exitCode=0 Jul 06 16:57:54.898628 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:57:54.898473 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-b-e8d6d5-884588b45-tc899" Jul 06 16:57:54.898628 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:57:54.898478 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-b-e8d6d5-884588b45-tc899" event={"ID":"3e8a2c8e-332c-4f7f-9ac5-366b65990c68","Type":"ContainerDied","Data":"96e8915fe5db0ef0a156a54a386a7220143ffebc91f0adcf0059523b94ebc0b8"} Jul 06 16:57:54.898628 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:57:54.898602 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-b-e8d6d5-884588b45-tc899" event={"ID":"3e8a2c8e-332c-4f7f-9ac5-366b65990c68","Type":"ContainerDied","Data":"b674357d8d6499e611c04fe4ae274f314e3a00dc50370f9461122359ae33dc87"} Jul 06 16:57:54.898769 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:57:54.898637 2584 scope.go:117] "RemoveContainer" containerID="96e8915fe5db0ef0a156a54a386a7220143ffebc91f0adcf0059523b94ebc0b8" Jul 06 16:57:54.907293 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:57:54.907274 2584 scope.go:117] "RemoveContainer" containerID="96e8915fe5db0ef0a156a54a386a7220143ffebc91f0adcf0059523b94ebc0b8" Jul 06 16:57:54.907559 ip-10-0-128-22 kubenswrapper[2584]: E0706 16:57:54.907542 2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"96e8915fe5db0ef0a156a54a386a7220143ffebc91f0adcf0059523b94ebc0b8\": container with ID starting with 96e8915fe5db0ef0a156a54a386a7220143ffebc91f0adcf0059523b94ebc0b8 not found: ID does not exist" containerID="96e8915fe5db0ef0a156a54a386a7220143ffebc91f0adcf0059523b94ebc0b8" Jul 06 16:57:54.907626 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:57:54.907570 2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"96e8915fe5db0ef0a156a54a386a7220143ffebc91f0adcf0059523b94ebc0b8"} err="failed to get container status \"96e8915fe5db0ef0a156a54a386a7220143ffebc91f0adcf0059523b94ebc0b8\": rpc error: code = NotFound desc = could not find container \"96e8915fe5db0ef0a156a54a386a7220143ffebc91f0adcf0059523b94ebc0b8\": container with ID starting with 96e8915fe5db0ef0a156a54a386a7220143ffebc91f0adcf0059523b94ebc0b8 not found: ID does not exist" Jul 06 16:57:54.918995 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:57:54.918958 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3e8a2c8e-332c-4f7f-9ac5-366b65990c68-openshift-service-ca\") pod \"3e8a2c8e-332c-4f7f-9ac5-366b65990c68\" (UID: \"3e8a2c8e-332c-4f7f-9ac5-366b65990c68\") " Jul 06 16:57:54.919127 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:57:54.919004 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3e8a2c8e-332c-4f7f-9ac5-366b65990c68-maas-api-tls\") pod \"3e8a2c8e-332c-4f7f-9ac5-366b65990c68\" (UID: \"3e8a2c8e-332c-4f7f-9ac5-366b65990c68\") " Jul 06 16:57:54.919127 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:57:54.919071 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lkd2d\" (UniqueName: \"kubernetes.io/projected/3e8a2c8e-332c-4f7f-9ac5-366b65990c68-kube-api-access-lkd2d\") pod \"3e8a2c8e-332c-4f7f-9ac5-366b65990c68\" (UID: \"3e8a2c8e-332c-4f7f-9ac5-366b65990c68\") " Jul 06 16:57:54.919387 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:57:54.919359 2584 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3e8a2c8e-332c-4f7f-9ac5-366b65990c68-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "3e8a2c8e-332c-4f7f-9ac5-366b65990c68" (UID: "3e8a2c8e-332c-4f7f-9ac5-366b65990c68"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 16:57:54.921153 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:57:54.921131 2584 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3e8a2c8e-332c-4f7f-9ac5-366b65990c68-kube-api-access-lkd2d" (OuterVolumeSpecName: "kube-api-access-lkd2d") pod "3e8a2c8e-332c-4f7f-9ac5-366b65990c68" (UID: "3e8a2c8e-332c-4f7f-9ac5-366b65990c68"). InnerVolumeSpecName "kube-api-access-lkd2d". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 16:57:54.921224 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:57:54.921205 2584 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3e8a2c8e-332c-4f7f-9ac5-366b65990c68-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "3e8a2c8e-332c-4f7f-9ac5-366b65990c68" (UID: "3e8a2c8e-332c-4f7f-9ac5-366b65990c68"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 16:57:55.020003 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:57:55.019969 2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lkd2d\" (UniqueName: \"kubernetes.io/projected/3e8a2c8e-332c-4f7f-9ac5-366b65990c68-kube-api-access-lkd2d\") on node \"ip-10-0-128-22.ec2.internal\" DevicePath \"\"" Jul 06 16:57:55.020003 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:57:55.020000 2584 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3e8a2c8e-332c-4f7f-9ac5-366b65990c68-openshift-service-ca\") on node \"ip-10-0-128-22.ec2.internal\" DevicePath \"\"" Jul 06 16:57:55.020003 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:57:55.020010 2584 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3e8a2c8e-332c-4f7f-9ac5-366b65990c68-maas-api-tls\") on node \"ip-10-0-128-22.ec2.internal\" DevicePath \"\"" Jul 06 16:57:55.223462 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:57:55.223427 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-inf-b-e8d6d5-884588b45-tc899"] Jul 06 16:57:55.228301 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:57:55.228273 2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-inf-b-e8d6d5-884588b45-tc899"] Jul 06 16:57:56.118706 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:57:56.118659 2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3e8a2c8e-332c-4f7f-9ac5-366b65990c68" path="/var/lib/kubelet/pods/3e8a2c8e-332c-4f7f-9ac5-366b65990c68/volumes" Jul 06 16:58:01.298428 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:58:01.298390 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:58:01.396095 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:58:01.396063 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:58:01.568562 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:58:01.568461 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:58:01.851191 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:58:01.851076 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:58:02.215848 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:58:02.215810 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:58:02.662135 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:58:02.662103 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:58:02.848303 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:58:02.848271 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:58:05.646744 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:58:05.646686 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:58:05.726956 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:58:05.726925 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:58:05.804030 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:58:05.803996 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:58:05.836079 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:58:05.836043 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-api-a-0064ec-776cb646df-hkgzl"] Jul 06 16:58:05.836468 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:58:05.836454 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3e8a2c8e-332c-4f7f-9ac5-366b65990c68" containerName="maas-api" Jul 06 16:58:05.836537 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:58:05.836469 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="3e8a2c8e-332c-4f7f-9ac5-366b65990c68" containerName="maas-api" Jul 06 16:58:05.836537 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:58:05.836487 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="71a024d6-a7a8-468b-8e70-3e6e2007d9ed" containerName="maas-api" Jul 06 16:58:05.836537 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:58:05.836493 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="71a024d6-a7a8-468b-8e70-3e6e2007d9ed" containerName="maas-api" Jul 06 16:58:05.836694 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:58:05.836572 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="71a024d6-a7a8-468b-8e70-3e6e2007d9ed" containerName="maas-api" Jul 06 16:58:05.836694 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:58:05.836583 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="3e8a2c8e-332c-4f7f-9ac5-366b65990c68" containerName="maas-api" Jul 06 16:58:05.841576 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:58:05.841550 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-a-0064ec-776cb646df-hkgzl" Jul 06 16:58:05.844298 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:58:05.844273 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-api-a-0064ec-serving-cert\"" Jul 06 16:58:05.850566 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:58:05.850538 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-api-a-0064ec-776cb646df-hkgzl"] Jul 06 16:58:05.901147 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:58:05.901028 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:58:05.930380 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:58:05.930318 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/084009d6-580c-4bcc-908e-479015fee5a6-maas-api-tls\") pod \"maas-api-e2e-api-a-0064ec-776cb646df-hkgzl\" (UID: \"084009d6-580c-4bcc-908e-479015fee5a6\") " pod="opendatahub/maas-api-e2e-api-a-0064ec-776cb646df-hkgzl" Jul 06 16:58:05.930595 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:58:05.930463 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/084009d6-580c-4bcc-908e-479015fee5a6-openshift-service-ca\") pod \"maas-api-e2e-api-a-0064ec-776cb646df-hkgzl\" (UID: \"084009d6-580c-4bcc-908e-479015fee5a6\") " pod="opendatahub/maas-api-e2e-api-a-0064ec-776cb646df-hkgzl" Jul 06 16:58:05.930595 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:58:05.930493 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8jgfj\" (UniqueName: \"kubernetes.io/projected/084009d6-580c-4bcc-908e-479015fee5a6-kube-api-access-8jgfj\") pod \"maas-api-e2e-api-a-0064ec-776cb646df-hkgzl\" (UID: \"084009d6-580c-4bcc-908e-479015fee5a6\") " pod="opendatahub/maas-api-e2e-api-a-0064ec-776cb646df-hkgzl" Jul 06 16:58:06.031822 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:58:06.031786 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/084009d6-580c-4bcc-908e-479015fee5a6-maas-api-tls\") pod \"maas-api-e2e-api-a-0064ec-776cb646df-hkgzl\" (UID: \"084009d6-580c-4bcc-908e-479015fee5a6\") " pod="opendatahub/maas-api-e2e-api-a-0064ec-776cb646df-hkgzl" Jul 06 16:58:06.032002 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:58:06.031842 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/084009d6-580c-4bcc-908e-479015fee5a6-openshift-service-ca\") pod \"maas-api-e2e-api-a-0064ec-776cb646df-hkgzl\" (UID: \"084009d6-580c-4bcc-908e-479015fee5a6\") " pod="opendatahub/maas-api-e2e-api-a-0064ec-776cb646df-hkgzl" Jul 06 16:58:06.032002 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:58:06.031871 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8jgfj\" (UniqueName: \"kubernetes.io/projected/084009d6-580c-4bcc-908e-479015fee5a6-kube-api-access-8jgfj\") pod \"maas-api-e2e-api-a-0064ec-776cb646df-hkgzl\" (UID: \"084009d6-580c-4bcc-908e-479015fee5a6\") " pod="opendatahub/maas-api-e2e-api-a-0064ec-776cb646df-hkgzl" Jul 06 16:58:06.032582 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:58:06.032557 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/084009d6-580c-4bcc-908e-479015fee5a6-openshift-service-ca\") pod \"maas-api-e2e-api-a-0064ec-776cb646df-hkgzl\" (UID: \"084009d6-580c-4bcc-908e-479015fee5a6\") " pod="opendatahub/maas-api-e2e-api-a-0064ec-776cb646df-hkgzl" Jul 06 16:58:06.034408 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:58:06.034382 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/084009d6-580c-4bcc-908e-479015fee5a6-maas-api-tls\") pod \"maas-api-e2e-api-a-0064ec-776cb646df-hkgzl\" (UID: \"084009d6-580c-4bcc-908e-479015fee5a6\") " pod="opendatahub/maas-api-e2e-api-a-0064ec-776cb646df-hkgzl" Jul 06 16:58:06.038147 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:58:06.038123 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:58:06.048405 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:58:06.048380 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8jgfj\" (UniqueName: \"kubernetes.io/projected/084009d6-580c-4bcc-908e-479015fee5a6-kube-api-access-8jgfj\") pod \"maas-api-e2e-api-a-0064ec-776cb646df-hkgzl\" (UID: \"084009d6-580c-4bcc-908e-479015fee5a6\") " pod="opendatahub/maas-api-e2e-api-a-0064ec-776cb646df-hkgzl" Jul 06 16:58:06.154635 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:58:06.154549 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-a-0064ec-776cb646df-hkgzl" Jul 06 16:58:06.297216 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:58:06.297188 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-api-a-0064ec-776cb646df-hkgzl"] Jul 06 16:58:06.299062 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:58:06.299033 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod084009d6_580c_4bcc_908e_479015fee5a6.slice/crio-517a791a97f0985953e36c2bda50bd5b48bf64e3a28833e23c0990516b75d26a WatchSource:0}: Error finding container 517a791a97f0985953e36c2bda50bd5b48bf64e3a28833e23c0990516b75d26a: Status 404 returned error can't find the container with id 517a791a97f0985953e36c2bda50bd5b48bf64e3a28833e23c0990516b75d26a Jul 06 16:58:06.579939 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:58:06.579905 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:58:06.685236 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:58:06.685205 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:58:06.972407 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:58:06.972366 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-a-0064ec-776cb646df-hkgzl" event={"ID":"084009d6-580c-4bcc-908e-479015fee5a6","Type":"ContainerStarted","Data":"cabc349e387679190fee5253fc4db8aab5c65c195ab8a31cf88d5bdf1eee63ff"} Jul 06 16:58:06.972611 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:58:06.972451 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-a-0064ec-776cb646df-hkgzl" event={"ID":"084009d6-580c-4bcc-908e-479015fee5a6","Type":"ContainerStarted","Data":"517a791a97f0985953e36c2bda50bd5b48bf64e3a28833e23c0990516b75d26a"} Jul 06 16:58:06.972611 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:58:06.972479 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-api-a-0064ec-776cb646df-hkgzl" Jul 06 16:58:06.998754 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:58:06.998684 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-api-a-0064ec-776cb646df-hkgzl" podStartSLOduration=1.598097411 podStartE2EDuration="1.998666836s" totalImagesPullingTime="400.569425ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 16:58:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 16:58:06.996553598 +0000 UTC m=+2037.610693719" watchObservedRunningTime="2026-07-06 16:58:06.998666836 +0000 UTC m=+2037.612806956" Jul 06 16:58:07.241101 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:58:07.241008 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:58:10.276610 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:58:10.276578 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:58:10.742560 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:58:10.742524 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:58:11.102175 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:58:11.102085 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:58:11.369836 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:58:11.369748 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:58:12.984083 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:58:12.984054 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-api-a-0064ec-776cb646df-hkgzl" Jul 06 16:58:14.011242 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:58:14.011199 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:58:14.126863 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:58:14.126828 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:58:14.146211 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:58:14.146180 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-api-b-7ed8f1-74d47988bf-2zcxj"] Jul 06 16:58:14.149884 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:58:14.149856 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-b-7ed8f1-74d47988bf-2zcxj" Jul 06 16:58:14.162966 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:58:14.162937 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-api-b-7ed8f1-74d47988bf-2zcxj"] Jul 06 16:58:14.167061 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:58:14.167031 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-api-b-7ed8f1-serving-cert\"" Jul 06 16:58:14.220872 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:58:14.220776 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q7vpf\" (UniqueName: \"kubernetes.io/projected/36ce1c34-d41a-4d68-b7d0-fe454bf5765c-kube-api-access-q7vpf\") pod \"maas-api-e2e-api-b-7ed8f1-74d47988bf-2zcxj\" (UID: \"36ce1c34-d41a-4d68-b7d0-fe454bf5765c\") " pod="opendatahub/maas-api-e2e-api-b-7ed8f1-74d47988bf-2zcxj" Jul 06 16:58:14.220872 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:58:14.220863 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/36ce1c34-d41a-4d68-b7d0-fe454bf5765c-maas-api-tls\") pod \"maas-api-e2e-api-b-7ed8f1-74d47988bf-2zcxj\" (UID: \"36ce1c34-d41a-4d68-b7d0-fe454bf5765c\") " pod="opendatahub/maas-api-e2e-api-b-7ed8f1-74d47988bf-2zcxj" Jul 06 16:58:14.221077 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:58:14.220912 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/36ce1c34-d41a-4d68-b7d0-fe454bf5765c-openshift-service-ca\") pod \"maas-api-e2e-api-b-7ed8f1-74d47988bf-2zcxj\" (UID: \"36ce1c34-d41a-4d68-b7d0-fe454bf5765c\") " pod="opendatahub/maas-api-e2e-api-b-7ed8f1-74d47988bf-2zcxj" Jul 06 16:58:14.235398 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:58:14.235364 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:58:14.322128 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:58:14.322076 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/36ce1c34-d41a-4d68-b7d0-fe454bf5765c-maas-api-tls\") pod \"maas-api-e2e-api-b-7ed8f1-74d47988bf-2zcxj\" (UID: \"36ce1c34-d41a-4d68-b7d0-fe454bf5765c\") " pod="opendatahub/maas-api-e2e-api-b-7ed8f1-74d47988bf-2zcxj" Jul 06 16:58:14.322343 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:58:14.322195 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/36ce1c34-d41a-4d68-b7d0-fe454bf5765c-openshift-service-ca\") pod \"maas-api-e2e-api-b-7ed8f1-74d47988bf-2zcxj\" (UID: \"36ce1c34-d41a-4d68-b7d0-fe454bf5765c\") " pod="opendatahub/maas-api-e2e-api-b-7ed8f1-74d47988bf-2zcxj" Jul 06 16:58:14.322343 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:58:14.322272 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q7vpf\" (UniqueName: \"kubernetes.io/projected/36ce1c34-d41a-4d68-b7d0-fe454bf5765c-kube-api-access-q7vpf\") pod \"maas-api-e2e-api-b-7ed8f1-74d47988bf-2zcxj\" (UID: \"36ce1c34-d41a-4d68-b7d0-fe454bf5765c\") " pod="opendatahub/maas-api-e2e-api-b-7ed8f1-74d47988bf-2zcxj" Jul 06 16:58:14.322915 ip-10-0-128-22 kubenswrapper[2584]: E0706 16:58:14.322892 2584 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-api-b-7ed8f1-serving-cert: secret "maas-api-e2e-api-b-7ed8f1-serving-cert" not found Jul 06 16:58:14.323672 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:58:14.323646 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/36ce1c34-d41a-4d68-b7d0-fe454bf5765c-openshift-service-ca\") pod \"maas-api-e2e-api-b-7ed8f1-74d47988bf-2zcxj\" (UID: \"36ce1c34-d41a-4d68-b7d0-fe454bf5765c\") " pod="opendatahub/maas-api-e2e-api-b-7ed8f1-74d47988bf-2zcxj" Jul 06 16:58:14.323835 ip-10-0-128-22 kubenswrapper[2584]: E0706 16:58:14.323821 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/36ce1c34-d41a-4d68-b7d0-fe454bf5765c-maas-api-tls podName:36ce1c34-d41a-4d68-b7d0-fe454bf5765c nodeName:}" failed. No retries permitted until 2026-07-06 16:58:14.823788465 +0000 UTC m=+2045.437928580 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/36ce1c34-d41a-4d68-b7d0-fe454bf5765c-maas-api-tls") pod "maas-api-e2e-api-b-7ed8f1-74d47988bf-2zcxj" (UID: "36ce1c34-d41a-4d68-b7d0-fe454bf5765c") : secret "maas-api-e2e-api-b-7ed8f1-serving-cert" not found Jul 06 16:58:14.360709 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:58:14.360673 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q7vpf\" (UniqueName: \"kubernetes.io/projected/36ce1c34-d41a-4d68-b7d0-fe454bf5765c-kube-api-access-q7vpf\") pod \"maas-api-e2e-api-b-7ed8f1-74d47988bf-2zcxj\" (UID: \"36ce1c34-d41a-4d68-b7d0-fe454bf5765c\") " pod="opendatahub/maas-api-e2e-api-b-7ed8f1-74d47988bf-2zcxj" Jul 06 16:58:14.532068 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:58:14.531977 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:58:14.826502 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:58:14.826414 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/36ce1c34-d41a-4d68-b7d0-fe454bf5765c-maas-api-tls\") pod \"maas-api-e2e-api-b-7ed8f1-74d47988bf-2zcxj\" (UID: \"36ce1c34-d41a-4d68-b7d0-fe454bf5765c\") " pod="opendatahub/maas-api-e2e-api-b-7ed8f1-74d47988bf-2zcxj" Jul 06 16:58:14.828876 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:58:14.828852 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/36ce1c34-d41a-4d68-b7d0-fe454bf5765c-maas-api-tls\") pod \"maas-api-e2e-api-b-7ed8f1-74d47988bf-2zcxj\" (UID: \"36ce1c34-d41a-4d68-b7d0-fe454bf5765c\") " pod="opendatahub/maas-api-e2e-api-b-7ed8f1-74d47988bf-2zcxj" Jul 06 16:58:15.003603 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:58:15.003562 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:58:15.059775 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:58:15.059741 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-b-7ed8f1-74d47988bf-2zcxj" Jul 06 16:58:15.102514 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:58:15.102420 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:58:15.215837 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:58:15.215811 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-api-b-7ed8f1-74d47988bf-2zcxj"] Jul 06 16:58:15.216615 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:58:15.216585 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod36ce1c34_d41a_4d68_b7d0_fe454bf5765c.slice/crio-1cf8d521bd7ecdba7925143f735b2e7fb646135e89d58f71f66780a39eb8f267 WatchSource:0}: Error finding container 1cf8d521bd7ecdba7925143f735b2e7fb646135e89d58f71f66780a39eb8f267: Status 404 returned error can't find the container with id 1cf8d521bd7ecdba7925143f735b2e7fb646135e89d58f71f66780a39eb8f267 Jul 06 16:58:16.017758 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:58:16.017634 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-b-7ed8f1-74d47988bf-2zcxj" event={"ID":"36ce1c34-d41a-4d68-b7d0-fe454bf5765c","Type":"ContainerStarted","Data":"6c510bd4d7b963807cd1de66198cf5db2ba9d0656c719a80f6da89f68884487d"} Jul 06 16:58:16.017758 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:58:16.017694 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-b-7ed8f1-74d47988bf-2zcxj" event={"ID":"36ce1c34-d41a-4d68-b7d0-fe454bf5765c","Type":"ContainerStarted","Data":"1cf8d521bd7ecdba7925143f735b2e7fb646135e89d58f71f66780a39eb8f267"} Jul 06 16:58:16.017992 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:58:16.017762 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-api-b-7ed8f1-74d47988bf-2zcxj" Jul 06 16:58:16.039524 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:58:16.039477 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-api-b-7ed8f1-74d47988bf-2zcxj" podStartSLOduration=1.5696079790000002 podStartE2EDuration="2.039459801s" totalImagesPullingTime="469.851822ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 16:58:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 16:58:16.038292409 +0000 UTC m=+2046.652432592" watchObservedRunningTime="2026-07-06 16:58:16.039459801 +0000 UTC m=+2046.653599921" Jul 06 16:58:18.343847 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:58:18.343806 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:58:22.028624 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:58:22.028594 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-api-b-7ed8f1-74d47988bf-2zcxj" Jul 06 16:58:26.597719 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:58:26.597676 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:58:27.018022 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:58:27.017991 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:58:27.426333 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:58:27.426298 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:58:27.531178 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:58:27.531143 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:58:27.749534 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:58:27.749428 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-api-a-0064ec-776cb646df-hkgzl"] Jul 06 16:58:27.750024 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:58:27.749685 2584 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-api-a-0064ec-776cb646df-hkgzl" podUID="084009d6-580c-4bcc-908e-479015fee5a6" containerName="maas-api" containerID="cri-o://cabc349e387679190fee5253fc4db8aab5c65c195ab8a31cf88d5bdf1eee63ff" gracePeriod=30 Jul 06 16:58:27.775382 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:58:27.775339 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:58:27.992871 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:58:27.992839 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-a-0064ec-776cb646df-hkgzl" Jul 06 16:58:28.074226 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:58:28.074132 2584 generic.go:356] "Generic (PLEG): container finished" podID="084009d6-580c-4bcc-908e-479015fee5a6" containerID="cabc349e387679190fee5253fc4db8aab5c65c195ab8a31cf88d5bdf1eee63ff" exitCode=0 Jul 06 16:58:28.074226 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:58:28.074184 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-a-0064ec-776cb646df-hkgzl" Jul 06 16:58:28.074408 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:58:28.074221 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-a-0064ec-776cb646df-hkgzl" event={"ID":"084009d6-580c-4bcc-908e-479015fee5a6","Type":"ContainerDied","Data":"cabc349e387679190fee5253fc4db8aab5c65c195ab8a31cf88d5bdf1eee63ff"} Jul 06 16:58:28.074408 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:58:28.074277 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-a-0064ec-776cb646df-hkgzl" event={"ID":"084009d6-580c-4bcc-908e-479015fee5a6","Type":"ContainerDied","Data":"517a791a97f0985953e36c2bda50bd5b48bf64e3a28833e23c0990516b75d26a"} Jul 06 16:58:28.074408 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:58:28.074321 2584 scope.go:117] "RemoveContainer" containerID="cabc349e387679190fee5253fc4db8aab5c65c195ab8a31cf88d5bdf1eee63ff" Jul 06 16:58:28.082828 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:58:28.082806 2584 scope.go:117] "RemoveContainer" containerID="cabc349e387679190fee5253fc4db8aab5c65c195ab8a31cf88d5bdf1eee63ff" Jul 06 16:58:28.083127 ip-10-0-128-22 kubenswrapper[2584]: E0706 16:58:28.083106 2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cabc349e387679190fee5253fc4db8aab5c65c195ab8a31cf88d5bdf1eee63ff\": container with ID starting with cabc349e387679190fee5253fc4db8aab5c65c195ab8a31cf88d5bdf1eee63ff not found: ID does not exist" containerID="cabc349e387679190fee5253fc4db8aab5c65c195ab8a31cf88d5bdf1eee63ff" Jul 06 16:58:28.083196 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:58:28.083139 2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cabc349e387679190fee5253fc4db8aab5c65c195ab8a31cf88d5bdf1eee63ff"} err="failed to get container status \"cabc349e387679190fee5253fc4db8aab5c65c195ab8a31cf88d5bdf1eee63ff\": rpc error: code = NotFound desc = could not find container \"cabc349e387679190fee5253fc4db8aab5c65c195ab8a31cf88d5bdf1eee63ff\": container with ID starting with cabc349e387679190fee5253fc4db8aab5c65c195ab8a31cf88d5bdf1eee63ff not found: ID does not exist" Jul 06 16:58:28.159150 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:58:28.159121 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/084009d6-580c-4bcc-908e-479015fee5a6-maas-api-tls\") pod \"084009d6-580c-4bcc-908e-479015fee5a6\" (UID: \"084009d6-580c-4bcc-908e-479015fee5a6\") " Jul 06 16:58:28.159150 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:58:28.159155 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8jgfj\" (UniqueName: \"kubernetes.io/projected/084009d6-580c-4bcc-908e-479015fee5a6-kube-api-access-8jgfj\") pod \"084009d6-580c-4bcc-908e-479015fee5a6\" (UID: \"084009d6-580c-4bcc-908e-479015fee5a6\") " Jul 06 16:58:28.159366 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:58:28.159204 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/084009d6-580c-4bcc-908e-479015fee5a6-openshift-service-ca\") pod \"084009d6-580c-4bcc-908e-479015fee5a6\" (UID: \"084009d6-580c-4bcc-908e-479015fee5a6\") " Jul 06 16:58:28.159614 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:58:28.159582 2584 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/084009d6-580c-4bcc-908e-479015fee5a6-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "084009d6-580c-4bcc-908e-479015fee5a6" (UID: "084009d6-580c-4bcc-908e-479015fee5a6"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 16:58:28.161267 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:58:28.161235 2584 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/084009d6-580c-4bcc-908e-479015fee5a6-kube-api-access-8jgfj" (OuterVolumeSpecName: "kube-api-access-8jgfj") pod "084009d6-580c-4bcc-908e-479015fee5a6" (UID: "084009d6-580c-4bcc-908e-479015fee5a6"). InnerVolumeSpecName "kube-api-access-8jgfj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 16:58:28.161364 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:58:28.161326 2584 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/084009d6-580c-4bcc-908e-479015fee5a6-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "084009d6-580c-4bcc-908e-479015fee5a6" (UID: "084009d6-580c-4bcc-908e-479015fee5a6"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 16:58:28.260787 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:58:28.260704 2584 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/084009d6-580c-4bcc-908e-479015fee5a6-maas-api-tls\") on node \"ip-10-0-128-22.ec2.internal\" DevicePath \"\"" Jul 06 16:58:28.260787 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:58:28.260785 2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8jgfj\" (UniqueName: \"kubernetes.io/projected/084009d6-580c-4bcc-908e-479015fee5a6-kube-api-access-8jgfj\") on node \"ip-10-0-128-22.ec2.internal\" DevicePath \"\"" Jul 06 16:58:28.261038 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:58:28.260802 2584 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/084009d6-580c-4bcc-908e-479015fee5a6-openshift-service-ca\") on node \"ip-10-0-128-22.ec2.internal\" DevicePath \"\"" Jul 06 16:58:28.407707 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:58:28.407672 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-api-a-0064ec-776cb646df-hkgzl"] Jul 06 16:58:28.427262 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:58:28.427231 2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-api-a-0064ec-776cb646df-hkgzl"] Jul 06 16:58:30.118362 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:58:30.118324 2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="084009d6-580c-4bcc-908e-479015fee5a6" path="/var/lib/kubelet/pods/084009d6-580c-4bcc-908e-479015fee5a6/volumes" Jul 06 16:58:30.978880 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:58:30.978830 2584 prober.go:120] "Probe failed" probeType="Readiness" pod="opendatahub/maas-api-e2e-api-a-0064ec-776cb646df-hkgzl" podUID="084009d6-580c-4bcc-908e-479015fee5a6" containerName="maas-api" probeResult="failure" output="Get \"https://10.134.0.38:8443/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Jul 06 16:58:33.292910 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:58:33.292875 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:58:33.363525 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:58:33.363488 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:58:33.584089 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:58:33.583996 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:58:33.802385 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:58:33.802345 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:58:33.987761 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:58:33.987718 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:58:34.375161 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:58:34.375071 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:58:34.441792 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:58:34.441759 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:58:34.516280 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:58:34.516246 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:58:34.680627 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:58:34.680590 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-api-b-7ed8f1-74d47988bf-2zcxj"] Jul 06 16:58:34.680897 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:58:34.680861 2584 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-api-b-7ed8f1-74d47988bf-2zcxj" podUID="36ce1c34-d41a-4d68-b7d0-fe454bf5765c" containerName="maas-api" containerID="cri-o://6c510bd4d7b963807cd1de66198cf5db2ba9d0656c719a80f6da89f68884487d" gracePeriod=30 Jul 06 16:58:34.707781 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:58:34.707750 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:58:34.931678 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:58:34.931610 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-b-7ed8f1-74d47988bf-2zcxj" Jul 06 16:58:35.032278 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:58:35.032241 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q7vpf\" (UniqueName: \"kubernetes.io/projected/36ce1c34-d41a-4d68-b7d0-fe454bf5765c-kube-api-access-q7vpf\") pod \"36ce1c34-d41a-4d68-b7d0-fe454bf5765c\" (UID: \"36ce1c34-d41a-4d68-b7d0-fe454bf5765c\") " Jul 06 16:58:35.032484 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:58:35.032310 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/36ce1c34-d41a-4d68-b7d0-fe454bf5765c-openshift-service-ca\") pod \"36ce1c34-d41a-4d68-b7d0-fe454bf5765c\" (UID: \"36ce1c34-d41a-4d68-b7d0-fe454bf5765c\") " Jul 06 16:58:35.032484 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:58:35.032360 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/36ce1c34-d41a-4d68-b7d0-fe454bf5765c-maas-api-tls\") pod \"36ce1c34-d41a-4d68-b7d0-fe454bf5765c\" (UID: \"36ce1c34-d41a-4d68-b7d0-fe454bf5765c\") " Jul 06 16:58:35.032673 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:58:35.032648 2584 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/36ce1c34-d41a-4d68-b7d0-fe454bf5765c-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "36ce1c34-d41a-4d68-b7d0-fe454bf5765c" (UID: "36ce1c34-d41a-4d68-b7d0-fe454bf5765c"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 16:58:35.034523 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:58:35.034495 2584 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/36ce1c34-d41a-4d68-b7d0-fe454bf5765c-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "36ce1c34-d41a-4d68-b7d0-fe454bf5765c" (UID: "36ce1c34-d41a-4d68-b7d0-fe454bf5765c"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 16:58:35.034610 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:58:35.034563 2584 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/36ce1c34-d41a-4d68-b7d0-fe454bf5765c-kube-api-access-q7vpf" (OuterVolumeSpecName: "kube-api-access-q7vpf") pod "36ce1c34-d41a-4d68-b7d0-fe454bf5765c" (UID: "36ce1c34-d41a-4d68-b7d0-fe454bf5765c"). InnerVolumeSpecName "kube-api-access-q7vpf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 16:58:35.110957 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:58:35.110925 2584 generic.go:356] "Generic (PLEG): container finished" podID="36ce1c34-d41a-4d68-b7d0-fe454bf5765c" containerID="6c510bd4d7b963807cd1de66198cf5db2ba9d0656c719a80f6da89f68884487d" exitCode=0 Jul 06 16:58:35.111141 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:58:35.110973 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-b-7ed8f1-74d47988bf-2zcxj" Jul 06 16:58:35.111141 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:58:35.111002 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-b-7ed8f1-74d47988bf-2zcxj" event={"ID":"36ce1c34-d41a-4d68-b7d0-fe454bf5765c","Type":"ContainerDied","Data":"6c510bd4d7b963807cd1de66198cf5db2ba9d0656c719a80f6da89f68884487d"} Jul 06 16:58:35.111141 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:58:35.111043 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-b-7ed8f1-74d47988bf-2zcxj" event={"ID":"36ce1c34-d41a-4d68-b7d0-fe454bf5765c","Type":"ContainerDied","Data":"1cf8d521bd7ecdba7925143f735b2e7fb646135e89d58f71f66780a39eb8f267"} Jul 06 16:58:35.111141 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:58:35.111064 2584 scope.go:117] "RemoveContainer" containerID="6c510bd4d7b963807cd1de66198cf5db2ba9d0656c719a80f6da89f68884487d" Jul 06 16:58:35.120136 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:58:35.120116 2584 scope.go:117] "RemoveContainer" containerID="6c510bd4d7b963807cd1de66198cf5db2ba9d0656c719a80f6da89f68884487d" Jul 06 16:58:35.120406 ip-10-0-128-22 kubenswrapper[2584]: E0706 16:58:35.120390 2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6c510bd4d7b963807cd1de66198cf5db2ba9d0656c719a80f6da89f68884487d\": container with ID starting with 6c510bd4d7b963807cd1de66198cf5db2ba9d0656c719a80f6da89f68884487d not found: ID does not exist" containerID="6c510bd4d7b963807cd1de66198cf5db2ba9d0656c719a80f6da89f68884487d" Jul 06 16:58:35.120452 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:58:35.120413 2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6c510bd4d7b963807cd1de66198cf5db2ba9d0656c719a80f6da89f68884487d"} err="failed to get container status \"6c510bd4d7b963807cd1de66198cf5db2ba9d0656c719a80f6da89f68884487d\": rpc error: code = NotFound desc = could not find container \"6c510bd4d7b963807cd1de66198cf5db2ba9d0656c719a80f6da89f68884487d\": container with ID starting with 6c510bd4d7b963807cd1de66198cf5db2ba9d0656c719a80f6da89f68884487d not found: ID does not exist" Jul 06 16:58:35.133822 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:58:35.133794 2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q7vpf\" (UniqueName: \"kubernetes.io/projected/36ce1c34-d41a-4d68-b7d0-fe454bf5765c-kube-api-access-q7vpf\") on node \"ip-10-0-128-22.ec2.internal\" DevicePath \"\"" Jul 06 16:58:35.133822 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:58:35.133818 2584 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/36ce1c34-d41a-4d68-b7d0-fe454bf5765c-openshift-service-ca\") on node \"ip-10-0-128-22.ec2.internal\" DevicePath \"\"" Jul 06 16:58:35.133822 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:58:35.133827 2584 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/36ce1c34-d41a-4d68-b7d0-fe454bf5765c-maas-api-tls\") on node \"ip-10-0-128-22.ec2.internal\" DevicePath \"\"" Jul 06 16:58:35.134364 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:58:35.134341 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-api-b-7ed8f1-74d47988bf-2zcxj"] Jul 06 16:58:35.140494 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:58:35.140464 2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-api-b-7ed8f1-74d47988bf-2zcxj"] Jul 06 16:58:36.124230 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:58:36.124188 2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="36ce1c34-d41a-4d68-b7d0-fe454bf5765c" path="/var/lib/kubelet/pods/36ce1c34-d41a-4d68-b7d0-fe454bf5765c/volumes" Jul 06 16:58:40.627440 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:58:40.627405 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:58:40.726945 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:58:40.726913 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:58:40.891055 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:58:40.890905 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:58:41.126619 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:58:41.126586 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:58:41.449119 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:58:41.449083 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:58:41.791390 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:58:41.791306 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:58:42.480896 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:58:42.480853 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:58:44.940871 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:58:44.940827 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:58:45.032260 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:58:45.032218 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:58:45.034451 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:58:45.034422 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-shared-a-0e87f3-5c78499d97-b97hb"] Jul 06 16:58:45.034879 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:58:45.034865 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="36ce1c34-d41a-4d68-b7d0-fe454bf5765c" containerName="maas-api" Jul 06 16:58:45.034929 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:58:45.034881 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="36ce1c34-d41a-4d68-b7d0-fe454bf5765c" containerName="maas-api" Jul 06 16:58:45.034929 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:58:45.034901 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="084009d6-580c-4bcc-908e-479015fee5a6" containerName="maas-api" Jul 06 16:58:45.034929 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:58:45.034908 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="084009d6-580c-4bcc-908e-479015fee5a6" containerName="maas-api" Jul 06 16:58:45.035026 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:58:45.034966 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="084009d6-580c-4bcc-908e-479015fee5a6" containerName="maas-api" Jul 06 16:58:45.035026 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:58:45.034978 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="36ce1c34-d41a-4d68-b7d0-fe454bf5765c" containerName="maas-api" Jul 06 16:58:45.038030 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:58:45.038008 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-a-0e87f3-5c78499d97-b97hb" Jul 06 16:58:45.040775 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:58:45.040710 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-shared-a-0e87f3-serving-cert\"" Jul 06 16:58:45.051212 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:58:45.051185 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-shared-a-0e87f3-5c78499d97-b97hb"] Jul 06 16:58:45.093958 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:58:45.093920 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:58:45.124161 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:58:45.124120 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f4f0b180-6ec8-475d-860d-cbdbc173b72e-maas-api-tls\") pod \"maas-api-e2e-shared-a-0e87f3-5c78499d97-b97hb\" (UID: \"f4f0b180-6ec8-475d-860d-cbdbc173b72e\") " pod="opendatahub/maas-api-e2e-shared-a-0e87f3-5c78499d97-b97hb" Jul 06 16:58:45.124328 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:58:45.124205 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lgzmx\" (UniqueName: \"kubernetes.io/projected/f4f0b180-6ec8-475d-860d-cbdbc173b72e-kube-api-access-lgzmx\") pod \"maas-api-e2e-shared-a-0e87f3-5c78499d97-b97hb\" (UID: \"f4f0b180-6ec8-475d-860d-cbdbc173b72e\") " pod="opendatahub/maas-api-e2e-shared-a-0e87f3-5c78499d97-b97hb" Jul 06 16:58:45.124328 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:58:45.124267 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f4f0b180-6ec8-475d-860d-cbdbc173b72e-openshift-service-ca\") pod \"maas-api-e2e-shared-a-0e87f3-5c78499d97-b97hb\" (UID: \"f4f0b180-6ec8-475d-860d-cbdbc173b72e\") " pod="opendatahub/maas-api-e2e-shared-a-0e87f3-5c78499d97-b97hb" Jul 06 16:58:45.225841 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:58:45.225746 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lgzmx\" (UniqueName: \"kubernetes.io/projected/f4f0b180-6ec8-475d-860d-cbdbc173b72e-kube-api-access-lgzmx\") pod \"maas-api-e2e-shared-a-0e87f3-5c78499d97-b97hb\" (UID: \"f4f0b180-6ec8-475d-860d-cbdbc173b72e\") " pod="opendatahub/maas-api-e2e-shared-a-0e87f3-5c78499d97-b97hb" Jul 06 16:58:45.225841 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:58:45.225809 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f4f0b180-6ec8-475d-860d-cbdbc173b72e-openshift-service-ca\") pod \"maas-api-e2e-shared-a-0e87f3-5c78499d97-b97hb\" (UID: \"f4f0b180-6ec8-475d-860d-cbdbc173b72e\") " pod="opendatahub/maas-api-e2e-shared-a-0e87f3-5c78499d97-b97hb" Jul 06 16:58:45.226032 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:58:45.225903 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f4f0b180-6ec8-475d-860d-cbdbc173b72e-maas-api-tls\") pod \"maas-api-e2e-shared-a-0e87f3-5c78499d97-b97hb\" (UID: \"f4f0b180-6ec8-475d-860d-cbdbc173b72e\") " pod="opendatahub/maas-api-e2e-shared-a-0e87f3-5c78499d97-b97hb" Jul 06 16:58:45.226032 ip-10-0-128-22 kubenswrapper[2584]: E0706 16:58:45.226022 2584 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-shared-a-0e87f3-serving-cert: secret "maas-api-e2e-shared-a-0e87f3-serving-cert" not found Jul 06 16:58:45.226100 ip-10-0-128-22 kubenswrapper[2584]: E0706 16:58:45.226091 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f4f0b180-6ec8-475d-860d-cbdbc173b72e-maas-api-tls podName:f4f0b180-6ec8-475d-860d-cbdbc173b72e nodeName:}" failed. No retries permitted until 2026-07-06 16:58:45.726072152 +0000 UTC m=+2076.340212251 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/f4f0b180-6ec8-475d-860d-cbdbc173b72e-maas-api-tls") pod "maas-api-e2e-shared-a-0e87f3-5c78499d97-b97hb" (UID: "f4f0b180-6ec8-475d-860d-cbdbc173b72e") : secret "maas-api-e2e-shared-a-0e87f3-serving-cert" not found Jul 06 16:58:45.226485 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:58:45.226467 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f4f0b180-6ec8-475d-860d-cbdbc173b72e-openshift-service-ca\") pod \"maas-api-e2e-shared-a-0e87f3-5c78499d97-b97hb\" (UID: \"f4f0b180-6ec8-475d-860d-cbdbc173b72e\") " pod="opendatahub/maas-api-e2e-shared-a-0e87f3-5c78499d97-b97hb" Jul 06 16:58:45.235395 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:58:45.235367 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lgzmx\" (UniqueName: \"kubernetes.io/projected/f4f0b180-6ec8-475d-860d-cbdbc173b72e-kube-api-access-lgzmx\") pod \"maas-api-e2e-shared-a-0e87f3-5c78499d97-b97hb\" (UID: \"f4f0b180-6ec8-475d-860d-cbdbc173b72e\") " pod="opendatahub/maas-api-e2e-shared-a-0e87f3-5c78499d97-b97hb" Jul 06 16:58:45.304122 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:58:45.304085 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:58:45.615072 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:58:45.614881 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:58:45.715439 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:58:45.715409 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:58:45.730258 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:58:45.730235 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f4f0b180-6ec8-475d-860d-cbdbc173b72e-maas-api-tls\") pod \"maas-api-e2e-shared-a-0e87f3-5c78499d97-b97hb\" (UID: \"f4f0b180-6ec8-475d-860d-cbdbc173b72e\") " pod="opendatahub/maas-api-e2e-shared-a-0e87f3-5c78499d97-b97hb" Jul 06 16:58:45.732820 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:58:45.732796 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f4f0b180-6ec8-475d-860d-cbdbc173b72e-maas-api-tls\") pod \"maas-api-e2e-shared-a-0e87f3-5c78499d97-b97hb\" (UID: \"f4f0b180-6ec8-475d-860d-cbdbc173b72e\") " pod="opendatahub/maas-api-e2e-shared-a-0e87f3-5c78499d97-b97hb" Jul 06 16:58:45.950903 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:58:45.950865 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-a-0e87f3-5c78499d97-b97hb" Jul 06 16:58:46.096257 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:58:46.096230 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-shared-a-0e87f3-5c78499d97-b97hb"] Jul 06 16:58:46.098229 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:58:46.098204 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf4f0b180_6ec8_475d_860d_cbdbc173b72e.slice/crio-ffea923b0c70775d50f128a96535955e98e85998a6388c922778ad05850581a7 WatchSource:0}: Error finding container ffea923b0c70775d50f128a96535955e98e85998a6388c922778ad05850581a7: Status 404 returned error can't find the container with id ffea923b0c70775d50f128a96535955e98e85998a6388c922778ad05850581a7 Jul 06 16:58:46.175551 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:58:46.175508 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-a-0e87f3-5c78499d97-b97hb" event={"ID":"f4f0b180-6ec8-475d-860d-cbdbc173b72e","Type":"ContainerStarted","Data":"ffea923b0c70775d50f128a96535955e98e85998a6388c922778ad05850581a7"} Jul 06 16:58:47.182708 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:58:47.182675 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-a-0e87f3-5c78499d97-b97hb" event={"ID":"f4f0b180-6ec8-475d-860d-cbdbc173b72e","Type":"ContainerStarted","Data":"e95fc2970f42bd791fe94c2614197aa95762867af5f4d000ae7d153239b0d7e7"} Jul 06 16:58:47.183173 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:58:47.182809 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-shared-a-0e87f3-5c78499d97-b97hb" Jul 06 16:58:47.208189 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:58:47.208130 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-shared-a-0e87f3-5c78499d97-b97hb" podStartSLOduration=1.722022415 podStartE2EDuration="2.208109813s" totalImagesPullingTime="486.087398ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 16:58:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 16:58:47.207140726 +0000 UTC m=+2077.821280848" watchObservedRunningTime="2026-07-06 16:58:47.208109813 +0000 UTC m=+2077.822249933" Jul 06 16:58:49.473066 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:58:49.473030 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:58:49.751865 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:58:49.751777 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:58:50.069752 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:58:50.069650 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:58:50.482129 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:58:50.482092 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:58:53.193692 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:58:53.193662 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-shared-a-0e87f3-5c78499d97-b97hb" Jul 06 16:58:53.242632 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:58:53.242598 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:58:53.345925 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:58:53.345887 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:58:53.356876 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:58:53.356840 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-shared-b-327802-64c5b78555-8rrpz"] Jul 06 16:58:53.360495 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:58:53.360466 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-b-327802-64c5b78555-8rrpz" Jul 06 16:58:53.363101 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:58:53.363079 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-shared-b-327802-serving-cert\"" Jul 06 16:58:53.369098 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:58:53.368820 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-shared-b-327802-64c5b78555-8rrpz"] Jul 06 16:58:53.402651 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:58:53.402612 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e5fa08e9-0846-49ac-9557-a22d4205a3f6-openshift-service-ca\") pod \"maas-api-e2e-shared-b-327802-64c5b78555-8rrpz\" (UID: \"e5fa08e9-0846-49ac-9557-a22d4205a3f6\") " pod="opendatahub/maas-api-e2e-shared-b-327802-64c5b78555-8rrpz" Jul 06 16:58:53.402864 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:58:53.402702 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qz8q6\" (UniqueName: \"kubernetes.io/projected/e5fa08e9-0846-49ac-9557-a22d4205a3f6-kube-api-access-qz8q6\") pod \"maas-api-e2e-shared-b-327802-64c5b78555-8rrpz\" (UID: \"e5fa08e9-0846-49ac-9557-a22d4205a3f6\") " pod="opendatahub/maas-api-e2e-shared-b-327802-64c5b78555-8rrpz" Jul 06 16:58:53.402864 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:58:53.402754 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e5fa08e9-0846-49ac-9557-a22d4205a3f6-maas-api-tls\") pod \"maas-api-e2e-shared-b-327802-64c5b78555-8rrpz\" (UID: \"e5fa08e9-0846-49ac-9557-a22d4205a3f6\") " pod="opendatahub/maas-api-e2e-shared-b-327802-64c5b78555-8rrpz" Jul 06 16:58:53.443159 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:58:53.443121 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:58:53.503485 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:58:53.503397 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e5fa08e9-0846-49ac-9557-a22d4205a3f6-openshift-service-ca\") pod \"maas-api-e2e-shared-b-327802-64c5b78555-8rrpz\" (UID: \"e5fa08e9-0846-49ac-9557-a22d4205a3f6\") " pod="opendatahub/maas-api-e2e-shared-b-327802-64c5b78555-8rrpz" Jul 06 16:58:53.503485 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:58:53.503472 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qz8q6\" (UniqueName: \"kubernetes.io/projected/e5fa08e9-0846-49ac-9557-a22d4205a3f6-kube-api-access-qz8q6\") pod \"maas-api-e2e-shared-b-327802-64c5b78555-8rrpz\" (UID: \"e5fa08e9-0846-49ac-9557-a22d4205a3f6\") " pod="opendatahub/maas-api-e2e-shared-b-327802-64c5b78555-8rrpz" Jul 06 16:58:53.503748 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:58:53.503494 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e5fa08e9-0846-49ac-9557-a22d4205a3f6-maas-api-tls\") pod \"maas-api-e2e-shared-b-327802-64c5b78555-8rrpz\" (UID: \"e5fa08e9-0846-49ac-9557-a22d4205a3f6\") " pod="opendatahub/maas-api-e2e-shared-b-327802-64c5b78555-8rrpz" Jul 06 16:58:53.503748 ip-10-0-128-22 kubenswrapper[2584]: E0706 16:58:53.503609 2584 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-shared-b-327802-serving-cert: secret "maas-api-e2e-shared-b-327802-serving-cert" not found Jul 06 16:58:53.503748 ip-10-0-128-22 kubenswrapper[2584]: E0706 16:58:53.503664 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e5fa08e9-0846-49ac-9557-a22d4205a3f6-maas-api-tls podName:e5fa08e9-0846-49ac-9557-a22d4205a3f6 nodeName:}" failed. No retries permitted until 2026-07-06 16:58:54.003647822 +0000 UTC m=+2084.617787922 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/e5fa08e9-0846-49ac-9557-a22d4205a3f6-maas-api-tls") pod "maas-api-e2e-shared-b-327802-64c5b78555-8rrpz" (UID: "e5fa08e9-0846-49ac-9557-a22d4205a3f6") : secret "maas-api-e2e-shared-b-327802-serving-cert" not found Jul 06 16:58:53.504046 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:58:53.504026 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e5fa08e9-0846-49ac-9557-a22d4205a3f6-openshift-service-ca\") pod \"maas-api-e2e-shared-b-327802-64c5b78555-8rrpz\" (UID: \"e5fa08e9-0846-49ac-9557-a22d4205a3f6\") " pod="opendatahub/maas-api-e2e-shared-b-327802-64c5b78555-8rrpz" Jul 06 16:58:53.515183 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:58:53.515155 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qz8q6\" (UniqueName: \"kubernetes.io/projected/e5fa08e9-0846-49ac-9557-a22d4205a3f6-kube-api-access-qz8q6\") pod \"maas-api-e2e-shared-b-327802-64c5b78555-8rrpz\" (UID: \"e5fa08e9-0846-49ac-9557-a22d4205a3f6\") " pod="opendatahub/maas-api-e2e-shared-b-327802-64c5b78555-8rrpz" Jul 06 16:58:53.902225 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:58:53.902132 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:58:54.008550 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:58:54.008504 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e5fa08e9-0846-49ac-9557-a22d4205a3f6-maas-api-tls\") pod \"maas-api-e2e-shared-b-327802-64c5b78555-8rrpz\" (UID: \"e5fa08e9-0846-49ac-9557-a22d4205a3f6\") " pod="opendatahub/maas-api-e2e-shared-b-327802-64c5b78555-8rrpz" Jul 06 16:58:54.011078 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:58:54.011057 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e5fa08e9-0846-49ac-9557-a22d4205a3f6-maas-api-tls\") pod \"maas-api-e2e-shared-b-327802-64c5b78555-8rrpz\" (UID: \"e5fa08e9-0846-49ac-9557-a22d4205a3f6\") " pod="opendatahub/maas-api-e2e-shared-b-327802-64c5b78555-8rrpz" Jul 06 16:58:54.086219 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:58:54.086188 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:58:54.186710 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:58:54.186672 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:58:54.272764 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:58:54.272663 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-b-327802-64c5b78555-8rrpz" Jul 06 16:58:54.419056 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:58:54.418960 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-shared-b-327802-64c5b78555-8rrpz"] Jul 06 16:58:54.421394 ip-10-0-128-22 kubenswrapper[2584]: W0706 16:58:54.421368 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode5fa08e9_0846_49ac_9557_a22d4205a3f6.slice/crio-15259bc80fddb504f34a6912463aafb8c68f2845395f2e8197e887bc0f7e56ec WatchSource:0}: Error finding container 15259bc80fddb504f34a6912463aafb8c68f2845395f2e8197e887bc0f7e56ec: Status 404 returned error can't find the container with id 15259bc80fddb504f34a6912463aafb8c68f2845395f2e8197e887bc0f7e56ec Jul 06 16:58:55.232170 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:58:55.232130 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-b-327802-64c5b78555-8rrpz" event={"ID":"e5fa08e9-0846-49ac-9557-a22d4205a3f6","Type":"ContainerStarted","Data":"314697ee16bb2b5cb9a792a6aca5f4aea50439cceabcaa746079b949c0923e29"} Jul 06 16:58:55.232377 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:58:55.232218 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-b-327802-64c5b78555-8rrpz" event={"ID":"e5fa08e9-0846-49ac-9557-a22d4205a3f6","Type":"ContainerStarted","Data":"15259bc80fddb504f34a6912463aafb8c68f2845395f2e8197e887bc0f7e56ec"} Jul 06 16:58:55.232377 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:58:55.232253 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-shared-b-327802-64c5b78555-8rrpz" Jul 06 16:58:55.255051 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:58:55.254995 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-shared-b-327802-64c5b78555-8rrpz" podStartSLOduration=1.7735520230000001 podStartE2EDuration="2.254977906s" totalImagesPullingTime="481.425883ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 16:58:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 16:58:55.253920112 +0000 UTC m=+2085.868060235" watchObservedRunningTime="2026-07-06 16:58:55.254977906 +0000 UTC m=+2085.869118027" Jul 06 16:58:57.684299 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:58:57.684260 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:59:01.242381 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:59:01.242347 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-shared-b-327802-64c5b78555-8rrpz" Jul 06 16:59:06.591274 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:59:06.591241 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:59:06.690456 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:59:06.690417 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:59:10.056130 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:59:10.056094 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-294bx_6a8a4796-dbbc-43e2-a459-7acf11f9468d/ovn-acl-logging/0.log" Jul 06 16:59:10.062966 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:59:10.062943 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-294bx_6a8a4796-dbbc-43e2-a459-7acf11f9468d/ovn-acl-logging/0.log" Jul 06 16:59:12.480862 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:59:12.480830 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:59:15.003962 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:59:15.003914 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:59:15.100949 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:59:15.100914 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:59:23.235017 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:59:23.234966 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:59:23.299965 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:59:23.299927 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 16:59:38.496346 ip-10-0-128-22 kubenswrapper[2584]: I0706 16:59:38.496307 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 17:00:00.144927 ip-10-0-128-22 kubenswrapper[2584]: I0706 17:00:00.144887 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-key-cleanup-29722620-nfwfw"] Jul 06 17:00:00.149074 ip-10-0-128-22 kubenswrapper[2584]: I0706 17:00:00.149040 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-key-cleanup-e2e-shared-a-0e87f3-29722620-m2r7x"] Jul 06 17:00:00.149225 ip-10-0-128-22 kubenswrapper[2584]: I0706 17:00:00.149182 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29722620-nfwfw" Jul 06 17:00:00.154802 ip-10-0-128-22 kubenswrapper[2584]: I0706 17:00:00.153682 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-key-cleanup-e2e-shared-b-327802-29722620-pk5fk"] Jul 06 17:00:00.156717 ip-10-0-128-22 kubenswrapper[2584]: I0706 17:00:00.156674 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-e2e-shared-a-0e87f3-29722620-m2r7x" Jul 06 17:00:00.160173 ip-10-0-128-22 kubenswrapper[2584]: I0706 17:00:00.159051 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-e2e-shared-b-327802-29722620-pk5fk" Jul 06 17:00:00.163145 ip-10-0-128-22 kubenswrapper[2584]: I0706 17:00:00.163114 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-e2e-shared-b-327802-29722620-pk5fk"] Jul 06 17:00:00.165688 ip-10-0-128-22 kubenswrapper[2584]: I0706 17:00:00.165656 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-29722620-nfwfw"] Jul 06 17:00:00.168950 ip-10-0-128-22 kubenswrapper[2584]: I0706 17:00:00.168924 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-e2e-shared-a-0e87f3-29722620-m2r7x"] Jul 06 17:00:00.250489 ip-10-0-128-22 kubenswrapper[2584]: I0706 17:00:00.250443 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gd2vv\" (UniqueName: \"kubernetes.io/projected/46735719-f64e-4413-9b2f-f73ec56e7a45-kube-api-access-gd2vv\") pod \"maas-api-key-cleanup-e2e-shared-a-0e87f3-29722620-m2r7x\" (UID: \"46735719-f64e-4413-9b2f-f73ec56e7a45\") " pod="opendatahub/maas-api-key-cleanup-e2e-shared-a-0e87f3-29722620-m2r7x" Jul 06 17:00:00.250683 ip-10-0-128-22 kubenswrapper[2584]: I0706 17:00:00.250497 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p9x7s\" (UniqueName: \"kubernetes.io/projected/36b4e830-9971-4c2f-8d25-7983611571a1-kube-api-access-p9x7s\") pod \"maas-api-key-cleanup-e2e-shared-b-327802-29722620-pk5fk\" (UID: \"36b4e830-9971-4c2f-8d25-7983611571a1\") " pod="opendatahub/maas-api-key-cleanup-e2e-shared-b-327802-29722620-pk5fk" Jul 06 17:00:00.250798 ip-10-0-128-22 kubenswrapper[2584]: I0706 17:00:00.250688 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-45j65\" (UniqueName: \"kubernetes.io/projected/8f441a6c-d8f9-473b-8faf-e19bb46f55db-kube-api-access-45j65\") pod \"maas-api-key-cleanup-29722620-nfwfw\" (UID: \"8f441a6c-d8f9-473b-8faf-e19bb46f55db\") " pod="opendatahub/maas-api-key-cleanup-29722620-nfwfw" Jul 06 17:00:00.351320 ip-10-0-128-22 kubenswrapper[2584]: I0706 17:00:00.351281 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gd2vv\" (UniqueName: \"kubernetes.io/projected/46735719-f64e-4413-9b2f-f73ec56e7a45-kube-api-access-gd2vv\") pod \"maas-api-key-cleanup-e2e-shared-a-0e87f3-29722620-m2r7x\" (UID: \"46735719-f64e-4413-9b2f-f73ec56e7a45\") " pod="opendatahub/maas-api-key-cleanup-e2e-shared-a-0e87f3-29722620-m2r7x" Jul 06 17:00:00.351320 ip-10-0-128-22 kubenswrapper[2584]: I0706 17:00:00.351324 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p9x7s\" (UniqueName: \"kubernetes.io/projected/36b4e830-9971-4c2f-8d25-7983611571a1-kube-api-access-p9x7s\") pod \"maas-api-key-cleanup-e2e-shared-b-327802-29722620-pk5fk\" (UID: \"36b4e830-9971-4c2f-8d25-7983611571a1\") " pod="opendatahub/maas-api-key-cleanup-e2e-shared-b-327802-29722620-pk5fk" Jul 06 17:00:00.351596 ip-10-0-128-22 kubenswrapper[2584]: I0706 17:00:00.351381 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-45j65\" (UniqueName: \"kubernetes.io/projected/8f441a6c-d8f9-473b-8faf-e19bb46f55db-kube-api-access-45j65\") pod \"maas-api-key-cleanup-29722620-nfwfw\" (UID: \"8f441a6c-d8f9-473b-8faf-e19bb46f55db\") " pod="opendatahub/maas-api-key-cleanup-29722620-nfwfw" Jul 06 17:00:00.362081 ip-10-0-128-22 kubenswrapper[2584]: I0706 17:00:00.362043 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p9x7s\" (UniqueName: \"kubernetes.io/projected/36b4e830-9971-4c2f-8d25-7983611571a1-kube-api-access-p9x7s\") pod \"maas-api-key-cleanup-e2e-shared-b-327802-29722620-pk5fk\" (UID: \"36b4e830-9971-4c2f-8d25-7983611571a1\") " pod="opendatahub/maas-api-key-cleanup-e2e-shared-b-327802-29722620-pk5fk" Jul 06 17:00:00.362081 ip-10-0-128-22 kubenswrapper[2584]: I0706 17:00:00.362043 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gd2vv\" (UniqueName: \"kubernetes.io/projected/46735719-f64e-4413-9b2f-f73ec56e7a45-kube-api-access-gd2vv\") pod \"maas-api-key-cleanup-e2e-shared-a-0e87f3-29722620-m2r7x\" (UID: \"46735719-f64e-4413-9b2f-f73ec56e7a45\") " pod="opendatahub/maas-api-key-cleanup-e2e-shared-a-0e87f3-29722620-m2r7x" Jul 06 17:00:00.362263 ip-10-0-128-22 kubenswrapper[2584]: I0706 17:00:00.362091 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-45j65\" (UniqueName: \"kubernetes.io/projected/8f441a6c-d8f9-473b-8faf-e19bb46f55db-kube-api-access-45j65\") pod \"maas-api-key-cleanup-29722620-nfwfw\" (UID: \"8f441a6c-d8f9-473b-8faf-e19bb46f55db\") " pod="opendatahub/maas-api-key-cleanup-29722620-nfwfw" Jul 06 17:00:00.466998 ip-10-0-128-22 kubenswrapper[2584]: I0706 17:00:00.466956 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29722620-nfwfw" Jul 06 17:00:00.478635 ip-10-0-128-22 kubenswrapper[2584]: I0706 17:00:00.478605 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-e2e-shared-a-0e87f3-29722620-m2r7x" Jul 06 17:00:00.484066 ip-10-0-128-22 kubenswrapper[2584]: I0706 17:00:00.484038 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-e2e-shared-b-327802-29722620-pk5fk" Jul 06 17:00:00.661404 ip-10-0-128-22 kubenswrapper[2584]: I0706 17:00:00.661316 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-29722620-nfwfw"] Jul 06 17:00:00.665966 ip-10-0-128-22 kubenswrapper[2584]: W0706 17:00:00.665939 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8f441a6c_d8f9_473b_8faf_e19bb46f55db.slice/crio-387eac0958843c3709982b3ee870c65068b94eb6b877032b93b47fe2bfc048fb WatchSource:0}: Error finding container 387eac0958843c3709982b3ee870c65068b94eb6b877032b93b47fe2bfc048fb: Status 404 returned error can't find the container with id 387eac0958843c3709982b3ee870c65068b94eb6b877032b93b47fe2bfc048fb Jul 06 17:00:00.692911 ip-10-0-128-22 kubenswrapper[2584]: I0706 17:00:00.692878 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-e2e-shared-b-327802-29722620-pk5fk"] Jul 06 17:00:00.694553 ip-10-0-128-22 kubenswrapper[2584]: W0706 17:00:00.694524 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod36b4e830_9971_4c2f_8d25_7983611571a1.slice/crio-c283e62e8fee49f6c4d73a526f39becd1649fa622ec3dd0e1da7413fe4088e7f WatchSource:0}: Error finding container c283e62e8fee49f6c4d73a526f39becd1649fa622ec3dd0e1da7413fe4088e7f: Status 404 returned error can't find the container with id c283e62e8fee49f6c4d73a526f39becd1649fa622ec3dd0e1da7413fe4088e7f Jul 06 17:00:00.724633 ip-10-0-128-22 kubenswrapper[2584]: I0706 17:00:00.724552 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-e2e-shared-a-0e87f3-29722620-m2r7x"] Jul 06 17:00:00.727912 ip-10-0-128-22 kubenswrapper[2584]: W0706 17:00:00.727882 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod46735719_f64e_4413_9b2f_f73ec56e7a45.slice/crio-38010916d2b0ef93e16109cf776605961b08583fe6690b0c9c718d75f616d4be WatchSource:0}: Error finding container 38010916d2b0ef93e16109cf776605961b08583fe6690b0c9c718d75f616d4be: Status 404 returned error can't find the container with id 38010916d2b0ef93e16109cf776605961b08583fe6690b0c9c718d75f616d4be Jul 06 17:00:01.589306 ip-10-0-128-22 kubenswrapper[2584]: I0706 17:00:01.589210 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-e2e-shared-a-0e87f3-29722620-m2r7x" event={"ID":"46735719-f64e-4413-9b2f-f73ec56e7a45","Type":"ContainerStarted","Data":"38010916d2b0ef93e16109cf776605961b08583fe6690b0c9c718d75f616d4be"} Jul 06 17:00:01.590614 ip-10-0-128-22 kubenswrapper[2584]: I0706 17:00:01.590586 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-e2e-shared-b-327802-29722620-pk5fk" event={"ID":"36b4e830-9971-4c2f-8d25-7983611571a1","Type":"ContainerStarted","Data":"c283e62e8fee49f6c4d73a526f39becd1649fa622ec3dd0e1da7413fe4088e7f"} Jul 06 17:00:01.591926 ip-10-0-128-22 kubenswrapper[2584]: I0706 17:00:01.591902 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29722620-nfwfw" event={"ID":"8f441a6c-d8f9-473b-8faf-e19bb46f55db","Type":"ContainerStarted","Data":"387eac0958843c3709982b3ee870c65068b94eb6b877032b93b47fe2bfc048fb"} Jul 06 17:00:04.280005 ip-10-0-128-22 kubenswrapper[2584]: I0706 17:00:04.279966 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 17:00:04.381388 ip-10-0-128-22 kubenswrapper[2584]: I0706 17:00:04.381341 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 17:00:05.618163 ip-10-0-128-22 kubenswrapper[2584]: I0706 17:00:05.618127 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29722620-nfwfw" event={"ID":"8f441a6c-d8f9-473b-8faf-e19bb46f55db","Type":"ContainerStarted","Data":"796d91681c767da641477ac8569eacdb14b837d6d7bb783715394c2f7c578519"} Jul 06 17:00:05.642865 ip-10-0-128-22 kubenswrapper[2584]: I0706 17:00:05.642791 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-key-cleanup-29722620-nfwfw" podStartSLOduration=1.536485217 podStartE2EDuration="5.642767506s" totalImagesPullingTime="4.106282289s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 17:00:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 17:00:05.640806496 +0000 UTC m=+2156.254946608" watchObservedRunningTime="2026-07-06 17:00:05.642767506 +0000 UTC m=+2156.256907630" Jul 06 17:00:06.624297 ip-10-0-128-22 kubenswrapper[2584]: I0706 17:00:06.624261 2584 generic.go:356] "Generic (PLEG): container finished" podID="8f441a6c-d8f9-473b-8faf-e19bb46f55db" containerID="796d91681c767da641477ac8569eacdb14b837d6d7bb783715394c2f7c578519" exitCode=0 Jul 06 17:00:06.624717 ip-10-0-128-22 kubenswrapper[2584]: I0706 17:00:06.624357 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29722620-nfwfw" event={"ID":"8f441a6c-d8f9-473b-8faf-e19bb46f55db","Type":"ContainerDied","Data":"796d91681c767da641477ac8569eacdb14b837d6d7bb783715394c2f7c578519"} Jul 06 17:00:07.769546 ip-10-0-128-22 kubenswrapper[2584]: I0706 17:00:07.769511 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29722620-nfwfw" Jul 06 17:00:07.831801 ip-10-0-128-22 kubenswrapper[2584]: I0706 17:00:07.831765 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-45j65\" (UniqueName: \"kubernetes.io/projected/8f441a6c-d8f9-473b-8faf-e19bb46f55db-kube-api-access-45j65\") pod \"8f441a6c-d8f9-473b-8faf-e19bb46f55db\" (UID: \"8f441a6c-d8f9-473b-8faf-e19bb46f55db\") " Jul 06 17:00:07.833863 ip-10-0-128-22 kubenswrapper[2584]: I0706 17:00:07.833839 2584 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8f441a6c-d8f9-473b-8faf-e19bb46f55db-kube-api-access-45j65" (OuterVolumeSpecName: "kube-api-access-45j65") pod "8f441a6c-d8f9-473b-8faf-e19bb46f55db" (UID: "8f441a6c-d8f9-473b-8faf-e19bb46f55db"). InnerVolumeSpecName "kube-api-access-45j65". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 17:00:07.933334 ip-10-0-128-22 kubenswrapper[2584]: I0706 17:00:07.933277 2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-45j65\" (UniqueName: \"kubernetes.io/projected/8f441a6c-d8f9-473b-8faf-e19bb46f55db-kube-api-access-45j65\") on node \"ip-10-0-128-22.ec2.internal\" DevicePath \"\"" Jul 06 17:00:08.635912 ip-10-0-128-22 kubenswrapper[2584]: I0706 17:00:08.635876 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29722620-nfwfw" event={"ID":"8f441a6c-d8f9-473b-8faf-e19bb46f55db","Type":"ContainerDied","Data":"387eac0958843c3709982b3ee870c65068b94eb6b877032b93b47fe2bfc048fb"} Jul 06 17:00:08.635912 ip-10-0-128-22 kubenswrapper[2584]: I0706 17:00:08.635911 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29722620-nfwfw" Jul 06 17:00:08.636155 ip-10-0-128-22 kubenswrapper[2584]: I0706 17:00:08.635925 2584 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="387eac0958843c3709982b3ee870c65068b94eb6b877032b93b47fe2bfc048fb" Jul 06 17:00:10.648968 ip-10-0-128-22 kubenswrapper[2584]: I0706 17:00:10.648921 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-e2e-shared-b-327802-29722620-pk5fk" event={"ID":"36b4e830-9971-4c2f-8d25-7983611571a1","Type":"ContainerStarted","Data":"a891644a929de841c6e9e0d91fe9e7203cd033a84e459b637fc13da132f769d9"} Jul 06 17:00:10.677492 ip-10-0-128-22 kubenswrapper[2584]: I0706 17:00:10.677436 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-key-cleanup-e2e-shared-b-327802-29722620-pk5fk" podStartSLOduration=1.181496487 podStartE2EDuration="10.677419302s" totalImagesPullingTime="9.495922815s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 17:00:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 17:00:10.676908363 +0000 UTC m=+2161.291048486" watchObservedRunningTime="2026-07-06 17:00:10.677419302 +0000 UTC m=+2161.291559422" Jul 06 17:00:11.655891 ip-10-0-128-22 kubenswrapper[2584]: I0706 17:00:11.655850 2584 generic.go:356] "Generic (PLEG): container finished" podID="36b4e830-9971-4c2f-8d25-7983611571a1" containerID="a891644a929de841c6e9e0d91fe9e7203cd033a84e459b637fc13da132f769d9" exitCode=0 Jul 06 17:00:11.656277 ip-10-0-128-22 kubenswrapper[2584]: I0706 17:00:11.655938 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-e2e-shared-b-327802-29722620-pk5fk" event={"ID":"36b4e830-9971-4c2f-8d25-7983611571a1","Type":"ContainerDied","Data":"a891644a929de841c6e9e0d91fe9e7203cd033a84e459b637fc13da132f769d9"} Jul 06 17:00:12.804049 ip-10-0-128-22 kubenswrapper[2584]: I0706 17:00:12.804023 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-e2e-shared-b-327802-29722620-pk5fk" Jul 06 17:00:12.877048 ip-10-0-128-22 kubenswrapper[2584]: I0706 17:00:12.877013 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p9x7s\" (UniqueName: \"kubernetes.io/projected/36b4e830-9971-4c2f-8d25-7983611571a1-kube-api-access-p9x7s\") pod \"36b4e830-9971-4c2f-8d25-7983611571a1\" (UID: \"36b4e830-9971-4c2f-8d25-7983611571a1\") " Jul 06 17:00:12.879132 ip-10-0-128-22 kubenswrapper[2584]: I0706 17:00:12.879101 2584 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/36b4e830-9971-4c2f-8d25-7983611571a1-kube-api-access-p9x7s" (OuterVolumeSpecName: "kube-api-access-p9x7s") pod "36b4e830-9971-4c2f-8d25-7983611571a1" (UID: "36b4e830-9971-4c2f-8d25-7983611571a1"). InnerVolumeSpecName "kube-api-access-p9x7s". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 17:00:12.978182 ip-10-0-128-22 kubenswrapper[2584]: I0706 17:00:12.978095 2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p9x7s\" (UniqueName: \"kubernetes.io/projected/36b4e830-9971-4c2f-8d25-7983611571a1-kube-api-access-p9x7s\") on node \"ip-10-0-128-22.ec2.internal\" DevicePath \"\"" Jul 06 17:00:13.667888 ip-10-0-128-22 kubenswrapper[2584]: I0706 17:00:13.667859 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-e2e-shared-b-327802-29722620-pk5fk" Jul 06 17:00:13.668040 ip-10-0-128-22 kubenswrapper[2584]: I0706 17:00:13.667858 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-e2e-shared-b-327802-29722620-pk5fk" event={"ID":"36b4e830-9971-4c2f-8d25-7983611571a1","Type":"ContainerDied","Data":"c283e62e8fee49f6c4d73a526f39becd1649fa622ec3dd0e1da7413fe4088e7f"} Jul 06 17:00:13.668040 ip-10-0-128-22 kubenswrapper[2584]: I0706 17:00:13.667988 2584 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c283e62e8fee49f6c4d73a526f39becd1649fa622ec3dd0e1da7413fe4088e7f" Jul 06 17:00:16.684105 ip-10-0-128-22 kubenswrapper[2584]: I0706 17:00:16.684073 2584 generic.go:356] "Generic (PLEG): container finished" podID="46735719-f64e-4413-9b2f-f73ec56e7a45" containerID="c9382a5f07ee1706f44a7665aaea39717ad4fffaf009700daf50bc99e00371c6" exitCode=0 Jul 06 17:00:16.684521 ip-10-0-128-22 kubenswrapper[2584]: I0706 17:00:16.684115 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-e2e-shared-a-0e87f3-29722620-m2r7x" event={"ID":"46735719-f64e-4413-9b2f-f73ec56e7a45","Type":"ContainerDied","Data":"c9382a5f07ee1706f44a7665aaea39717ad4fffaf009700daf50bc99e00371c6"} Jul 06 17:00:17.826710 ip-10-0-128-22 kubenswrapper[2584]: I0706 17:00:17.826680 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-e2e-shared-a-0e87f3-29722620-m2r7x" Jul 06 17:00:17.930059 ip-10-0-128-22 kubenswrapper[2584]: I0706 17:00:17.930023 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gd2vv\" (UniqueName: \"kubernetes.io/projected/46735719-f64e-4413-9b2f-f73ec56e7a45-kube-api-access-gd2vv\") pod \"46735719-f64e-4413-9b2f-f73ec56e7a45\" (UID: \"46735719-f64e-4413-9b2f-f73ec56e7a45\") " Jul 06 17:00:17.932092 ip-10-0-128-22 kubenswrapper[2584]: I0706 17:00:17.932067 2584 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/46735719-f64e-4413-9b2f-f73ec56e7a45-kube-api-access-gd2vv" (OuterVolumeSpecName: "kube-api-access-gd2vv") pod "46735719-f64e-4413-9b2f-f73ec56e7a45" (UID: "46735719-f64e-4413-9b2f-f73ec56e7a45"). InnerVolumeSpecName "kube-api-access-gd2vv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 17:00:18.030873 ip-10-0-128-22 kubenswrapper[2584]: I0706 17:00:18.030784 2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gd2vv\" (UniqueName: \"kubernetes.io/projected/46735719-f64e-4413-9b2f-f73ec56e7a45-kube-api-access-gd2vv\") on node \"ip-10-0-128-22.ec2.internal\" DevicePath \"\"" Jul 06 17:00:18.695046 ip-10-0-128-22 kubenswrapper[2584]: I0706 17:00:18.695015 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-e2e-shared-a-0e87f3-29722620-m2r7x" Jul 06 17:00:18.695263 ip-10-0-128-22 kubenswrapper[2584]: I0706 17:00:18.695054 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-e2e-shared-a-0e87f3-29722620-m2r7x" event={"ID":"46735719-f64e-4413-9b2f-f73ec56e7a45","Type":"ContainerDied","Data":"38010916d2b0ef93e16109cf776605961b08583fe6690b0c9c718d75f616d4be"} Jul 06 17:00:18.695263 ip-10-0-128-22 kubenswrapper[2584]: I0706 17:00:18.695101 2584 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="38010916d2b0ef93e16109cf776605961b08583fe6690b0c9c718d75f616d4be" Jul 06 17:01:12.033671 ip-10-0-128-22 kubenswrapper[2584]: I0706 17:01:12.033640 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 17:01:43.619194 ip-10-0-128-22 kubenswrapper[2584]: I0706 17:01:43.619099 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 17:01:44.605814 ip-10-0-128-22 kubenswrapper[2584]: I0706 17:01:44.605776 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 17:01:56.412419 ip-10-0-128-22 kubenswrapper[2584]: I0706 17:01:56.412380 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 17:02:16.089504 ip-10-0-128-22 kubenswrapper[2584]: I0706 17:02:16.089470 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 17:02:32.578688 ip-10-0-128-22 kubenswrapper[2584]: I0706 17:02:32.578651 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 17:02:32.671319 ip-10-0-128-22 kubenswrapper[2584]: I0706 17:02:32.671271 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 17:03:19.429596 ip-10-0-128-22 kubenswrapper[2584]: I0706 17:03:19.429560 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 17:03:20.844314 ip-10-0-128-22 kubenswrapper[2584]: I0706 17:03:20.844269 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 17:03:40.312145 ip-10-0-128-22 kubenswrapper[2584]: I0706 17:03:40.312112 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 17:04:07.680703 ip-10-0-128-22 kubenswrapper[2584]: I0706 17:04:07.680657 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 17:04:07.786675 ip-10-0-128-22 kubenswrapper[2584]: I0706 17:04:07.786636 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 17:04:10.092508 ip-10-0-128-22 kubenswrapper[2584]: I0706 17:04:10.092474 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-294bx_6a8a4796-dbbc-43e2-a459-7acf11f9468d/ovn-acl-logging/0.log" Jul 06 17:04:10.097863 ip-10-0-128-22 kubenswrapper[2584]: I0706 17:04:10.097835 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-294bx_6a8a4796-dbbc-43e2-a459-7acf11f9468d/ovn-acl-logging/0.log" Jul 06 17:04:15.874016 ip-10-0-128-22 kubenswrapper[2584]: I0706 17:04:15.873972 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 17:04:15.971936 ip-10-0-128-22 kubenswrapper[2584]: I0706 17:04:15.971898 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 17:04:17.003161 ip-10-0-128-22 kubenswrapper[2584]: I0706 17:04:17.003126 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 17:04:19.570950 ip-10-0-128-22 kubenswrapper[2584]: I0706 17:04:19.570917 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 17:05:02.653025 ip-10-0-128-22 kubenswrapper[2584]: I0706 17:05:02.652989 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 17:05:03.516254 ip-10-0-128-22 kubenswrapper[2584]: I0706 17:05:03.516215 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 17:05:46.903566 ip-10-0-128-22 kubenswrapper[2584]: I0706 17:05:46.903524 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 17:05:47.740209 ip-10-0-128-22 kubenswrapper[2584]: I0706 17:05:47.740172 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 17:05:47.850685 ip-10-0-128-22 kubenswrapper[2584]: I0706 17:05:47.850634 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 17:05:47.856589 ip-10-0-128-22 kubenswrapper[2584]: I0706 17:05:47.856558 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-shared-a-0e87f3-5c78499d97-b97hb"] Jul 06 17:05:47.856889 ip-10-0-128-22 kubenswrapper[2584]: I0706 17:05:47.856859 2584 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-shared-a-0e87f3-5c78499d97-b97hb" podUID="f4f0b180-6ec8-475d-860d-cbdbc173b72e" containerName="maas-api" containerID="cri-o://e95fc2970f42bd791fe94c2614197aa95762867af5f4d000ae7d153239b0d7e7" gracePeriod=30 Jul 06 17:05:47.903184 ip-10-0-128-22 kubenswrapper[2584]: I0706 17:05:47.903151 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-key-cleanup-e2e-shared-a-0e87f3-29722620-m2r7x"] Jul 06 17:05:47.907261 ip-10-0-128-22 kubenswrapper[2584]: I0706 17:05:47.907224 2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-key-cleanup-e2e-shared-a-0e87f3-29722620-m2r7x"] Jul 06 17:05:47.937262 ip-10-0-128-22 kubenswrapper[2584]: I0706 17:05:47.937220 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 17:05:48.106257 ip-10-0-128-22 kubenswrapper[2584]: I0706 17:05:48.106231 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-a-0e87f3-5c78499d97-b97hb" Jul 06 17:05:48.119203 ip-10-0-128-22 kubenswrapper[2584]: I0706 17:05:48.119168 2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="46735719-f64e-4413-9b2f-f73ec56e7a45" path="/var/lib/kubelet/pods/46735719-f64e-4413-9b2f-f73ec56e7a45/volumes" Jul 06 17:05:48.163650 ip-10-0-128-22 kubenswrapper[2584]: I0706 17:05:48.163620 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f4f0b180-6ec8-475d-860d-cbdbc173b72e-maas-api-tls\") pod \"f4f0b180-6ec8-475d-860d-cbdbc173b72e\" (UID: \"f4f0b180-6ec8-475d-860d-cbdbc173b72e\") " Jul 06 17:05:48.163825 ip-10-0-128-22 kubenswrapper[2584]: I0706 17:05:48.163667 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lgzmx\" (UniqueName: \"kubernetes.io/projected/f4f0b180-6ec8-475d-860d-cbdbc173b72e-kube-api-access-lgzmx\") pod \"f4f0b180-6ec8-475d-860d-cbdbc173b72e\" (UID: \"f4f0b180-6ec8-475d-860d-cbdbc173b72e\") " Jul 06 17:05:48.163825 ip-10-0-128-22 kubenswrapper[2584]: I0706 17:05:48.163817 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f4f0b180-6ec8-475d-860d-cbdbc173b72e-openshift-service-ca\") pod \"f4f0b180-6ec8-475d-860d-cbdbc173b72e\" (UID: \"f4f0b180-6ec8-475d-860d-cbdbc173b72e\") " Jul 06 17:05:48.164175 ip-10-0-128-22 kubenswrapper[2584]: I0706 17:05:48.164147 2584 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f4f0b180-6ec8-475d-860d-cbdbc173b72e-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "f4f0b180-6ec8-475d-860d-cbdbc173b72e" (UID: "f4f0b180-6ec8-475d-860d-cbdbc173b72e"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 17:05:48.165948 ip-10-0-128-22 kubenswrapper[2584]: I0706 17:05:48.165915 2584 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f4f0b180-6ec8-475d-860d-cbdbc173b72e-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "f4f0b180-6ec8-475d-860d-cbdbc173b72e" (UID: "f4f0b180-6ec8-475d-860d-cbdbc173b72e"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 17:05:48.166055 ip-10-0-128-22 kubenswrapper[2584]: I0706 17:05:48.166040 2584 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f4f0b180-6ec8-475d-860d-cbdbc173b72e-kube-api-access-lgzmx" (OuterVolumeSpecName: "kube-api-access-lgzmx") pod "f4f0b180-6ec8-475d-860d-cbdbc173b72e" (UID: "f4f0b180-6ec8-475d-860d-cbdbc173b72e"). InnerVolumeSpecName "kube-api-access-lgzmx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 17:05:48.264702 ip-10-0-128-22 kubenswrapper[2584]: I0706 17:05:48.264668 2584 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f4f0b180-6ec8-475d-860d-cbdbc173b72e-openshift-service-ca\") on node \"ip-10-0-128-22.ec2.internal\" DevicePath \"\"" Jul 06 17:05:48.264702 ip-10-0-128-22 kubenswrapper[2584]: I0706 17:05:48.264702 2584 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f4f0b180-6ec8-475d-860d-cbdbc173b72e-maas-api-tls\") on node \"ip-10-0-128-22.ec2.internal\" DevicePath \"\"" Jul 06 17:05:48.264702 ip-10-0-128-22 kubenswrapper[2584]: I0706 17:05:48.264711 2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lgzmx\" (UniqueName: \"kubernetes.io/projected/f4f0b180-6ec8-475d-860d-cbdbc173b72e-kube-api-access-lgzmx\") on node \"ip-10-0-128-22.ec2.internal\" DevicePath \"\"" Jul 06 17:05:48.377177 ip-10-0-128-22 kubenswrapper[2584]: I0706 17:05:48.377093 2584 generic.go:356] "Generic (PLEG): container finished" podID="f4f0b180-6ec8-475d-860d-cbdbc173b72e" containerID="e95fc2970f42bd791fe94c2614197aa95762867af5f4d000ae7d153239b0d7e7" exitCode=0 Jul 06 17:05:48.377177 ip-10-0-128-22 kubenswrapper[2584]: I0706 17:05:48.377143 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-a-0e87f3-5c78499d97-b97hb" Jul 06 17:05:48.377177 ip-10-0-128-22 kubenswrapper[2584]: I0706 17:05:48.377152 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-a-0e87f3-5c78499d97-b97hb" event={"ID":"f4f0b180-6ec8-475d-860d-cbdbc173b72e","Type":"ContainerDied","Data":"e95fc2970f42bd791fe94c2614197aa95762867af5f4d000ae7d153239b0d7e7"} Jul 06 17:05:48.377416 ip-10-0-128-22 kubenswrapper[2584]: I0706 17:05:48.377196 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-a-0e87f3-5c78499d97-b97hb" event={"ID":"f4f0b180-6ec8-475d-860d-cbdbc173b72e","Type":"ContainerDied","Data":"ffea923b0c70775d50f128a96535955e98e85998a6388c922778ad05850581a7"} Jul 06 17:05:48.377416 ip-10-0-128-22 kubenswrapper[2584]: I0706 17:05:48.377218 2584 scope.go:117] "RemoveContainer" containerID="e95fc2970f42bd791fe94c2614197aa95762867af5f4d000ae7d153239b0d7e7" Jul 06 17:05:48.386337 ip-10-0-128-22 kubenswrapper[2584]: I0706 17:05:48.386317 2584 scope.go:117] "RemoveContainer" containerID="e95fc2970f42bd791fe94c2614197aa95762867af5f4d000ae7d153239b0d7e7" Jul 06 17:05:48.386585 ip-10-0-128-22 kubenswrapper[2584]: E0706 17:05:48.386569 2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e95fc2970f42bd791fe94c2614197aa95762867af5f4d000ae7d153239b0d7e7\": container with ID starting with e95fc2970f42bd791fe94c2614197aa95762867af5f4d000ae7d153239b0d7e7 not found: ID does not exist" containerID="e95fc2970f42bd791fe94c2614197aa95762867af5f4d000ae7d153239b0d7e7" Jul 06 17:05:48.386625 ip-10-0-128-22 kubenswrapper[2584]: I0706 17:05:48.386596 2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e95fc2970f42bd791fe94c2614197aa95762867af5f4d000ae7d153239b0d7e7"} err="failed to get container status \"e95fc2970f42bd791fe94c2614197aa95762867af5f4d000ae7d153239b0d7e7\": rpc error: code = NotFound desc = could not find container \"e95fc2970f42bd791fe94c2614197aa95762867af5f4d000ae7d153239b0d7e7\": container with ID starting with e95fc2970f42bd791fe94c2614197aa95762867af5f4d000ae7d153239b0d7e7 not found: ID does not exist" Jul 06 17:05:48.408611 ip-10-0-128-22 kubenswrapper[2584]: I0706 17:05:48.408579 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-shared-a-0e87f3-5c78499d97-b97hb"] Jul 06 17:05:48.413661 ip-10-0-128-22 kubenswrapper[2584]: I0706 17:05:48.413637 2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-shared-a-0e87f3-5c78499d97-b97hb"] Jul 06 17:05:50.119298 ip-10-0-128-22 kubenswrapper[2584]: I0706 17:05:50.119267 2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f4f0b180-6ec8-475d-860d-cbdbc173b72e" path="/var/lib/kubelet/pods/f4f0b180-6ec8-475d-860d-cbdbc173b72e/volumes" Jul 06 17:05:54.303565 ip-10-0-128-22 kubenswrapper[2584]: I0706 17:05:54.303528 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 17:05:54.402334 ip-10-0-128-22 kubenswrapper[2584]: I0706 17:05:54.402300 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 17:05:54.611238 ip-10-0-128-22 kubenswrapper[2584]: I0706 17:05:54.611149 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 17:05:54.813092 ip-10-0-128-22 kubenswrapper[2584]: I0706 17:05:54.813047 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 17:05:55.019189 ip-10-0-128-22 kubenswrapper[2584]: I0706 17:05:55.019159 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 17:05:55.394767 ip-10-0-128-22 kubenswrapper[2584]: I0706 17:05:55.394651 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 17:05:55.563034 ip-10-0-128-22 kubenswrapper[2584]: I0706 17:05:55.563003 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 17:05:55.667150 ip-10-0-128-22 kubenswrapper[2584]: I0706 17:05:55.667119 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 17:05:55.831414 ip-10-0-128-22 kubenswrapper[2584]: I0706 17:05:55.831377 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-shared-b-327802-64c5b78555-8rrpz"] Jul 06 17:05:55.831674 ip-10-0-128-22 kubenswrapper[2584]: I0706 17:05:55.831648 2584 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-shared-b-327802-64c5b78555-8rrpz" podUID="e5fa08e9-0846-49ac-9557-a22d4205a3f6" containerName="maas-api" containerID="cri-o://314697ee16bb2b5cb9a792a6aca5f4aea50439cceabcaa746079b949c0923e29" gracePeriod=30 Jul 06 17:05:55.881828 ip-10-0-128-22 kubenswrapper[2584]: I0706 17:05:55.881789 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 17:05:55.893122 ip-10-0-128-22 kubenswrapper[2584]: I0706 17:05:55.893093 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-key-cleanup-e2e-shared-b-327802-29722620-pk5fk"] Jul 06 17:05:55.899981 ip-10-0-128-22 kubenswrapper[2584]: I0706 17:05:55.899953 2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-key-cleanup-e2e-shared-b-327802-29722620-pk5fk"] Jul 06 17:05:56.083443 ip-10-0-128-22 kubenswrapper[2584]: I0706 17:05:56.083415 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-b-327802-64c5b78555-8rrpz" Jul 06 17:05:56.120062 ip-10-0-128-22 kubenswrapper[2584]: I0706 17:05:56.120031 2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="36b4e830-9971-4c2f-8d25-7983611571a1" path="/var/lib/kubelet/pods/36b4e830-9971-4c2f-8d25-7983611571a1/volumes" Jul 06 17:05:56.242432 ip-10-0-128-22 kubenswrapper[2584]: I0706 17:05:56.242322 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e5fa08e9-0846-49ac-9557-a22d4205a3f6-maas-api-tls\") pod \"e5fa08e9-0846-49ac-9557-a22d4205a3f6\" (UID: \"e5fa08e9-0846-49ac-9557-a22d4205a3f6\") " Jul 06 17:05:56.242432 ip-10-0-128-22 kubenswrapper[2584]: I0706 17:05:56.242376 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e5fa08e9-0846-49ac-9557-a22d4205a3f6-openshift-service-ca\") pod \"e5fa08e9-0846-49ac-9557-a22d4205a3f6\" (UID: \"e5fa08e9-0846-49ac-9557-a22d4205a3f6\") " Jul 06 17:05:56.242699 ip-10-0-128-22 kubenswrapper[2584]: I0706 17:05:56.242462 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qz8q6\" (UniqueName: \"kubernetes.io/projected/e5fa08e9-0846-49ac-9557-a22d4205a3f6-kube-api-access-qz8q6\") pod \"e5fa08e9-0846-49ac-9557-a22d4205a3f6\" (UID: \"e5fa08e9-0846-49ac-9557-a22d4205a3f6\") " Jul 06 17:05:56.242851 ip-10-0-128-22 kubenswrapper[2584]: I0706 17:05:56.242823 2584 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e5fa08e9-0846-49ac-9557-a22d4205a3f6-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "e5fa08e9-0846-49ac-9557-a22d4205a3f6" (UID: "e5fa08e9-0846-49ac-9557-a22d4205a3f6"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 17:05:56.244585 ip-10-0-128-22 kubenswrapper[2584]: I0706 17:05:56.244554 2584 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e5fa08e9-0846-49ac-9557-a22d4205a3f6-kube-api-access-qz8q6" (OuterVolumeSpecName: "kube-api-access-qz8q6") pod "e5fa08e9-0846-49ac-9557-a22d4205a3f6" (UID: "e5fa08e9-0846-49ac-9557-a22d4205a3f6"). InnerVolumeSpecName "kube-api-access-qz8q6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 17:05:56.244690 ip-10-0-128-22 kubenswrapper[2584]: I0706 17:05:56.244589 2584 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e5fa08e9-0846-49ac-9557-a22d4205a3f6-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "e5fa08e9-0846-49ac-9557-a22d4205a3f6" (UID: "e5fa08e9-0846-49ac-9557-a22d4205a3f6"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 17:05:56.343446 ip-10-0-128-22 kubenswrapper[2584]: I0706 17:05:56.343399 2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qz8q6\" (UniqueName: \"kubernetes.io/projected/e5fa08e9-0846-49ac-9557-a22d4205a3f6-kube-api-access-qz8q6\") on node \"ip-10-0-128-22.ec2.internal\" DevicePath \"\"" Jul 06 17:05:56.343446 ip-10-0-128-22 kubenswrapper[2584]: I0706 17:05:56.343440 2584 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e5fa08e9-0846-49ac-9557-a22d4205a3f6-maas-api-tls\") on node \"ip-10-0-128-22.ec2.internal\" DevicePath \"\"" Jul 06 17:05:56.343446 ip-10-0-128-22 kubenswrapper[2584]: I0706 17:05:56.343450 2584 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e5fa08e9-0846-49ac-9557-a22d4205a3f6-openshift-service-ca\") on node \"ip-10-0-128-22.ec2.internal\" DevicePath \"\"" Jul 06 17:05:56.416637 ip-10-0-128-22 kubenswrapper[2584]: I0706 17:05:56.416605 2584 generic.go:356] "Generic (PLEG): container finished" podID="e5fa08e9-0846-49ac-9557-a22d4205a3f6" containerID="314697ee16bb2b5cb9a792a6aca5f4aea50439cceabcaa746079b949c0923e29" exitCode=0 Jul 06 17:05:56.417112 ip-10-0-128-22 kubenswrapper[2584]: I0706 17:05:56.416650 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-b-327802-64c5b78555-8rrpz" Jul 06 17:05:56.417112 ip-10-0-128-22 kubenswrapper[2584]: I0706 17:05:56.416674 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-b-327802-64c5b78555-8rrpz" event={"ID":"e5fa08e9-0846-49ac-9557-a22d4205a3f6","Type":"ContainerDied","Data":"314697ee16bb2b5cb9a792a6aca5f4aea50439cceabcaa746079b949c0923e29"} Jul 06 17:05:56.417112 ip-10-0-128-22 kubenswrapper[2584]: I0706 17:05:56.416717 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-b-327802-64c5b78555-8rrpz" event={"ID":"e5fa08e9-0846-49ac-9557-a22d4205a3f6","Type":"ContainerDied","Data":"15259bc80fddb504f34a6912463aafb8c68f2845395f2e8197e887bc0f7e56ec"} Jul 06 17:05:56.417112 ip-10-0-128-22 kubenswrapper[2584]: I0706 17:05:56.416751 2584 scope.go:117] "RemoveContainer" containerID="314697ee16bb2b5cb9a792a6aca5f4aea50439cceabcaa746079b949c0923e29" Jul 06 17:05:56.426556 ip-10-0-128-22 kubenswrapper[2584]: I0706 17:05:56.426537 2584 scope.go:117] "RemoveContainer" containerID="314697ee16bb2b5cb9a792a6aca5f4aea50439cceabcaa746079b949c0923e29" Jul 06 17:05:56.426839 ip-10-0-128-22 kubenswrapper[2584]: E0706 17:05:56.426820 2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"314697ee16bb2b5cb9a792a6aca5f4aea50439cceabcaa746079b949c0923e29\": container with ID starting with 314697ee16bb2b5cb9a792a6aca5f4aea50439cceabcaa746079b949c0923e29 not found: ID does not exist" containerID="314697ee16bb2b5cb9a792a6aca5f4aea50439cceabcaa746079b949c0923e29" Jul 06 17:05:56.426883 ip-10-0-128-22 kubenswrapper[2584]: I0706 17:05:56.426847 2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"314697ee16bb2b5cb9a792a6aca5f4aea50439cceabcaa746079b949c0923e29"} err="failed to get container status \"314697ee16bb2b5cb9a792a6aca5f4aea50439cceabcaa746079b949c0923e29\": rpc error: code = NotFound desc = could not find container \"314697ee16bb2b5cb9a792a6aca5f4aea50439cceabcaa746079b949c0923e29\": container with ID starting with 314697ee16bb2b5cb9a792a6aca5f4aea50439cceabcaa746079b949c0923e29 not found: ID does not exist" Jul 06 17:05:56.441103 ip-10-0-128-22 kubenswrapper[2584]: I0706 17:05:56.441071 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-shared-b-327802-64c5b78555-8rrpz"] Jul 06 17:05:56.446776 ip-10-0-128-22 kubenswrapper[2584]: I0706 17:05:56.446748 2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-shared-b-327802-64c5b78555-8rrpz"] Jul 06 17:05:58.118508 ip-10-0-128-22 kubenswrapper[2584]: I0706 17:05:58.118476 2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e5fa08e9-0846-49ac-9557-a22d4205a3f6" path="/var/lib/kubelet/pods/e5fa08e9-0846-49ac-9557-a22d4205a3f6/volumes" Jul 06 17:06:02.732847 ip-10-0-128-22 kubenswrapper[2584]: I0706 17:06:02.732749 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 17:06:02.799973 ip-10-0-128-22 kubenswrapper[2584]: I0706 17:06:02.799935 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 17:06:03.082680 ip-10-0-128-22 kubenswrapper[2584]: I0706 17:06:03.082578 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 17:06:03.258966 ip-10-0-128-22 kubenswrapper[2584]: I0706 17:06:03.258933 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 17:06:08.742314 ip-10-0-128-22 kubenswrapper[2584]: I0706 17:06:08.742279 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-5f4b5b787d-wfpdm_277e2adb-cbb6-428b-9a13-57bcb090522f/maas-api/0.log" Jul 06 17:06:08.979190 ip-10-0-128-22 kubenswrapper[2584]: I0706 17:06:08.979156 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-key-cleanup-29722620-nfwfw_8f441a6c-d8f9-473b-8faf-e19bb46f55db/cleanup/0.log" Jul 06 17:06:09.496908 ip-10-0-128-22 kubenswrapper[2584]: I0706 17:06:09.496880 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7896b945bc-77h26_639cf8f4-3234-4042-a9f1-2daa5378ba85/copy-manifests/0.log" Jul 06 17:06:09.523237 ip-10-0-128-22 kubenswrapper[2584]: I0706 17:06:09.523204 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7896b945bc-77h26_639cf8f4-3234-4042-a9f1-2daa5378ba85/manager/0.log" Jul 06 17:06:12.027620 ip-10-0-128-22 kubenswrapper[2584]: I0706 17:06:12.027590 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-7hbqb_7a9f90da-048f-424b-a819-591e54a697b7/manager/0.log" Jul 06 17:06:12.835120 ip-10-0-128-22 kubenswrapper[2584]: I0706 17:06:12.835087 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-6b4fs_fd2a40b6-f0b5-4bb2-80d1-d1806c7ade9d/discovery/0.log" Jul 06 17:06:13.397928 ip-10-0-128-22 kubenswrapper[2584]: I0706 17:06:13.397894 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-92twh_6dd63d9e-d113-46dc-a653-e728d115e34a/payload-pre-processing/0.log" Jul 06 17:06:14.179183 ip-10-0-128-22 kubenswrapper[2584]: I0706 17:06:14.179143 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-m69w8_8145c2cb-b892-4f0e-bc2e-0198c0a82136/main/0.log" Jul 06 17:06:14.445335 ip-10-0-128-22 kubenswrapper[2584]: I0706 17:06:14.445294 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbqtx98_e24aae4f-7b71-4aac-b0bb-d80a238f053d/main/0.log" Jul 06 17:06:18.885458 ip-10-0-128-22 kubenswrapper[2584]: I0706 17:06:18.885417 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 17:06:18.970832 ip-10-0-128-22 kubenswrapper[2584]: I0706 17:06:18.970794 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 17:06:22.413263 ip-10-0-128-22 kubenswrapper[2584]: I0706 17:06:22.413232 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-wvcq5_d695ec8b-3624-40aa-afe2-841d320d0a66/global-pull-secret-syncer/0.log" Jul 06 17:06:22.503526 ip-10-0-128-22 kubenswrapper[2584]: I0706 17:06:22.503495 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" Jul 06 17:06:22.660452 ip-10-0-128-22 kubenswrapper[2584]: I0706 17:06:22.660422 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-128-22.ec2.internal_a4d219c131416ebd840bc1507523f196/haproxy/0.log" Jul 06 17:06:27.918694 ip-10-0-128-22 kubenswrapper[2584]: I0706 17:06:27.918663 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-7hbqb_7a9f90da-048f-424b-a819-591e54a697b7/manager/0.log" Jul 06 17:06:29.860598 ip-10-0-128-22 kubenswrapper[2584]: I0706 17:06:29.860558 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_772a7200-5b39-41ff-aff9-db11c2da83da/alertmanager/0.log" Jul 06 17:06:29.893506 ip-10-0-128-22 kubenswrapper[2584]: I0706 17:06:29.893481 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_772a7200-5b39-41ff-aff9-db11c2da83da/config-reloader/0.log" Jul 06 17:06:29.936589 ip-10-0-128-22 kubenswrapper[2584]: I0706 17:06:29.936562 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_772a7200-5b39-41ff-aff9-db11c2da83da/kube-rbac-proxy-web/0.log" Jul 06 17:06:29.967634 ip-10-0-128-22 kubenswrapper[2584]: I0706 17:06:29.967608 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_772a7200-5b39-41ff-aff9-db11c2da83da/kube-rbac-proxy/0.log" Jul 06 17:06:30.000051 ip-10-0-128-22 kubenswrapper[2584]: I0706 17:06:30.000021 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_772a7200-5b39-41ff-aff9-db11c2da83da/kube-rbac-proxy-metric/0.log" Jul 06 17:06:30.031993 ip-10-0-128-22 kubenswrapper[2584]: I0706 17:06:30.031960 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_772a7200-5b39-41ff-aff9-db11c2da83da/prom-label-proxy/0.log" Jul 06 17:06:30.065510 ip-10-0-128-22 kubenswrapper[2584]: I0706 17:06:30.065422 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_772a7200-5b39-41ff-aff9-db11c2da83da/init-config-reloader/0.log" Jul 06 17:06:30.193164 ip-10-0-128-22 kubenswrapper[2584]: I0706 17:06:30.193134 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-7c9dbb8b7d-z5c75_ed031c36-5db2-40a8-87e1-2d21d54e5c18/kube-state-metrics/0.log" Jul 06 17:06:30.225445 ip-10-0-128-22 kubenswrapper[2584]: I0706 17:06:30.225417 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-7c9dbb8b7d-z5c75_ed031c36-5db2-40a8-87e1-2d21d54e5c18/kube-rbac-proxy-main/0.log" Jul 06 17:06:30.253690 ip-10-0-128-22 kubenswrapper[2584]: I0706 17:06:30.253656 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-7c9dbb8b7d-z5c75_ed031c36-5db2-40a8-87e1-2d21d54e5c18/kube-rbac-proxy-self/0.log" Jul 06 17:06:30.292464 ip-10-0-128-22 kubenswrapper[2584]: I0706 17:06:30.292433 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-5b7f5d65f6-wj8m9_4ef11f66-522d-4dc8-99cc-3b883daa2f6a/metrics-server/0.log" Jul 06 17:06:30.335470 ip-10-0-128-22 kubenswrapper[2584]: I0706 17:06:30.335382 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-d5dcff7cf-gg9zd_bfc4049e-f24b-403c-8fad-70c53a6eb323/monitoring-plugin/0.log" Jul 06 17:06:30.382388 ip-10-0-128-22 kubenswrapper[2584]: I0706 17:06:30.382358 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-7pwll_facad3ae-3b4c-4812-93b3-b398a5811e12/node-exporter/0.log" Jul 06 17:06:30.417678 ip-10-0-128-22 kubenswrapper[2584]: I0706 17:06:30.417644 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-7pwll_facad3ae-3b4c-4812-93b3-b398a5811e12/kube-rbac-proxy/0.log" Jul 06 17:06:30.450821 ip-10-0-128-22 kubenswrapper[2584]: I0706 17:06:30.450798 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-7pwll_facad3ae-3b4c-4812-93b3-b398a5811e12/init-textfile/0.log" Jul 06 17:06:30.731208 ip-10-0-128-22 kubenswrapper[2584]: I0706 17:06:30.731177 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-54cf96dfcf-vk9j9_0fbab4dc-9098-4407-924a-20c35bbff2ee/kube-rbac-proxy-main/0.log" Jul 06 17:06:30.765486 ip-10-0-128-22 kubenswrapper[2584]: I0706 17:06:30.765455 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-54cf96dfcf-vk9j9_0fbab4dc-9098-4407-924a-20c35bbff2ee/kube-rbac-proxy-self/0.log" Jul 06 17:06:30.796460 ip-10-0-128-22 kubenswrapper[2584]: I0706 17:06:30.796431 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-54cf96dfcf-vk9j9_0fbab4dc-9098-4407-924a-20c35bbff2ee/openshift-state-metrics/0.log" Jul 06 17:06:30.844791 ip-10-0-128-22 kubenswrapper[2584]: I0706 17:06:30.844754 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_05dd8b1c-17dd-49ce-a9f6-5e4fa25a8605/prometheus/0.log" Jul 06 17:06:30.876704 ip-10-0-128-22 kubenswrapper[2584]: I0706 17:06:30.876676 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_05dd8b1c-17dd-49ce-a9f6-5e4fa25a8605/config-reloader/0.log" Jul 06 17:06:30.910929 ip-10-0-128-22 kubenswrapper[2584]: I0706 17:06:30.910902 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_05dd8b1c-17dd-49ce-a9f6-5e4fa25a8605/thanos-sidecar/0.log" Jul 06 17:06:30.943744 ip-10-0-128-22 kubenswrapper[2584]: I0706 17:06:30.943705 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_05dd8b1c-17dd-49ce-a9f6-5e4fa25a8605/kube-rbac-proxy-web/0.log" Jul 06 17:06:30.972157 ip-10-0-128-22 kubenswrapper[2584]: I0706 17:06:30.972123 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_05dd8b1c-17dd-49ce-a9f6-5e4fa25a8605/kube-rbac-proxy/0.log" Jul 06 17:06:31.002247 ip-10-0-128-22 kubenswrapper[2584]: I0706 17:06:31.002127 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_05dd8b1c-17dd-49ce-a9f6-5e4fa25a8605/kube-rbac-proxy-thanos/0.log" Jul 06 17:06:31.051524 ip-10-0-128-22 kubenswrapper[2584]: I0706 17:06:31.051496 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_05dd8b1c-17dd-49ce-a9f6-5e4fa25a8605/init-config-reloader/0.log" Jul 06 17:06:31.092381 ip-10-0-128-22 kubenswrapper[2584]: I0706 17:06:31.092351 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-567f7b8b7c-hpfd9_d96c7a27-3a8e-4050-affe-c9be14fcbfa4/prometheus-operator/0.log" Jul 06 17:06:31.130776 ip-10-0-128-22 kubenswrapper[2584]: I0706 17:06:31.130747 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-567f7b8b7c-hpfd9_d96c7a27-3a8e-4050-affe-c9be14fcbfa4/kube-rbac-proxy/0.log" Jul 06 17:06:31.241700 ip-10-0-128-22 kubenswrapper[2584]: I0706 17:06:31.241650 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-55dc9fbd88-bkmtv_b223d48a-945b-4d16-ab8e-ffab4a19c40a/telemeter-client/0.log" Jul 06 17:06:31.308059 ip-10-0-128-22 kubenswrapper[2584]: I0706 17:06:31.307946 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-55dc9fbd88-bkmtv_b223d48a-945b-4d16-ab8e-ffab4a19c40a/reload/0.log" Jul 06 17:06:31.370323 ip-10-0-128-22 kubenswrapper[2584]: I0706 17:06:31.370272 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-55dc9fbd88-bkmtv_b223d48a-945b-4d16-ab8e-ffab4a19c40a/kube-rbac-proxy/0.log" Jul 06 17:06:31.432973 ip-10-0-128-22 kubenswrapper[2584]: I0706 17:06:31.432944 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-599bcfcb5b-8wd62_db061724-64a7-4ee4-9021-d01e787b6c9a/thanos-query/0.log" Jul 06 17:06:31.492906 ip-10-0-128-22 kubenswrapper[2584]: I0706 17:06:31.492878 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-599bcfcb5b-8wd62_db061724-64a7-4ee4-9021-d01e787b6c9a/kube-rbac-proxy-web/0.log" Jul 06 17:06:31.554815 ip-10-0-128-22 kubenswrapper[2584]: I0706 17:06:31.554780 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-599bcfcb5b-8wd62_db061724-64a7-4ee4-9021-d01e787b6c9a/kube-rbac-proxy/0.log" Jul 06 17:06:31.613541 ip-10-0-128-22 kubenswrapper[2584]: I0706 17:06:31.613451 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-599bcfcb5b-8wd62_db061724-64a7-4ee4-9021-d01e787b6c9a/prom-label-proxy/0.log" Jul 06 17:06:31.671942 ip-10-0-128-22 kubenswrapper[2584]: I0706 17:06:31.671908 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-599bcfcb5b-8wd62_db061724-64a7-4ee4-9021-d01e787b6c9a/kube-rbac-proxy-rules/0.log" Jul 06 17:06:31.725389 ip-10-0-128-22 kubenswrapper[2584]: I0706 17:06:31.725358 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-599bcfcb5b-8wd62_db061724-64a7-4ee4-9021-d01e787b6c9a/kube-rbac-proxy-metrics/0.log" Jul 06 17:06:31.968121 ip-10-0-128-22 kubenswrapper[2584]: I0706 17:06:31.968085 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-842gk/perf-node-gather-daemonset-8bfq4"] Jul 06 17:06:31.968514 ip-10-0-128-22 kubenswrapper[2584]: I0706 17:06:31.968474 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e5fa08e9-0846-49ac-9557-a22d4205a3f6" containerName="maas-api" Jul 06 17:06:31.968514 ip-10-0-128-22 kubenswrapper[2584]: I0706 17:06:31.968486 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="e5fa08e9-0846-49ac-9557-a22d4205a3f6" containerName="maas-api" Jul 06 17:06:31.968514 ip-10-0-128-22 kubenswrapper[2584]: I0706 17:06:31.968494 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8f441a6c-d8f9-473b-8faf-e19bb46f55db" containerName="cleanup" Jul 06 17:06:31.968514 ip-10-0-128-22 kubenswrapper[2584]: I0706 17:06:31.968501 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="8f441a6c-d8f9-473b-8faf-e19bb46f55db" containerName="cleanup" Jul 06 17:06:31.968514 ip-10-0-128-22 kubenswrapper[2584]: I0706 17:06:31.968510 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="36b4e830-9971-4c2f-8d25-7983611571a1" containerName="cleanup" Jul 06 17:06:31.968514 ip-10-0-128-22 kubenswrapper[2584]: I0706 17:06:31.968516 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="36b4e830-9971-4c2f-8d25-7983611571a1" containerName="cleanup" Jul 06 17:06:31.968809 ip-10-0-128-22 kubenswrapper[2584]: I0706 17:06:31.968528 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f4f0b180-6ec8-475d-860d-cbdbc173b72e" containerName="maas-api" Jul 06 17:06:31.968809 ip-10-0-128-22 kubenswrapper[2584]: I0706 17:06:31.968534 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="f4f0b180-6ec8-475d-860d-cbdbc173b72e" containerName="maas-api" Jul 06 17:06:31.968809 ip-10-0-128-22 kubenswrapper[2584]: I0706 17:06:31.968551 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="46735719-f64e-4413-9b2f-f73ec56e7a45" containerName="cleanup" Jul 06 17:06:31.968809 ip-10-0-128-22 kubenswrapper[2584]: I0706 17:06:31.968558 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="46735719-f64e-4413-9b2f-f73ec56e7a45" containerName="cleanup" Jul 06 17:06:31.968809 ip-10-0-128-22 kubenswrapper[2584]: I0706 17:06:31.968631 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="e5fa08e9-0846-49ac-9557-a22d4205a3f6" containerName="maas-api" Jul 06 17:06:31.968809 ip-10-0-128-22 kubenswrapper[2584]: I0706 17:06:31.968642 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="f4f0b180-6ec8-475d-860d-cbdbc173b72e" containerName="maas-api" Jul 06 17:06:31.968809 ip-10-0-128-22 kubenswrapper[2584]: I0706 17:06:31.968648 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="8f441a6c-d8f9-473b-8faf-e19bb46f55db" containerName="cleanup" Jul 06 17:06:31.968809 ip-10-0-128-22 kubenswrapper[2584]: I0706 17:06:31.968654 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="36b4e830-9971-4c2f-8d25-7983611571a1" containerName="cleanup" Jul 06 17:06:31.968809 ip-10-0-128-22 kubenswrapper[2584]: I0706 17:06:31.968660 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="46735719-f64e-4413-9b2f-f73ec56e7a45" containerName="cleanup" Jul 06 17:06:31.972354 ip-10-0-128-22 kubenswrapper[2584]: I0706 17:06:31.972333 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-842gk/perf-node-gather-daemonset-8bfq4" Jul 06 17:06:31.975135 ip-10-0-128-22 kubenswrapper[2584]: I0706 17:06:31.975114 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-842gk\"/\"default-dockercfg-zbpwj\"" Jul 06 17:06:31.976323 ip-10-0-128-22 kubenswrapper[2584]: I0706 17:06:31.976276 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-842gk\"/\"kube-root-ca.crt\"" Jul 06 17:06:31.976323 ip-10-0-128-22 kubenswrapper[2584]: I0706 17:06:31.976292 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-842gk\"/\"openshift-service-ca.crt\"" Jul 06 17:06:31.982836 ip-10-0-128-22 kubenswrapper[2584]: I0706 17:06:31.982811 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-842gk/perf-node-gather-daemonset-8bfq4"] Jul 06 17:06:32.080737 ip-10-0-128-22 kubenswrapper[2584]: I0706 17:06:32.080701 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/d53b1a1a-f720-4240-bfd5-9a8642683942-proc\") pod \"perf-node-gather-daemonset-8bfq4\" (UID: \"d53b1a1a-f720-4240-bfd5-9a8642683942\") " pod="openshift-must-gather-842gk/perf-node-gather-daemonset-8bfq4" Jul 06 17:06:32.080920 ip-10-0-128-22 kubenswrapper[2584]: I0706 17:06:32.080824 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d53b1a1a-f720-4240-bfd5-9a8642683942-sys\") pod \"perf-node-gather-daemonset-8bfq4\" (UID: \"d53b1a1a-f720-4240-bfd5-9a8642683942\") " pod="openshift-must-gather-842gk/perf-node-gather-daemonset-8bfq4" Jul 06 17:06:32.080920 ip-10-0-128-22 kubenswrapper[2584]: I0706 17:06:32.080855 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jj7dn\" (UniqueName: \"kubernetes.io/projected/d53b1a1a-f720-4240-bfd5-9a8642683942-kube-api-access-jj7dn\") pod \"perf-node-gather-daemonset-8bfq4\" (UID: \"d53b1a1a-f720-4240-bfd5-9a8642683942\") " pod="openshift-must-gather-842gk/perf-node-gather-daemonset-8bfq4" Jul 06 17:06:32.080999 ip-10-0-128-22 kubenswrapper[2584]: I0706 17:06:32.080933 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/d53b1a1a-f720-4240-bfd5-9a8642683942-podres\") pod \"perf-node-gather-daemonset-8bfq4\" (UID: \"d53b1a1a-f720-4240-bfd5-9a8642683942\") " pod="openshift-must-gather-842gk/perf-node-gather-daemonset-8bfq4" Jul 06 17:06:32.080999 ip-10-0-128-22 kubenswrapper[2584]: I0706 17:06:32.080989 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d53b1a1a-f720-4240-bfd5-9a8642683942-lib-modules\") pod \"perf-node-gather-daemonset-8bfq4\" (UID: \"d53b1a1a-f720-4240-bfd5-9a8642683942\") " pod="openshift-must-gather-842gk/perf-node-gather-daemonset-8bfq4" Jul 06 17:06:32.182520 ip-10-0-128-22 kubenswrapper[2584]: I0706 17:06:32.182486 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d53b1a1a-f720-4240-bfd5-9a8642683942-lib-modules\") pod \"perf-node-gather-daemonset-8bfq4\" (UID: \"d53b1a1a-f720-4240-bfd5-9a8642683942\") " pod="openshift-must-gather-842gk/perf-node-gather-daemonset-8bfq4" Jul 06 17:06:32.182704 ip-10-0-128-22 kubenswrapper[2584]: I0706 17:06:32.182682 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d53b1a1a-f720-4240-bfd5-9a8642683942-lib-modules\") pod \"perf-node-gather-daemonset-8bfq4\" (UID: \"d53b1a1a-f720-4240-bfd5-9a8642683942\") " pod="openshift-must-gather-842gk/perf-node-gather-daemonset-8bfq4" Jul 06 17:06:32.182787 ip-10-0-128-22 kubenswrapper[2584]: I0706 17:06:32.182771 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/d53b1a1a-f720-4240-bfd5-9a8642683942-proc\") pod \"perf-node-gather-daemonset-8bfq4\" (UID: \"d53b1a1a-f720-4240-bfd5-9a8642683942\") " pod="openshift-must-gather-842gk/perf-node-gather-daemonset-8bfq4" Jul 06 17:06:32.182934 ip-10-0-128-22 kubenswrapper[2584]: I0706 17:06:32.182911 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/d53b1a1a-f720-4240-bfd5-9a8642683942-proc\") pod \"perf-node-gather-daemonset-8bfq4\" (UID: \"d53b1a1a-f720-4240-bfd5-9a8642683942\") " pod="openshift-must-gather-842gk/perf-node-gather-daemonset-8bfq4" Jul 06 17:06:32.183018 ip-10-0-128-22 kubenswrapper[2584]: I0706 17:06:32.182958 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d53b1a1a-f720-4240-bfd5-9a8642683942-sys\") pod \"perf-node-gather-daemonset-8bfq4\" (UID: \"d53b1a1a-f720-4240-bfd5-9a8642683942\") " pod="openshift-must-gather-842gk/perf-node-gather-daemonset-8bfq4" Jul 06 17:06:32.183018 ip-10-0-128-22 kubenswrapper[2584]: I0706 17:06:32.182992 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jj7dn\" (UniqueName: \"kubernetes.io/projected/d53b1a1a-f720-4240-bfd5-9a8642683942-kube-api-access-jj7dn\") pod \"perf-node-gather-daemonset-8bfq4\" (UID: \"d53b1a1a-f720-4240-bfd5-9a8642683942\") " pod="openshift-must-gather-842gk/perf-node-gather-daemonset-8bfq4" Jul 06 17:06:32.183131 ip-10-0-128-22 kubenswrapper[2584]: I0706 17:06:32.183043 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/d53b1a1a-f720-4240-bfd5-9a8642683942-podres\") pod \"perf-node-gather-daemonset-8bfq4\" (UID: \"d53b1a1a-f720-4240-bfd5-9a8642683942\") " pod="openshift-must-gather-842gk/perf-node-gather-daemonset-8bfq4" Jul 06 17:06:32.183228 ip-10-0-128-22 kubenswrapper[2584]: I0706 17:06:32.183212 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/d53b1a1a-f720-4240-bfd5-9a8642683942-podres\") pod \"perf-node-gather-daemonset-8bfq4\" (UID: \"d53b1a1a-f720-4240-bfd5-9a8642683942\") " pod="openshift-must-gather-842gk/perf-node-gather-daemonset-8bfq4" Jul 06 17:06:32.183717 ip-10-0-128-22 kubenswrapper[2584]: I0706 17:06:32.183691 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d53b1a1a-f720-4240-bfd5-9a8642683942-sys\") pod \"perf-node-gather-daemonset-8bfq4\" (UID: \"d53b1a1a-f720-4240-bfd5-9a8642683942\") " pod="openshift-must-gather-842gk/perf-node-gather-daemonset-8bfq4" Jul 06 17:06:32.193538 ip-10-0-128-22 kubenswrapper[2584]: I0706 17:06:32.193501 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jj7dn\" (UniqueName: \"kubernetes.io/projected/d53b1a1a-f720-4240-bfd5-9a8642683942-kube-api-access-jj7dn\") pod \"perf-node-gather-daemonset-8bfq4\" (UID: \"d53b1a1a-f720-4240-bfd5-9a8642683942\") " pod="openshift-must-gather-842gk/perf-node-gather-daemonset-8bfq4" Jul 06 17:06:32.283176 ip-10-0-128-22 kubenswrapper[2584]: I0706 17:06:32.283085 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-842gk/perf-node-gather-daemonset-8bfq4" Jul 06 17:06:32.436514 ip-10-0-128-22 kubenswrapper[2584]: I0706 17:06:32.436487 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-842gk/perf-node-gather-daemonset-8bfq4"] Jul 06 17:06:32.438012 ip-10-0-128-22 kubenswrapper[2584]: W0706 17:06:32.437979 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podd53b1a1a_f720_4240_bfd5_9a8642683942.slice/crio-85dce513e5b5e1d7fa7d78bc7ef03a642ab0aafa87f03d0f022294fa42c2017b WatchSource:0}: Error finding container 85dce513e5b5e1d7fa7d78bc7ef03a642ab0aafa87f03d0f022294fa42c2017b: Status 404 returned error can't find the container with id 85dce513e5b5e1d7fa7d78bc7ef03a642ab0aafa87f03d0f022294fa42c2017b Jul 06 17:06:32.597264 ip-10-0-128-22 kubenswrapper[2584]: I0706 17:06:32.597229 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-842gk/perf-node-gather-daemonset-8bfq4" event={"ID":"d53b1a1a-f720-4240-bfd5-9a8642683942","Type":"ContainerStarted","Data":"275e6921dbd294ad5bebf34641fc651f877add829dc08c99f154bd27a5ffb67f"} Jul 06 17:06:32.597455 ip-10-0-128-22 kubenswrapper[2584]: I0706 17:06:32.597291 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-842gk/perf-node-gather-daemonset-8bfq4" event={"ID":"d53b1a1a-f720-4240-bfd5-9a8642683942","Type":"ContainerStarted","Data":"85dce513e5b5e1d7fa7d78bc7ef03a642ab0aafa87f03d0f022294fa42c2017b"} Jul 06 17:06:32.597455 ip-10-0-128-22 kubenswrapper[2584]: I0706 17:06:32.597366 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-842gk/perf-node-gather-daemonset-8bfq4" Jul 06 17:06:32.623160 ip-10-0-128-22 kubenswrapper[2584]: I0706 17:06:32.623106 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-842gk/perf-node-gather-daemonset-8bfq4" podStartSLOduration=1.62308815 podStartE2EDuration="1.62308815s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 17:06:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 17:06:32.621797499 +0000 UTC m=+2543.235937620" watchObservedRunningTime="2026-07-06 17:06:32.62308815 +0000 UTC m=+2543.237228270" Jul 06 17:06:35.816887 ip-10-0-128-22 kubenswrapper[2584]: I0706 17:06:35.816863 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-t9nd7_61652b90-d462-4c62-8c64-6e0843e2ebd5/dns/0.log" Jul 06 17:06:35.840943 ip-10-0-128-22 kubenswrapper[2584]: I0706 17:06:35.840900 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-t9nd7_61652b90-d462-4c62-8c64-6e0843e2ebd5/kube-rbac-proxy/0.log" Jul 06 17:06:35.928420 ip-10-0-128-22 kubenswrapper[2584]: I0706 17:06:35.928389 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-wg98x_4c862983-24dd-471d-928b-aa352b202fd9/dns-node-resolver/0.log" Jul 06 17:06:36.606639 ip-10-0-128-22 kubenswrapper[2584]: I0706 17:06:36.606609 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-27rkm_e98b6fdb-3725-4c37-8a97-6a59fbc25f76/node-ca/0.log" Jul 06 17:06:37.829918 ip-10-0-128-22 kubenswrapper[2584]: I0706 17:06:37.829882 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-6b4fs_fd2a40b6-f0b5-4bb2-80d1-d1806c7ade9d/discovery/0.log" Jul 06 17:06:38.014965 ip-10-0-128-22 kubenswrapper[2584]: I0706 17:06:38.014930 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-92twh_6dd63d9e-d113-46dc-a653-e728d115e34a/payload-pre-processing/0.log" Jul 06 17:06:38.611988 ip-10-0-128-22 kubenswrapper[2584]: I0706 17:06:38.611958 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-842gk/perf-node-gather-daemonset-8bfq4" Jul 06 17:06:38.834158 ip-10-0-128-22 kubenswrapper[2584]: I0706 17:06:38.834129 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-wj5s2_0ab9e759-73b4-4f75-b1ab-276f94c7ec1e/serve-healthcheck-canary/0.log" Jul 06 17:06:39.368449 ip-10-0-128-22 kubenswrapper[2584]: I0706 17:06:39.368420 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-pdz7v_ffbbbede-1d14-4088-853c-02b22a61e00a/kube-rbac-proxy/0.log" Jul 06 17:06:39.395065 ip-10-0-128-22 kubenswrapper[2584]: I0706 17:06:39.395019 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-pdz7v_ffbbbede-1d14-4088-853c-02b22a61e00a/exporter/0.log" Jul 06 17:06:39.423080 ip-10-0-128-22 kubenswrapper[2584]: I0706 17:06:39.423044 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-pdz7v_ffbbbede-1d14-4088-853c-02b22a61e00a/extractor/0.log" Jul 06 17:06:41.503981 ip-10-0-128-22 kubenswrapper[2584]: I0706 17:06:41.503944 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k4gbb_9bf128db-4ed4-483d-a886-cb334e6fa4ef/konnectivity-agent/0.log" limit=1024 Jul 06 17:06:42.583687 ip-10-0-128-22 kubenswrapper[2584]: I0706 17:06:42.583656 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-5f4b5b787d-wfpdm_277e2adb-cbb6-428b-9a13-57bcb090522f/maas-api/0.log" Jul 06 17:06:42.595789 ip-10-0-128-22 kubenswrapper[2584]: I0706 17:06:42.595762 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-6b4fs_fd2a40b6-f0b5-4bb2-80d1-d1806c7ade9d/discovery/0.log" Jul 06 17:06:42.679937 ip-10-0-128-22 kubenswrapper[2584]: I0706 17:06:42.679885 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-key-cleanup-29722620-nfwfw_8f441a6c-d8f9-473b-8faf-e19bb46f55db/cleanup/0.log" Jul 06 17:06:42.773634 ip-10-0-128-22 kubenswrapper[2584]: I0706 17:06:42.773574 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-92twh_6dd63d9e-d113-46dc-a653-e728d115e34a/payload-pre-processing/0.log" Jul 06 17:06:43.092302 ip-10-0-128-22 kubenswrapper[2584]: I0706 17:06:43.092246 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7896b945bc-77h26_639cf8f4-3234-4042-a9f1-2daa5378ba85/manager/0.log" Jul 06 17:06:43.168923 ip-10-0-128-22 kubenswrapper[2584]: I0706 17:06:43.168896 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7896b945bc-77h26_639cf8f4-3234-4042-a9f1-2daa5378ba85/copy-manifests/0.log" Jul 06 17:06:44.370283 ip-10-0-128-22 kubenswrapper[2584]: I0706 17:06:44.370250 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-6b4fs_fd2a40b6-f0b5-4bb2-80d1-d1806c7ade9d/discovery/0.log" Jul 06 17:06:44.746862 ip-10-0-128-22 kubenswrapper[2584]: I0706 17:06:44.746800 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-d4b66856f-hkvth_0a38af7a-6fe5-4c26-8dc2-6f1be1104699/manager/0.log" Jul 06 17:06:44.773896 ip-10-0-128-22 kubenswrapper[2584]: I0706 17:06:44.773862 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-92twh_6dd63d9e-d113-46dc-a653-e728d115e34a/payload-pre-processing/0.log"